lexer: Change the pipeline to allow more flexible use of macros.
authorBen Pfaff <blp@cs.stanford.edu>
Sun, 18 Jul 2021 21:21:24 +0000 (14:21 -0700)
committerBen Pfaff <blp@cs.stanford.edu>
Sun, 18 Jul 2021 21:23:18 +0000 (14:23 -0700)
commit3d5011bbd45f2bd802caf55211bc060ed777a321
tree308ea6037e2b58d0c9dc7782abb9477b03c37282
parent61d50bb6faca2b65483ee1eac6a214a5a1263a13
lexer: Change the pipeline to allow more flexible use of macros.

Frans Houweling reported that a use of macros in the following way:

    DEFINE !dir() "/directory/to/my/work" !ENDDEFINE.
    GET FILE=!dir + "/filename.sav".

did not work properly with the newly implemented PSPP macro facility.
Indeed, PSPP has until now implemented string concatenation early in the
lexical pipeline, before macro expansion, so that the above could
not work.  This commit reworks it so that string concatenation happens
as the last stage in lexical analysis.  It allows the above syntax
to work as expected.
13 files changed:
src/language/control/define.c
src/language/lexer/lexer.c
src/language/lexer/macro.c
src/language/lexer/macro.h
src/language/lexer/scan.c
src/language/lexer/scan.h
src/language/lexer/token.h
src/libpspp/message.c
src/libpspp/message.h
tests/language/control/define.at
tests/language/lexer/lexer.at
tests/language/lexer/scan-test.c
tests/language/lexer/scan.at