From: Ben Pfaff <blp@cs.stanford.edu>
Date: Sat, 6 Oct 2018 22:12:07 +0000 (-0700)
Subject: lexer: Fix details of journaling.
X-Git-Url: https://pintos-os.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=35c08c1615757db98c03c3947c151ea3b2179f71;p=pspp

lexer: Fix details of journaling.

This fixes test 1075: ASCII driver syntax printback.

Reported by John Darrington and Jeremy Lavergne.
Fixes: b4f585e70eca ("lexer: Fix buffer overread in journaling when source does not end in \n.")
Fixes: e0f9210e814d ("lexer: Add support for embedded \0 bytes and missing trailing new-line.")
---

diff --git a/src/language/lexer/lexer.c b/src/language/lexer/lexer.c
index 98cf8ec298..36680f52bc 100644
--- a/src/language/lexer/lexer.c
+++ b/src/language/lexer/lexer.c
@@ -1417,8 +1417,13 @@ lex_source_get__ (const struct lex_source *src_)
       /* Beginning of line. */
       const char *line = &src->buffer[src->journal_pos - src->tail];
 
-      /* Calculate line length, including \n or \r\n end-of-line if present. */
-      size_t max_len = state.line_pos - src->journal_pos;
+      /* Calculate line length, including \n or \r\n end-of-line if present.
+
+         We use src->head even though that may be beyond what we've actually
+         converted to tokens (which is only through state.line_pos).  That's
+         because, if we're emitting the line due to SEG_END_COMMAND, we want to
+         take the whole line through the newline, not just through the '.'. */
+      size_t max_len = src->head - src->journal_pos;
       const char *newline = memchr (line, '\n', max_len);
       size_t line_len = newline ? newline - line + 1 : max_len;