data-parser: Treat " ," the same as ",".

The data parser (used e.g. by DATA LIST) was unintentionally treating
soft separators (e.g. white space) followed by a hard separator (e.g.
a comma) as an empty field, but in fact it should be treated as just a
single separator.  This fixes it and adds a test.

Thanks to pascal barbedor <pbarbedor@gmail.com> for reporting the problem.

diff --git a/src/language/data-io/data-parser.c b/src/language/data-io/data-parser.c
index dfc04be44c52b2cfc5f09a2b2dfc9432018ca10d..a3a438b70fc63a909f474569f24a36dd35741fcb 100644 (file)
--- a/src/language/data-io/data-parser.c
+++ b/src/language/data-io/data-parser.c
@@ -480,7 +480,8 @@ cut_field (const struct data_parser *parser, struct dfm_reader *reader,
       /* Regular field. */
       ss_get_chars (&p, ss_cspan (p, ds_ss (&parser->any_sep)), field);
       *last_column = dfm_column_start (reader);
-      if (!ss_ltrim (&p, parser->soft_seps) || ss_is_empty (p))
+      if (!ss_ltrim (&p, parser->soft_seps) || ss_is_empty (p)
+          || ss_find_char (parser->hard_seps, p.string[0]) != SIZE_MAX)
         {
           /* Advance past a trailing hard separator,
              regardless of whether one actually existed.  If

diff --git a/tests/command/data-list.sh b/tests/command/data-list.sh
index 7d3890c3430bf5fe34c22467cfcc9338bf4ff6f7..3f821199ff3e2d10f22627e6ce35c93ec7b1f157 100755 (executable)
--- a/tests/command/data-list.sh
+++ b/tests/command/data-list.sh
@@ -76,7 +76,7 @@ begin data.
 6
 7,
 8 9
-0,1,,,
+0,1 ,,,
 ,,,,
 2