From 880924c2c10557f211e71af79d014cd1fd26607a Mon Sep 17 00:00:00 2001 From: Ben Pfaff Date: Fri, 8 Jan 2021 23:02:23 -0800 Subject: [PATCH] spv-light-decoder: Add back character set encoding support. Originally, SPV light member decoding would obtain the member's declared character set encoding and recode all strings from that encoding into UTF-8. The SPV file submitted along with bug #59837, however, showed that SPV files actually contain UTF-8 strings despite their declared character encoding. This SPV file declared windows-1253 encoding. It contained 221 unique strings, 52 of which contained non-ASCII characters, and all of which were valid UTF-8. Therefore, commit db9a44802bb9 ("spv-light-decoder: Text strings are all UTF-8 encoded.") changed the SPV light member decoder to treat all strings as UTF-8 encoded. Unfortunately, further examination of the SPV corpus showed that there is no consistency. Some files do contain all UTF-8 despite declaring another character set. For example: 00764b1c.spv: windows-1251: 481 unique strings, 123 non-ASCII, 123 UTF-8. 009bf8ba.spv: windows-1252: 76 unique strings, 9 non-ASCII, 9 UTF-8. 00b033b6.spv: windows-1252: 389 unique strings, 16 non-ASCII, 16 UTF-8. 014fc3df.spv: ISO_8859-1:1987: 81 unique strings, 4 non-ASCII, 4 UTF-8. 01a20a32.spv: windows-1254: 71 unique strings, 10 non-ASCII, 10 UTF-8. 01d41135.spv: windows-1251: 142 unique strings, 51 non-ASCII, 51 UTF-8. 01d7942a.spv: windows-1251: 64 unique strings, 43 non-ASCII, 0 UTF-8. 0203e88a.spv: windows-1254: 82 unique strings, 4 non-ASCII, 0 UTF-8. 0247cf5a.spv: windows-1256: 236 unique strings, 5 non-ASCII, 0 UTF-8. 026777ed.spv: 027b66dd.spv: windows-1252: 224 unique strings, 2 non-ASCII, 0 UTF-8. 02cc8c22.spv: windows-1254: 79 unique strings, 1 non-ASCII, 1 UTF-8. ... Others are entirely non-UTF-8: 00029fbf.spv: windows-1251: 115 unique strings, 88 non-ASCII, 0 UTF-8. 00f101f5.spv: windows-1252: 94 unique strings, 1 non-ASCII, 0 UTF-8. 00ff0628.spv: windows-1252: 112 unique strings, 14 non-ASCII, 0 UTF-8. 01d7942a.spv: windows-1251: 64 unique strings, 43 non-ASCII, 0 UTF-8. 0203e88a.spv: windows-1254: 82 unique strings, 4 non-ASCII, 0 UTF-8. 0247cf5a.spv: windows-1256: 236 unique strings, 5 non-ASCII, 0 UTF-8. 027b66dd.spv: windows-1252: 224 unique strings, 2 non-ASCII, 0 UTF-8. 03235aa7.spv: windows-1254: 198 unique strings, 18 non-ASCII, 0 UTF-8. 07a43c5c.spv: ISO-8859-15: 124 unique strings, 13 non-ASCII, 0 UTF-8. 07a85498.spv: windows-1254: 86 unique strings, 1 non-ASCII, 0 UTF-8. 07a91f3e.spv: windows-1252: 111 unique strings, 13 non-ASCII, 0 UTF-8. 0ad295d8.spv: windows-1252: 81 unique strings, 1 non-ASCII, 0 UTF-8. 0ceb843b.spv: windows-1252: 3108 unique strings, 392 non-ASCII, 0 UTF-8. and still others are a mix: 02f274e6.spv: windows-1252: 746 unique strings, 77 non-ASCII, 27 UTF-8. 0a7be05b.spv: windows-1255: 334 unique strings, 122 non-ASCII, 113 UTF-8. 4c7a575d.spv: windows-1250: 400 unique strings, 73 non-ASCII, 72 UTF-8. 785b0737.spv: windows-1250: 322 unique strings, 164 non-ASCII, 158 UTF-8. 94365e08.spv: windows-1250: 353 unique strings, 77 non-ASCII, 2 UTF-8. This commit just gives up and interprets any string that is valid UTF-8 as UTF-8. So far, it works in practice. --- doc/dev/spv-file-format.texi | 57 +++++++++-- src/output/spv/spv-light-decoder.c | 158 +++++++++++++++++------------ 2 files changed, 143 insertions(+), 72 deletions(-) diff --git a/doc/dev/spv-file-format.texi b/doc/dev/spv-file-format.texi index b27037cce7..723550a372 100644 --- a/doc/dev/spv-file-format.texi +++ b/doc/dev/spv-file-format.texi @@ -921,8 +921,8 @@ A 32-bit IEEE floating-point number. @item string @itemx bestring A 32-bit unsigned integer, in little-endian or big-endian byte order, -respectively, followed by the specified number of bytes of UTF-8 -encoded character data. +respectively, followed by the specified number of bytes of character +data. (The encoding is indicated by the Formats nonterminal.) @item @var{x}? @var{x} is optional, e.g.@: 00? is an optional zero byte. @@ -1365,8 +1365,8 @@ If @code{n-widths} is nonzero, then the accompanying integers are column widths as manually adjusted by the user. @code{locale} is a locale including an encoding, such as -@code{en_US.windows-1252} or @code{it_IT.windows-1252}. The encoding -string (like other strings in the member) is encoded in UTF-8. +@code{en_US.windows-1252} or @code{it_IT.windows-1252}. +(@code{locale} is often duplicated in Y1, described below). @code{epoch} is the year that starts the epoch. A 2-digit year is interpreted as belonging to the 100 years beginning at the epoch. The @@ -1408,7 +1408,7 @@ output, in English. It is not necessarily the literal syntax name of the procedure: for example, NPAR TESTS becomes ``Nonparametric Tests.'' @code{command-local} is the procedure's name, translated into the output language; it is often empty and, when it is not, -sometimes the same as @code{command}.q +sometimes the same as @code{command}. @code{missing} is the character used to indicate that a cell contains a missing value. It is always observed as @samp{.}. @@ -1435,9 +1435,7 @@ X1 => @code{lang} may indicate the language in use. Some values seem to be 0: @t{en}, 1: @t{de}, 2: @t{es}, 3: @t{it}, 5: @t{ko}, 6: @t{pl}, 8: -@t{zh-tw}, 10: @t{pt_BR}, 11: @t{fr}. The @code{locale} in Formats -and the @code{language}, @code{charset}, and @code{locale} in X0 are -more likely to be useful in practice. +@t{zh-tw}, 10: @t{pt_BR}, 11: @t{fr}. @code{show-variables} determines how variables are displayed by default. A value of 1 means to display variable names, 2 to display @@ -1526,6 +1524,49 @@ will). A writer may safely use 4 for @code{x21} and omit @code{x22} and the other optional bytes at the end. +@subsubheading Encoding + +Formats contains several indications of character encoding: + +@itemize @bullet +@item +@code{locale} in Formats itself. + +@item +@code{locale} in Y1 (in version 1, Y1 is optionally nested inside X0; +in version 3, Y1 is nested inside X3). + +@item +@code{charset} in version 3, in Y1. + +@item +@code{lang} in X1, in version 3. +@end itemize + +@code{charset}, if present, is a good indication of character +encoding, and in its absence the encoding suffix on @code{locale} in +Formats will work. + +@code{locale} in Y1 can be disregarded: it is normally the same as +@code{locale} in Formats, and it is only present if @code{charset} is +also. + +@code{lang} is not helpful and should be ignored for character +encoding purposes. + +However, the corpus contains many examples of light members whose +strings are encoded in UTF-8 despite declaring some other character +set. Furthermore, the corpus contains several examples of light +members in which some strings are encoded in UTF-8 (and contain +multibyte characters) and other strings are encoded in another +character set (and contain non-ASCII characters). PSPP treats any +valid UTF-8 string as UTF-8 and only falls back to the declared +encoding for strings that are not valid UTF-8. + +The @command{pspp-output} program's @command{strings} command can help +analyze the encoding in an SPV light member. Use @code{pspp-output +--help-dev} to see its usage. + @node SPV Light Member Dimensions @subsection Dimensions diff --git a/src/output/spv/spv-light-decoder.c b/src/output/spv/spv-light-decoder.c index cbf2aacdab..20f7afa11b 100644 --- a/src/output/spv/spv-light-decoder.c +++ b/src/output/spv/spv-light-decoder.c @@ -22,6 +22,7 @@ #include #include #include +#include #include "libpspp/i18n.h" #include "libpspp/message.h" @@ -33,10 +34,21 @@ #include "gl/xalloc.h" #include "gl/xsize.h" +/* Returns a copy of S converted to UTF-8. S might be in UTF-8 already or it + might be in ENCODING (yes, this makes no sense). */ static char * -xstrdup_if_nonempty (const char *s) +to_utf8 (const char *s, const char *encoding) { - return s && s[0] ? xstrdup (s) : NULL; + size_t length = strlen (s); + return (u8_check (CHAR_CAST (const uint8_t *, s), length) + ? recode_string ("UTF-8", encoding, s, length) + : xstrdup (s)); +} + +static char * +to_utf8_if_nonempty (const char *s, const char *encoding) +{ + return s && s[0] ? to_utf8 (s, encoding) : NULL; } static void @@ -102,7 +114,7 @@ decode_spvlb_color_u32 (uint32_t x) static char * WARN_UNUSED_RESULT decode_spvlb_font_style (const struct spvlb_font_style *in, - struct font_style **outp) + const char *encoding, struct font_style **outp) { if (!in) { @@ -124,7 +136,7 @@ decode_spvlb_font_style (const struct spvlb_font_style *in, .underline = in->underline, .fg = { fg, fg }, .bg = { bg, bg }, - .typeface = xstrdup (in->typeface), + .typeface = to_utf8 (in->typeface, encoding), .size = in->size / 1.33, }; return NULL; @@ -220,17 +232,17 @@ decode_spvlb_cell_style (const struct spvlb_cell_style *in, static char *decode_spvlb_value ( const struct pivot_table *, const struct spvlb_value *, - struct pivot_value **) WARN_UNUSED_RESULT; + const char *encoding, struct pivot_value **) WARN_UNUSED_RESULT; static char * WARN_UNUSED_RESULT decode_spvlb_argument (const struct pivot_table *table, const struct spvlb_argument *in, - struct pivot_argument *out) + const char *encoding, struct pivot_argument *out) { if (in->value) { struct pivot_value *value; - char *error = decode_spvlb_value (table, in->value, &value); + char *error = decode_spvlb_value (table, in->value, encoding, &value); if (error) return error; @@ -244,7 +256,7 @@ decode_spvlb_argument (const struct pivot_table *table, out->values = xnmalloc (in->n_values, sizeof *out->values); for (size_t i = 0; i < in->n_values; i++) { - char *error = decode_spvlb_value (table, in->values[i], + char *error = decode_spvlb_value (table, in->values[i], encoding, &out->values[i]); if (error) { @@ -274,7 +286,8 @@ decode_spvlb_value_show (uint8_t in, enum settings_value_show *out) static char * WARN_UNUSED_RESULT decode_spvlb_value (const struct pivot_table *table, - const struct spvlb_value *in, struct pivot_value **outp) + const struct spvlb_value *in, + const char *encoding, struct pivot_value **outp) { *outp = NULL; @@ -302,16 +315,18 @@ decode_spvlb_value (const struct pivot_table *table, error = decode_spvlb_value_show (in->type_02.show, &out->numeric.show); if (error) return NULL; - out->numeric.var_name = xstrdup_if_nonempty (in->type_02.var_name); - out->numeric.value_label = xstrdup_if_nonempty (in->type_02.value_label); + out->numeric.var_name = to_utf8_if_nonempty (in->type_02.var_name, + encoding); + out->numeric.value_label = to_utf8_if_nonempty (in->type_02.value_label, + encoding); break; case 3: vm = in->type_03.value_mod; out->type = PIVOT_VALUE_TEXT; - out->text.local = xstrdup (in->type_03.local); - out->text.c = xstrdup (in->type_03.c); - out->text.id = xstrdup (in->type_03.id); + out->text.local = to_utf8 (in->type_03.local, encoding); + out->text.c = to_utf8 (in->type_03.c, encoding); + out->text.id = to_utf8 (in->type_03.id, encoding); out->text.user_provided = !in->type_03.fixed; break; @@ -321,10 +336,11 @@ decode_spvlb_value (const struct pivot_table *table, error = decode_spvlb_value_show (in->type_04.show, &out->string.show); if (error) return NULL; - out->string.s = xstrdup (in->type_04.s); + out->string.s = to_utf8 (in->type_04.s, encoding); out->string.hex = (in->type_04.format >> 16) == fmt_to_io (FMT_AHEX); - out->string.var_name = xstrdup (in->type_04.var_name); - out->string.value_label = xstrdup_if_nonempty (in->type_04.value_label); + out->string.var_name = to_utf8 (in->type_04.var_name, encoding); + out->string.value_label = to_utf8_if_nonempty (in->type_04.value_label, + encoding); break; case 5: @@ -333,23 +349,24 @@ decode_spvlb_value (const struct pivot_table *table, error = decode_spvlb_value_show (in->type_05.show, &out->variable.show); if (error) return error; - out->variable.var_name = xstrdup (in->type_05.var_name); - out->variable.var_label = xstrdup_if_nonempty (in->type_05.var_label); + out->variable.var_name = to_utf8 (in->type_05.var_name, encoding); + out->variable.var_label = to_utf8_if_nonempty (in->type_05.var_label, + encoding); break; case 6: vm = in->type_06.value_mod; out->type = PIVOT_VALUE_TEXT; - out->text.local = xstrdup (in->type_06.local); - out->text.c = xstrdup (in->type_06.c); - out->text.id = xstrdup (in->type_06.id); + out->text.local = to_utf8 (in->type_06.local, encoding); + out->text.c = to_utf8 (in->type_06.c, encoding); + out->text.id = to_utf8 (in->type_06.id, encoding); out->text.user_provided = false; break; case -1: vm = in->type_else.value_mod; out->type = PIVOT_VALUE_TEMPLATE; - out->template.local = xstrdup (in->type_else.template); + out->template.local = to_utf8 (in->type_else.template, encoding); out->template.id = out->template.local; out->template.n_args = 0; out->template.args = xnmalloc (in->type_else.n_args, @@ -357,7 +374,7 @@ decode_spvlb_value (const struct pivot_table *table, for (size_t i = 0; i < in->type_else.n_args; i++) { error = decode_spvlb_argument (table, in->type_else.args[i], - &out->template.args[i]); + encoding, &out->template.args[i]); if (error) { pivot_value_destroy (out); @@ -379,7 +396,7 @@ decode_spvlb_value (const struct pivot_table *table, out->subscripts = xnmalloc (vm->n_subscripts, sizeof *out->subscripts); for (size_t i = 0; i < vm->n_subscripts; i++) - out->subscripts[i] = xstrdup (vm->subscripts[i]); + out->subscripts[i] = to_utf8 (vm->subscripts[i], encoding); } if (vm->n_refs) @@ -403,7 +420,7 @@ decode_spvlb_value (const struct pivot_table *table, if (vm->style_pair) { error = decode_spvlb_font_style (vm->style_pair->font_style, - &out->font_style); + encoding, &out->font_style); if (!error) error = decode_spvlb_cell_style (vm->style_pair->cell_style, &out->cell_style); @@ -418,7 +435,7 @@ decode_spvlb_value (const struct pivot_table *table, && vm->template_string->id && vm->template_string->id[0] && out->type == PIVOT_VALUE_TEMPLATE) - out->template.id = xstrdup (vm->template_string->id); + out->template.id = to_utf8 (vm->template_string->id, encoding); } *outp = out; @@ -426,7 +443,8 @@ decode_spvlb_value (const struct pivot_table *table, } static char * WARN_UNUSED_RESULT -decode_spvlb_area (const struct spvlb_area *in, struct table_area_style *out) +decode_spvlb_area (const struct spvlb_area *in, struct table_area_style *out, + const char *encoding) { char *error; @@ -466,7 +484,7 @@ decode_spvlb_area (const struct spvlb_area *in, struct table_area_style *out) .underline = in->underline, .fg = { fg0, in->alternate ? fg1 : fg0 }, .bg = { bg0, in->alternate ? bg1 : bg0 }, - .typeface = xstrdup (in->typeface), + .typeface = to_utf8 (in->typeface, encoding), .size = in->size / 1.33, }, .cell_style = { @@ -487,14 +505,16 @@ decode_spvlb_group (const struct pivot_table *, size_t n_categories, bool show_label, struct pivot_category *parent, - struct pivot_dimension *); + struct pivot_dimension *, + const char *encoding); static char * WARN_UNUSED_RESULT decode_spvlb_categories (const struct pivot_table *table, struct spvlb_category **categories, size_t n_categories, struct pivot_category *parent, - struct pivot_dimension *dimension) + struct pivot_dimension *dimension, + const char *encoding) { for (size_t i = 0; i < n_categories; i++) { @@ -503,7 +523,7 @@ decode_spvlb_categories (const struct pivot_table *table, { char *error = decode_spvlb_categories ( table, in->group->subcategories, in->group->n_subcategories, - parent, dimension); + parent, dimension, encoding); if (error) return error; @@ -511,7 +531,7 @@ decode_spvlb_categories (const struct pivot_table *table, } struct pivot_value *name; - char *error = decode_spvlb_value (table, in->name, &name); + char *error = decode_spvlb_value (table, in->name, encoding, &name); if (error) return error; @@ -523,7 +543,7 @@ decode_spvlb_categories (const struct pivot_table *table, { char *error = decode_spvlb_group (table, in->group->subcategories, in->group->n_subcategories, - true, out, dimension); + true, out, dimension, encoding); if (error) { pivot_category_destroy (out); @@ -553,7 +573,8 @@ decode_spvlb_group (const struct pivot_table *table, struct spvlb_category **categories, size_t n_categories, bool show_label, struct pivot_category *category, - struct pivot_dimension *dimension) + struct pivot_dimension *dimension, + const char *encoding) { category->subs = XCALLOC (n_categories, struct pivot_category *); category->n_subs = 0; @@ -561,7 +582,7 @@ decode_spvlb_group (const struct pivot_table *table, category->show_label = show_label; return decode_spvlb_categories (table, categories, n_categories, category, - dimension); + dimension, encoding); } static char * WARN_UNUSED_RESULT @@ -594,11 +615,12 @@ fill_leaves (struct pivot_category *category, static char * WARN_UNUSED_RESULT decode_spvlb_dimension (const struct pivot_table *table, const struct spvlb_dimension *in, - size_t idx, struct pivot_dimension **outp) + size_t idx, const char *encoding, + struct pivot_dimension **outp) { /* Convert most of the dimension. */ struct pivot_value *name; - char *error = decode_spvlb_value (table, in->name, &name); + char *error = decode_spvlb_value (table, in->name, encoding, &name); if (error) return error; @@ -616,7 +638,7 @@ decode_spvlb_dimension (const struct pivot_table *table, }; error = decode_spvlb_group (table, in->categories, in->n_categories, !in->props->hide_dim_label, out->root, - out); + out, encoding); if (error) goto error; @@ -726,7 +748,7 @@ decode_data_index (uint64_t in, const struct pivot_table *table, static char * WARN_UNUSED_RESULT decode_spvlb_cells (struct spvlb_cell **in, size_t n_in, - struct pivot_table *table) + struct pivot_table *table, const char *encoding) { if (!table->n_dimensions) return NULL; @@ -737,7 +759,7 @@ decode_spvlb_cells (struct spvlb_cell **in, size_t n_in, struct pivot_value *value; char *error = decode_data_index (in[i]->index, table, dindexes); if (!error) - error = decode_spvlb_value (table, in[i]->value, &value); + error = decode_spvlb_value (table, in[i]->value, encoding, &value); if (error) { free (dindexes); @@ -751,18 +773,18 @@ decode_spvlb_cells (struct spvlb_cell **in, size_t n_in, } static char * WARN_UNUSED_RESULT -decode_spvlb_footnote (const struct spvlb_footnote *in, +decode_spvlb_footnote (const struct spvlb_footnote *in, const char *encoding, size_t idx, struct pivot_table *table) { struct pivot_value *content; - char *error = decode_spvlb_value (table, in->text, &content); + char *error = decode_spvlb_value (table, in->text, encoding, &content); if (error) return error; struct pivot_value *marker = NULL; if (in->marker) { - error = decode_spvlb_value (table, in->marker, &marker); + error = decode_spvlb_value (table, in->marker, encoding, &marker); if (error) { pivot_value_destroy (content); @@ -816,6 +838,11 @@ decode_spvlb_table (const struct spvlb_table *in, struct pivot_table **outp) out->look = pivot_table_look_new_builtin_default (); out->settings = (struct fmt_settings) FMT_SETTINGS_INIT; + const struct spvlb_y1 *y1 = (in->formats->x0 ? in->formats->x0->y1 + : in->formats->x3 ? in->formats->x3->y1 + : NULL); + const char *encoding = spvlb_table_get_encoding (in); + /* Display settings. */ out->look->show_numeric_markers = !in->ts->show_alphabetic_markers; out->rotate_inner_column_labels = in->header->rotate_inner_column_labels; @@ -871,8 +898,8 @@ decode_spvlb_table (const struct spvlb_table *in, struct pivot_table **outp) &out->sizing[TABLE_HORZ].keeps, &out->sizing[TABLE_HORZ].n_keeps); - out->notes = xstrdup_if_nonempty (in->ts->notes); - out->look->name = xstrdup_if_nonempty (in->ts->table_look); + out->notes = to_utf8_if_nonempty (in->ts->notes, encoding); + out->look->name = to_utf8_if_nonempty (in->ts->table_look, encoding); /* Print settings. */ out->look->print_all_layers = in->ps->all_layers; @@ -881,7 +908,7 @@ decode_spvlb_table (const struct spvlb_table *in, struct pivot_table **outp) out->look->shrink_to_fit[TABLE_VERT] = in->ps->fit_length; out->look->top_continuation = in->ps->top_continuation; out->look->bottom_continuation = in->ps->bottom_continuation; - out->look->continuation = xstrdup (in->ps->continuation_string); + out->look->continuation = to_utf8 (in->ps->continuation_string, encoding); out->look->n_orphan_lines = in->ps->n_orphan_lines; /* Format settings. */ @@ -908,16 +935,13 @@ decode_spvlb_table (const struct spvlb_table *in, struct pivot_table **outp) out->small = in->formats->x3 ? in->formats->x3->small : 0; /* Command information. */ - const struct spvlb_y1 *y1 = (in->formats->x0 ? in->formats->x0->y1 - : in->formats->x3 ? in->formats->x3->y1 - : NULL); if (y1) { - out->command_local = xstrdup (y1->command_local); - out->command_c = xstrdup (y1->command); - out->language = xstrdup (y1->language); + out->command_local = to_utf8 (y1->command_local, encoding); + out->command_c = to_utf8 (y1->command, encoding); + out->language = to_utf8 (y1->language, encoding); /* charset? */ - out->locale = xstrdup (y1->locale); + out->locale = to_utf8 (y1->locale, encoding); } /* Source information. */ @@ -925,8 +949,8 @@ decode_spvlb_table (const struct spvlb_table *in, struct pivot_table **outp) if (x3) { if (x3->dataset && x3->dataset[0] && x3->dataset[0] != 4) - out->dataset = xstrdup (x3->dataset); - out->datafile = xstrdup_if_nonempty (x3->datafile); + out->dataset = to_utf8 (x3->dataset, encoding); + out->datafile = to_utf8_if_nonempty (x3->datafile, encoding); out->date = x3->date; } @@ -948,32 +972,36 @@ decode_spvlb_table (const struct spvlb_table *in, struct pivot_table **outp) pivot_table_create_footnote__ (out, fn->n_footnotes - 1, NULL, NULL); for (size_t i = 0; i < fn->n_footnotes; i++) { - error = decode_spvlb_footnote (in->footnotes->footnotes[i], i, out); + error = decode_spvlb_footnote (in->footnotes->footnotes[i], + encoding, i, out); if (error) goto error; } } /* Title and caption. */ - error = decode_spvlb_value (out, in->titles->user_title, &out->title); + error = decode_spvlb_value (out, in->titles->user_title, encoding, + &out->title); if (error) goto error; - error = decode_spvlb_value (out, in->titles->subtype, &out->subtype); + error = decode_spvlb_value (out, in->titles->subtype, encoding, + &out->subtype); if (error) goto error; if (in->titles->corner_text) { error = decode_spvlb_value (out, in->titles->corner_text, - &out->corner_text); + encoding, &out->corner_text); if (error) goto error; } if (in->titles->caption) { - error = decode_spvlb_value (out, in->titles->caption, &out->caption); + error = decode_spvlb_value (out, in->titles->caption, encoding, + &out->caption); if (error) goto error; } @@ -982,7 +1010,8 @@ decode_spvlb_table (const struct spvlb_table *in, struct pivot_table **outp) /* Styles. */ for (size_t i = 0; i < PIVOT_N_AREAS; i++) { - error = decode_spvlb_area (in->areas->areas[i], &out->look->areas[i]); + error = decode_spvlb_area (in->areas->areas[i], &out->look->areas[i], + encoding); if (error) goto error; } @@ -999,7 +1028,7 @@ decode_spvlb_table (const struct spvlb_table *in, struct pivot_table **outp) for (size_t i = 0; i < out->n_dimensions; i++) { error = decode_spvlb_dimension (out, in->dimensions->dims[i], - i, &out->dimensions[i]); + i, encoding, &out->dimensions[i]); if (error) goto error; } @@ -1035,7 +1064,8 @@ decode_spvlb_table (const struct spvlb_table *in, struct pivot_table **outp) goto error; /* Data. */ - error = decode_spvlb_cells (in->cells->cells, in->cells->n_cells, out); + error = decode_spvlb_cells (in->cells->cells, in->cells->n_cells, out, + encoding); *outp = out; return NULL; -- 2.30.2