From: Ben Pfaff Date: Sun, 22 Dec 2013 07:33:57 +0000 (-0800) Subject: pspp-convert: Add support for decrypting encrypted system files. X-Git-Url: https://pintos-os.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=01f35481fe6f80ee35c79bf873bc69af11331217;p=pspp pspp-convert: Add support for decrypting encrypted system files. --- diff --git a/NEWS b/NEWS index 911c47e092..5b74b1d19a 100644 --- a/NEWS +++ b/NEWS @@ -9,9 +9,16 @@ Changes from 0.8.1 to 0.8.1.1: * Charts are now rendered with colours from the Tango palette instead of fully saturated primaries. - * PSPP can now read and write ZCOMPRESSED system files, a new format - variant that compresses data much more effectively than the - previous form of compression (which is still supported). + * Support for new system file variants: + + - PSPP can now read and write ZCOMPRESSED system files, which + compress data much more effectively than older "compressed" + files. (The older format is still supported.) + + - PSPP can now decrypt encrypted system files, using the new + pspp-convert utility. The encrypted system file format is + unacceptably insecure, so to discourage its use PSPP and PSPPIRE + do not directly read or write this format. * Missing values for long string variables are now read from and written to system files in an SPSS-compatible fashion. @@ -23,8 +30,8 @@ Changes from 0.8.1 to 0.8.1.1: * pspp-convert, a new standalone utility for converting SPSS system and portable files to other formats, is now included. The initial - version is only capable of converting to comma-separated value - files.. + version supports comma-separated value files as output format. + pspp-convert can also decrypt encrypted system files. * Build changes: diff --git a/Smake b/Smake index e976e46379..5a312e616b 100644 --- a/Smake +++ b/Smake @@ -20,6 +20,7 @@ GNULIB_MODULES = \ crc \ crypto/md4 \ crypto/md5 \ + crypto/rijndael \ dirname \ dtoastr \ environ \ @@ -33,6 +34,7 @@ GNULIB_MODULES = \ ftello \ fwriteerror \ getline \ + getpass \ gettext \ gettimeofday \ getopt-gnu \ diff --git a/doc/dev/system-file-format.texi b/doc/dev/system-file-format.texi index 89c35aab3f..0da1997b3f 100644 --- a/doc/dev/system-file-format.texi +++ b/doc/dev/system-file-format.texi @@ -120,6 +120,7 @@ Each type of record is described separately below. * Miscellaneous Informational Records:: * Dictionary Termination Record:: * Data Record:: +* Encrypted System Files:: @end menu @node File Header Record @@ -1321,10 +1322,10 @@ VARIABLE ATTRIBUTE VARIABLES=dummy ATTRIBUTE=bert('123'). will contain a variable attribute record with the following contents: @example -00000000 07 00 00 00 12 00 00 00 01 00 00 00 22 00 00 00 |............"...| -00000010 64 75 6d 6d 79 3a 66 72 65 64 28 27 32 33 27 0a |dummy:fred('23'.| -00000020 27 33 34 27 0a 29 62 65 72 74 28 27 31 32 33 27 |'34'.)bert('123'| -00000030 0a 29 |.) | +0000 07 00 00 00 12 00 00 00 01 00 00 00 22 00 00 00 |............"...| +0010 64 75 6d 6d 79 3a 66 72 65 64 28 27 32 33 27 0a |dummy:fred('23'.| +0020 27 33 34 27 0a 29 62 65 72 74 28 27 31 32 33 27 |'34'.)bert('123'| +0030 0a 29 |.) | @end example @menu @@ -1632,3 +1633,165 @@ system file. @end table @setfilename ignored + +@node Encrypted System Files +@section Encrypted System Files + +SPSS 21 and later support an encrypted system file format. + +@quotation Warning +The SPSS encrypted file format is poorly designed. It is much cheaper +and faster to decrypt a file encrypted this way than if a well +designed alternative were used. If you must use this format, use a +10-byte randomly generated password. +@end quotation + +@subheading Encrypted File Format + +Encrypted system files begin with the following 36-byte fixed header: + +@example +0000 1c 00 00 00 00 00 00 00 45 4e 43 52 59 50 54 45 |........ENCRYPTE| +0010 44 53 41 56 15 00 00 00 00 00 00 00 00 00 00 00 |DSAV............| +0020 00 00 00 00 |....| +@end example + +Following the fixed header is a complete system file in the usual +format, except that each 16-byte block is encrypted with AES-256 in +ECB mode. The AES-256 key is derived from a password in the following +way: + +@enumerate +@item +Start from the literal password typed by the user. Truncate it to at +most 10 bytes, then append (between 1 and 22) null bytes until there +are exactly 32 bytes. Call this @var{password}. + +@item +Let @var{constant} be the following 73-byte constant: + +@example +0000 00 00 00 01 35 27 13 cc 53 a7 78 89 87 53 22 11 +0010 d6 5b 31 58 dc fe 2e 7e 94 da 2f 00 cc 15 71 80 +0020 0a 6c 63 53 00 38 c3 38 ac 22 f3 63 62 0e ce 85 +0030 3f b8 07 4c 4e 2b 77 c7 21 f5 1a 80 1d 67 fb e1 +0040 e1 83 07 d8 0d 00 00 01 00 +@end example + +@item +Compute CMAC-AES-256(@var{password}, @var{constant}). Call the +16-byte result @var{cmac}. + +@item +The 32-byte AES-256 key is @var{cmac} || @var{cmac}, that is, +@var{cmac} repeated twice. +@end enumerate + +@subsubheading Example + +Consider the password @samp{pspp}. @var{password} is: + +@example +0000 70 73 70 70 00 00 00 00 00 00 00 00 00 00 00 00 |pspp............| +0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| +@end example + +@noindent +@var{cmac} is: + +@example +0000 3e da 09 8e 66 04 d4 fd f9 63 0c 2c a8 6f b0 45 +@end example + +@noindent +The AES-256 key is: + +@example +0000 3e da 09 8e 66 04 d4 fd f9 63 0c 2c a8 6f b0 45 +0010 3e da 09 8e 66 04 d4 fd f9 63 0c 2c a8 6f b0 45 +@end example + +@subheading Password Encoding + +SPSS also supports what it calls ``encrypted passwords.'' These are +not encrypted. They are encoded with a simple, fixed scheme. An +encoded password is always a multiple of 2 characters long, and never +longer than 20 characters. The characters in an encoded password are +always in the graphic ASCII range 33 through 126. Each successive +pair of characters in the password encodes a single byte in the +plaintext password. + +Use the following algorithm to decode a pair of characters: + +@enumerate +@item +Let @var{a} be the ASCII code of the first character, and @var{b} be +the ASCII code of the second character. + +@item +Let @var{ah} be the most significant 4 bits of @var{a}. Find the line +in the table below that has @var{ah} on the left side. The right side +of the line is a set of possible values for the most significant 4 +bits of the decoded byte. + +@display +@t{2 } @result{} @t{2367} +@t{3 } @result{} @t{0145} +@t{47} @result{} @t{89cd} +@t{56} @result{} @t{abef} +@end display + +@item +Let @var{bh} be the most significant 4 bits of @var{b}. Find the line +in the second table below that has @var{bh} on the left side. The +right side of the line is a set of possible values for the most +significant 4 bits of the decoded byte. Together with the results of +the previous step, only a single possibility is left. + +@display +@t{2 } @result{} @t{139b} +@t{3 } @result{} @t{028a} +@t{47} @result{} @t{46ce} +@t{56} @result{} @t{57df} +@end display + +@item +Let @var{al} be the least significant 4 bits of @var{a}. Find the +line in the table below that has @var{al} on the left side. The right +side of the line is a set of possible values for the least significant +4 bits of the decoded byte. + +@display +@t{03cf} @result{} @t{0145} +@t{12de} @result{} @t{2367} +@t{478b} @result{} @t{89cd} +@t{569a} @result{} @t{abef} +@end display + +@item +Let @var{bl} be the least significant 4 bits of @var{b}. Find the +line in the table below that has @var{bl} on the left side. The right +side of the line is a set of possible values for the least significant +4 bits of the decoded byte. Together with the results of the previous +step, only a single possibility is left. + +@display +@t{03cf} @result{} @t{028a} +@t{12de} @result{} @t{139b} +@t{478b} @result{} @t{46ce} +@t{569a} @result{} @t{57df} +@end display +@end enumerate + +@subsubheading Example + +Consider the encoded character pair @samp{-|}. @var{a} is +0x2d and @var{b} is 0x7c, so @var{ah} is 2, @var{bh} is 7, @var{al} is +0xd, and @var{bl} is 0xc. @var{ah} means that the most significant +four bits of the decoded character is 2, 3, 6, or 7, and @var{bh} +means that they are 4, 6, 0xc, or 0xe. The single possibility in +common is 6, so the most significant four bits are 6. Similarly, +@var{al} means that the least significant four bits are 2, 3, 6, or 7, +and @var{bl} means they are 0, 2, 8, or 0xa, so the least significant +four bits are 2. The decoded character is therefore 0x62, the letter +@samp{b}. diff --git a/src/data/automake.mk b/src/data/automake.mk index 9ce405bb65..5f2120c038 100644 --- a/src/data/automake.mk +++ b/src/data/automake.mk @@ -101,6 +101,8 @@ src_data_libdata_la_SOURCES = \ src/data/subcase.c \ src/data/subcase.h \ src/data/sys-file-encoding.c \ + src/data/sys-file-encryption.c \ + src/data/sys-file-encryption.h \ src/data/sys-file-private.c \ src/data/sys-file-private.h \ src/data/sys-file-reader.c \ diff --git a/src/data/sys-file-encryption.c b/src/data/sys-file-encryption.c new file mode 100644 index 0000000000..fa4e1b9309 --- /dev/null +++ b/src/data/sys-file-encryption.c @@ -0,0 +1,356 @@ +/* PSPP - a program for statistical analysis. + Copyright (C) 2013 Free Software Foundation, Inc. + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . */ + +#include + +#include "data/sys-file-encryption.h" + +#include +#include + +#include "data/file-name.h" +#include "libpspp/assertion.h" +#include "libpspp/cast.h" +#include "libpspp/cmac-aes256.h" +#include "libpspp/message.h" + +#include "gl/minmax.h" +#include "gl/rijndael-alg-fst.h" +#include "gl/xalloc.h" + +#include "gettext.h" +#define _(msgid) gettext (msgid) + +struct encrypted_sys_file + { + FILE *file; + int error; + + uint8_t ciphertext[16]; + uint8_t plaintext[16]; + unsigned int n; + + uint32_t rk[4 * (RIJNDAEL_MAXNR + 1)]; + int Nr; + }; + +static bool try_password(struct encrypted_sys_file *, const char *password); +static bool decode_password (const char *input, char output[11]); +static bool fill_buffer (struct encrypted_sys_file *); + +/* If FILENAME names an encrypted system file, returns 1 and initializes *FP + for further use by the caller. + + If FILENAME can be opened and read, but is not an encrypted system file, + returns 0. + + If FILENAME cannot be open or read, returns a negative errno value. */ +int +encrypted_sys_file_open (struct encrypted_sys_file **fp, const char *filename) +{ + struct encrypted_sys_file *f; + char header[36 + 16]; + int retval; + int n; + + f = xmalloc (sizeof *f); + f->error = 0; + f->file = fn_open (filename, "rb"); + if (f->file == NULL) + { + msg (ME, _("An error occurred while opening `%s': %s."), + filename, strerror (errno)); + retval = -errno; + goto error; + } + + n = fread (header, 1, sizeof header, f->file); + if (n != sizeof header) + { + int error = feof (f->file) ? 0 : errno; + if (error) + msg (ME, _("An error occurred while reading `%s': %s."), + filename, strerror (error)); + retval = -error; + goto error; + } + + if (memcmp (header + 8, "ENCRYPTEDSAV", 12)) + { + retval = 0; + goto error; + } + + memcpy (f->ciphertext, header + 36, 16); + f->n = 16; + *fp = f; + return 1; + +error: + if (f->file) + fclose (f->file); + free (f); + *fp = NULL; + + return retval; +} + +/* Attempts to use PASSWORD, which may be a plaintext or "encrypted" password, + to unlock F. Returns true if successful, otherwise false. */ +bool +encrypted_sys_file_unlock (struct encrypted_sys_file *f, const char *password) +{ + char decoded_password[11]; + + return (try_password (f, password) + || (decode_password (password, decoded_password) + && try_password (f, decoded_password))); +} + +/* Attempts to read N bytes of plaintext from F into BUF. Returns the number + of bytes successfully read. A return value less than N may indicate end of + file or an error; use encrypted_sys_file_close() to distinguish. + + This function can only be used after encrypted_sys_file_unlock() returns + true. */ +size_t +encrypted_sys_file_read (struct encrypted_sys_file *f, void *buf_, size_t n) +{ + uint8_t *buf = buf_; + size_t ofs = 0; + + if (f->error) + return 0; + + while (ofs < n) + { + unsigned int chunk = MIN (n - ofs, f->n); + if (chunk > 0) + { + memcpy (buf + ofs, &f->plaintext[16 - f->n], chunk); + ofs += chunk; + f->n -= chunk; + } + else + { + if (!fill_buffer (f)) + return ofs; + } + } + + return ofs; +} + +/* Closes F. Returns 0 if no read errors occurred, otherwise a positive errno + value. */ +int +encrypted_sys_file_close (struct encrypted_sys_file *f) +{ + int error = f->error; + if (fclose (f->file) == EOF && !error) + error = errno; + free (f); + + return error; +} + +#define b(x) (1 << (x)) + +static const uint16_t m0[4][2] = { + { b(2), b(2) | b(3) | b(6) | b(7) }, + { b(3), b(0) | b(1) | b(4) | b(5) }, + { b(4) | b(7), b(8) | b(9) | b(12) | b(14) }, + { b(5) | b(6), b(10) | b(11) | b(14) | b(15) }, +}; + +static const uint16_t m1[4][2] = { + { b(0) | b(3) | b(12) | b(15), b(0) | b(1) | b(4) | b(5) }, + { b(1) | b(2) | b(13) | b(14), b(2) | b(3) | b(6) | b(7) }, + { b(4) | b(7) | b(8) | b(11), b(8) | b(9) | b(12) | b(13) }, + { b(5) | b(6) | b(9) | b(10), b(10) | b(11) | b(14) | b(15) }, +}; + +static const uint16_t m2[4][2] = { + { b(2), b(1) | b(3) | b(9) | b(11) }, + { b(3), b(0) | b(2) | b(8) | b(10) }, + { b(4) | b(7), b(4) | b(6) | b(12) | b(14) }, + { b(5) | b(6), b(5) | b(7) | b(13) | b(15) }, +}; + +static const uint16_t m3[4][2] = { + { b(0) | b(3) | b(12) | b(15), b(0) | b(2) | b(8) | b(10) }, + { b(1) | b(2) | b(13) | b(14), b(1) | b(3) | b(9) | b(11) }, + { b(4) | b(7) | b(8) | b(11), b(4) | b(6) | b(12) | b(14) }, + { b(5) | b(6) | b(9) | b(10), b(5) | b(7) | b(13) | b(15) }, +}; + +static int +decode_nibble (const uint16_t table[4][2], int nibble) +{ + int i; + + for (i = 0; i < 4; i++) + if (table[i][0] & (1 << nibble)) + return table[i][1]; + + return 0; +} + +/* Returns true if X has exactly one 1-bit, false otherwise. */ +static bool +is_pow2 (int x) +{ + return x && (x & (x - 1)) == 0; +} + +/* If X has exactly one 1-bit, returns its index, where bit 0 is the LSB. + Otherwise, returns 0. */ +static int +find_1bit (uint16_t x) +{ + int i; + + if (!is_pow2 (x)) + return -1; + + for (i = 0; i < 16; i++) + if (x & (1u << i)) + return i; + + abort (); +} + +/* Attempts to decode a pair of encoded password characters A and B into a + single byte of the plaintext password. Returns 0 if A and B are not a valid + encoded password pair, otherwise a byte of the plaintext password. */ +static int +decode_password_2bytes (uint8_t a, uint8_t b) +{ + int x = find_1bit (decode_nibble (m0, a >> 4) & decode_nibble (m2, b >> 4)); + int y = find_1bit (decode_nibble (m1, a & 15) & decode_nibble (m3, b & 15)); + return x < 0 || y < 0 ? 0 : (x << 4) | y; +} + +/* Decodes an SPSS so-called "encrypted" password INPUT into OUTPUT. + + An encoded password is always an even number of bytes long and no longer + than 20 bytes. A decoded password is never longer than 10 bytes plus a null + terminator. + + Returns true if successful, otherwise false. */ +static bool +decode_password (const char *input, char output[11]) +{ + size_t len; + + len = strlen (input); + if (len > 20 || len % 2) + return false; + + for (; *input; input += 2) + { + int c = decode_password_2bytes (input[0], input[1]); + if (!c) + return false; + *output++ = c; + } + *output = '\0'; + + return true; +} + +/* If CIPHERTEXT is the first ciphertext block in an encrypted .sav file for + PASSWORD, initializes rk[] and returns an nonzero Nr value. + + Otherwise, returns zero. */ +static bool +try_password(struct encrypted_sys_file *f, const char *password) +{ + /* NIST SP 800-108 fixed data. */ + static const uint8_t fixed[] = { + /* i */ + 0x00, 0x00, 0x00, 0x01, + + /* label */ + 0x35, 0x27, 0x13, 0xcc, 0x53, 0xa7, 0x78, 0x89, + 0x87, 0x53, 0x22, 0x11, 0xd6, 0x5b, 0x31, 0x58, + 0xdc, 0xfe, 0x2e, 0x7e, 0x94, 0xda, 0x2f, 0x00, + 0xcc, 0x15, 0x71, 0x80, 0x0a, 0x6c, 0x63, 0x53, + + /* delimiter */ + 0x00, + + /* context */ + 0x38, 0xc3, 0x38, 0xac, 0x22, 0xf3, 0x63, 0x62, + 0x0e, 0xce, 0x85, 0x3f, 0xb8, 0x07, 0x4c, 0x4e, + 0x2b, 0x77, 0xc7, 0x21, 0xf5, 0x1a, 0x80, 0x1d, + 0x67, 0xfb, 0xe1, 0xe1, 0x83, 0x07, 0xd8, 0x0d, + + /* L */ + 0x00, 0x00, 0x01, 0x00, + }; + + char padded_password[32]; + size_t password_len; + uint8_t cmac[16]; + uint8_t key[32]; + + /* Truncate password to at most 10 bytes. */ + password_len = strlen (password); + if (password_len > 10) + password_len = 10; + + /* padded_password = password padded with zeros to 32 bytes. */ + memset (padded_password, 0, sizeof padded_password); + memcpy (padded_password, password, password_len); + + /* cmac = CMAC(padded_password, fixed). */ + cmac_aes256 (CHAR_CAST (const uint8_t *, padded_password), + fixed, sizeof fixed, cmac); + + /* The key is the cmac repeated twice. */ + memcpy(key, cmac, 16); + memcpy(key + 16, cmac, 16); + + /* Use key to initialize AES. */ + assert (sizeof key == 32); + f->Nr = rijndaelKeySetupDec (f->rk, CHAR_CAST (const char *, key), 256); + + /* Check for magic number "$FL" always present in SPSS .sav file. */ + rijndaelDecrypt (f->rk, f->Nr, + CHAR_CAST (const char *, f->ciphertext), + CHAR_CAST (char *, f->plaintext)); + return !memcmp (f->plaintext, "$FL", 3); +} + +static bool +fill_buffer (struct encrypted_sys_file *f) +{ + f->n = fread (f->ciphertext, 1, sizeof f->ciphertext, f->file); + if (f->n == sizeof f->ciphertext) + { + rijndaelDecrypt (f->rk, f->Nr, + CHAR_CAST (const char *, f->ciphertext), + CHAR_CAST (char *, f->plaintext)); + return true; + } + else + { + if (ferror (f->file)) + f->error = errno; + return false; + } +} diff --git a/src/data/sys-file-encryption.h b/src/data/sys-file-encryption.h new file mode 100644 index 0000000000..b406cb2644 --- /dev/null +++ b/src/data/sys-file-encryption.h @@ -0,0 +1,34 @@ +/* PSPP - a program for statistical analysis. + Copyright (C) 2013 Free Software Foundation, Inc. + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . */ + +#ifndef SYS_FILE_ENCRYPTION_H +#define SYS_FILE_ENCRYPTION_H 1 + +#include +#include + +/* Reading encrypted system files. */ + +struct encrypted_sys_file; + +int encrypted_sys_file_open (struct encrypted_sys_file **, + const char *filename); +bool encrypted_sys_file_unlock (struct encrypted_sys_file *, + const char *password); +size_t encrypted_sys_file_read (struct encrypted_sys_file *, void *, size_t); +int encrypted_sys_file_close (struct encrypted_sys_file *); + +#endif /* sys-file-encryption.h */ diff --git a/src/libpspp/automake.mk b/src/libpspp/automake.mk index 3f29e6d584..4d791bf708 100644 --- a/src/libpspp/automake.mk +++ b/src/libpspp/automake.mk @@ -15,6 +15,8 @@ src_libpspp_liblibpspp_la_SOURCES = \ src/libpspp/bt.c \ src/libpspp/bt.h \ src/libpspp/cast.h \ + src/libpspp/cmac-aes256.c \ + src/libpspp/cmac-aes256.h \ src/libpspp/compiler.h \ src/libpspp/copyleft.c \ src/libpspp/copyleft.h \ diff --git a/src/libpspp/cmac-aes256.c b/src/libpspp/cmac-aes256.c new file mode 100644 index 0000000000..fb8ea165fa --- /dev/null +++ b/src/libpspp/cmac-aes256.c @@ -0,0 +1,87 @@ +/* PSPP - a program for statistical analysis. + Copyright (C) 2013 Free Software Foundation, Inc. + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . */ + +#include + +#include + +#include "libpspp/cmac-aes256.h" +#include "libpspp/cast.h" + +#include "gl/rijndael-alg-fst.h" + +static void +gen_subkey (const uint8_t in[16], uint8_t out[16]) +{ + size_t i; + + for (i = 0; i < 15; i++) + out[i] = (in[i] << 1) | (in[i + 1] >> 7); + out[15] = in[15] << 1; + + if (in[0] & 0x80) + out[15] ^= 0x87; +} + +/* Computes CMAC-AES-256 of the SIZE bytes in DATA, using the 256-bit AES key + KEY. Stores the result in the 128-bit CMAC. */ +void +cmac_aes256(const uint8_t key[32], + const void *data_, size_t size, + uint8_t cmac[16]) +{ + const char zeros[16] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 }; + uint32_t rk[4 * RIJNDAEL_MAXNR + 1]; + uint8_t k1[16], k2[16], L[16]; + const uint8_t *data = data_; + uint8_t c[16], tmp[16]; + int Nr; + int i; + + Nr = rijndaelKeySetupEnc (rk, CHAR_CAST (const char *, key), 256); + + rijndaelEncrypt (rk, Nr, zeros, CHAR_CAST (char *, L)); + gen_subkey (L, k1); + gen_subkey (k1, k2); + + memset (c, 0, 16); + while (size > 16) + { + for (i = 0; i < 16; i++) + tmp[i] = c[i] ^ data[i]; + rijndaelEncrypt (rk, Nr, CHAR_CAST (const char *, tmp), + CHAR_CAST (char *, c)); + + size -= 16; + data += 16; + } + + if (size == 16) + { + for (i = 0; i < 16; i++) + tmp[i] = c[i] ^ data[i] ^ k1[i]; + } + else + { + for (i = 0; i < 16; i++) + tmp[i] = c[i] ^ k2[i]; + for (i = 0; i < size; i++) + tmp[i] ^= data[i]; + tmp[size] ^= 0x80; + } + rijndaelEncrypt (rk, Nr, CHAR_CAST (const char *, tmp), + CHAR_CAST (char *, cmac)); +} diff --git a/src/libpspp/cmac-aes256.h b/src/libpspp/cmac-aes256.h new file mode 100644 index 0000000000..41f3829e98 --- /dev/null +++ b/src/libpspp/cmac-aes256.h @@ -0,0 +1,27 @@ +/* PSPP - a program for statistical analysis. + Copyright (C) 2013 Free Software Foundation, Inc. + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . */ + +#ifndef CMAC_AES256_H +#define CMAC_AES256_H 1 + +#include +#include + +void cmac_aes256(const uint8_t key[32], + const void *data, size_t size, + uint8_t cmac[16]); + +#endif /* libpspp/cmac-aes256.h */ diff --git a/tests/automake.mk b/tests/automake.mk index c104ecca47..fe00456ba3 100644 --- a/tests/automake.mk +++ b/tests/automake.mk @@ -9,6 +9,7 @@ check_PROGRAMS += \ tests/language/lexer/segment-test \ tests/libpspp/abt-test \ tests/libpspp/bt-test \ + tests/libpspp/cmac-aes256-test \ tests/libpspp/encoding-guesser-test \ tests/libpspp/heap-test \ tests/libpspp/hmap-test \ @@ -93,6 +94,11 @@ tests_libpspp_bt_test_SOURCES = \ tests/libpspp/bt-test.c tests_libpspp_bt_test_CPPFLAGS = $(AM_CPPFLAGS) -DASSERT_LEVEL=10 +tests_libpspp_cmac_aes256_test_SOURCES = \ + src/libpspp/cmac-aes256.c \ + tests/libpspp/cmac-aes256-test.c +tests_libpspp_cmac_aes256_test_CPPFLAGS = $(AM_CPPFLAGS) -DASSERT_LEVEL=10 + tests_libpspp_range_map_test_SOURCES = \ src/libpspp/bt.c \ src/libpspp/range-map.c \ @@ -246,6 +252,7 @@ TESTSUITE_AT = \ tests/data/por-file.at \ tests/data/sys-file-reader.at \ tests/data/sys-file.at \ + tests/data/sys-file-encryption.at \ tests/language/command.at \ tests/language/control/do-if.at \ tests/language/control/do-repeat.at \ @@ -358,7 +365,7 @@ TESTSUITE_AT = \ TESTSUITE = $(srcdir)/tests/testsuite DISTCLEANFILES += tests/atconfig tests/atlocal $(TESTSUITE) -AUTOTEST_PATH = tests/data:tests/language/lexer:tests/libpspp:tests/output:src/ui/terminal +AUTOTEST_PATH = tests/data:tests/language/lexer:tests/libpspp:tests/output:src/ui/terminal:utilities $(srcdir)/tests/testsuite.at: tests/testsuite.in tests/automake.mk cp $< $@ diff --git a/tests/data/hotel-encrypted.sav b/tests/data/hotel-encrypted.sav new file mode 100644 index 0000000000..2d9f531102 Binary files /dev/null and b/tests/data/hotel-encrypted.sav differ diff --git a/tests/data/sys-file-encryption.at b/tests/data/sys-file-encryption.at new file mode 100644 index 0000000000..337595d3ee --- /dev/null +++ b/tests/data/sys-file-encryption.at @@ -0,0 +1,27 @@ +AT_BANNER([system file encryption]) + +AT_SETUP([decrypt an encrypted system file]) +AT_KEYWORDS([system file decrypt pspp-convert]) +AT_CHECK([pspp-convert $srcdir/data/hotel-encrypted.sav hotel.sav -p pspp]) +AT_CHECK([pspp-convert hotel.sav hotel.csv]) +AT_CHECK([cat hotel.csv], [0], [dnl +v1,v2,v3,v4,v5 +4,2,3,4,1 +1,1,3,1,1 +5,2,2,3,4 +3,1,3,1,2 +5,3,1,5,3 +1,2,5,4,2 +3,2,4,3,1 +1,4,5,2,1 +3,2,3,1,2 +2,5,4,2,1 +4,2,2,3,5 +2,1,4,1,1 +1,2,5,5,2 +2,3,3,3,1 +4,1,1,1,3 +1,1,5,1,2 +2,5,5,2,2 +]) +AT_CLEANUP diff --git a/tests/libpspp/cmac-aes256-test.c b/tests/libpspp/cmac-aes256-test.c new file mode 100644 index 0000000000..9673d4031f --- /dev/null +++ b/tests/libpspp/cmac-aes256-test.c @@ -0,0 +1,114 @@ +/* PSPP - a program for statistical analysis. + Copyright (C) 2013 Free Software Foundation, Inc. + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . */ + +#include + +#include + +#include "libpspp/cmac-aes256.h" + +#undef NDEBUG +#include + +static void +test_cmac (const uint8_t key[32], size_t key_size, + const uint8_t *data, size_t data_size, + const uint8_t *exp_cmac, size_t exp_cmac_size) +{ + uint8_t cmac[16]; + + assert (key_size == 32); + assert (exp_cmac_size <= 16); + + cmac_aes256 (key, data, data_size, cmac); + assert (!memcmp (cmac, exp_cmac, exp_cmac_size)); +} + +/* From NIST CMAC test vectors. */ +static void +test_cmac1 (void) +{ + static const uint8_t key[] = { + 0x0b,0x12,0x2a,0xc8, 0xf3,0x4e,0xd1,0xfe, + 0x08,0x2a,0x36,0x25, 0xd1,0x57,0x56,0x14, + 0x54,0x16,0x7a,0xc1, 0x45,0xa1,0x0b,0xbf, + 0x77,0xc6,0xa7,0x05, 0x96,0xd5,0x74,0xf1 + }; + static const uint8_t data[] = { + 0x49,0x8b,0x53,0xfd, 0xec,0x87,0xed,0xcb, + 0xf0,0x70,0x97,0xdc, 0xcd,0xe9,0x3a,0x08, + 0x4b,0xad,0x75,0x01, 0xa2,0x24,0xe3,0x88, + 0xdf,0x34,0x9c,0xe1, 0x89,0x59,0xfe,0x84, + 0x85,0xf8,0xad,0x15, 0x37,0xf0,0xd8,0x96, + 0xea,0x73,0xbe,0xdc, 0x72,0x14,0x71,0x3f, + }; + static const uint8_t exp_cmac[] = { 0xf6,0x2c,0x46,0x32, 0x9b }; + + test_cmac (key, sizeof key, + data, sizeof data, + exp_cmac, sizeof exp_cmac); +} + +/* CMAC-AES-256 test vectors from NIST's updated SP800-38B examples. */ +static void +test_cmac2 (void) +{ + static const uint8_t key[] = { + 0x60,0x3d,0xeb,0x10, 0x15,0xca,0x71,0xbe, + 0x2b,0x73,0xae,0xf0, 0x85,0x7d,0x77,0x81, + 0x1f,0x35,0x2c,0x07, 0x3b,0x61,0x08,0xd7, + 0x2d,0x98,0x10,0xa3, 0x09,0x14,0xdf,0xf4, + }; + static const uint8_t data[] = { + 0x6b,0xc1,0xbe,0xe2, 0x2e,0x40,0x9f,0x96, + 0xe9,0x3d,0x7e,0x11, 0x73,0x93,0x17,0x2a, + 0xae,0x2d,0x8a,0x57, 0x1e,0x03,0xac,0x9c, + 0x9e,0xb7,0x6f,0xac, 0x45,0xaf,0x8e,0x51, + 0x30,0xc8,0x1c,0x46, 0xa3,0x5c,0xe4,0x11, + 0xe5,0xfb,0xc1,0x19, 0x1a,0x0a,0x52,0xef, + 0xf6,0x9f,0x24,0x45, 0xdf,0x4f,0x9b,0x17, + 0xad,0x2b,0x41,0x7b, 0xe6,0x6c,0x37,0x10, + }; + static const uint8_t exp_cmac0[] = { + 0x02,0x89,0x62,0xf6, 0x1b,0x7b,0xf8,0x9e, + 0xfc,0x6b,0x55,0x1f, 0x46,0x67,0xd9,0x83, + }; + static const uint8_t exp_cmac16[] = { + 0x28,0xa7,0x02,0x3f, 0x45,0x2e,0x8f,0x82, + 0xbd,0x4b,0xf2,0x8d, 0x8c,0x37,0xc3,0x5c, + }; + static const uint8_t exp_cmac40[] = { + 0xaa,0xf3,0xd8,0xf1, 0xde,0x56,0x40,0xc2, + 0x32,0xf5,0xb1,0x69, 0xb9,0xc9,0x11,0xe6, + }; + static const uint8_t exp_cmac64[] = { + 0xe1,0x99,0x21,0x90, 0x54,0x9f,0x6e,0xd5, + 0x69,0x6a,0x2c,0x05, 0x6c,0x31,0x54,0x10, + }; + + test_cmac (key, sizeof key, data, 0, exp_cmac0, sizeof exp_cmac0); + test_cmac (key, sizeof key, data, 16, exp_cmac16, sizeof exp_cmac16); + test_cmac (key, sizeof key, data, 40, exp_cmac40, sizeof exp_cmac40); + test_cmac (key, sizeof key, data, 64, exp_cmac64, sizeof exp_cmac64); +} + +int +main(void) +{ + test_cmac1 (); + test_cmac2 (); + return 0; +} diff --git a/utilities/pspp-convert.1 b/utilities/pspp-convert.1 index f1580cd6b5..d4c3a7b11d 100644 --- a/utilities/pspp-convert.1 +++ b/utilities/pspp-convert.1 @@ -46,6 +46,11 @@ SPSS portable file. Use \fB\-O \fIextension\fR to override the inferred format or to specify the format for unrecognized extensions. . +.PP +\fBpspp\-convert\fR can convert most input formats to most output +formats, with one exception: if the input file is an encrypted system +file, then the output file must also be an (unencrypted) system file. +. .SH "OPTIONS" . .IP "\fB\-O format\fR" @@ -66,6 +71,16 @@ Overrides the encoding in which character strings in \fIinput\fR are interpreted. This option is necessary because old SPSS system files do not self-identify their encoding. . +.IP "\fB\-p \fIpassword\fR" +.IQ "\fB\-\-password=\fIpassword\fR" +Specifies the password to use to decrypt an encrypted SPSS system file +\fIinput\fR. If this option is not specified, \fBpspp\-convert\fR +prompts for the password. +. +.IP +On multiuser systems, this option may not be safe because other users +may be able to see the password in process listings. +. .IP "\fB\-h\fR" .IQ "\fB\-\-help\fR" Prints a usage message on stdout and exits. diff --git a/utilities/pspp-convert.c b/utilities/pspp-convert.c index 61e346d124..2dea20d2f3 100644 --- a/utilities/pspp-convert.c +++ b/utilities/pspp-convert.c @@ -16,16 +16,20 @@ #include +#include #include #include #include +#include #include "data/any-reader.h" #include "data/casereader.h" #include "data/casewriter.h" #include "data/csv-file-writer.h" +#include "data/file-name.h" #include "data/por-file-writer.h" #include "data/settings.h" +#include "data/sys-file-encryption.h" #include "data/sys-file-writer.h" #include "data/file-handle-def.h" #include "libpspp/assertion.h" @@ -33,6 +37,7 @@ #include "libpspp/i18n.h" #include "gl/error.h" +#include "gl/getpass.h" #include "gl/progname.h" #include "gl/version-etc.h" @@ -41,6 +46,11 @@ static void usage (void); +static void decrypt_sav_file (struct encrypted_sys_file *enc, + const char *input_filename, + const char *output_filename, + const char *password); + int main (int argc, char *argv[]) { @@ -52,10 +62,12 @@ main (int argc, char *argv[]) struct casereader *reader; struct file_handle *input_fh; const char *encoding = NULL; + struct encrypted_sys_file *enc; const char *output_format = NULL; struct file_handle *output_fh; struct casewriter *writer; + const char *password = NULL; long long int i; @@ -70,6 +82,7 @@ main (int argc, char *argv[]) { { "cases", required_argument, NULL, 'c' }, { "encoding", required_argument, NULL, 'e' }, + { "password", required_argument, NULL, 'p' }, { "output-format", required_argument, NULL, 'O' }, @@ -80,7 +93,7 @@ main (int argc, char *argv[]) int c; - c = getopt_long (argc, argv, "c:e:O:hv", long_options, NULL); + c = getopt_long (argc, argv, "c:e:p:O:hv", long_options, NULL); if (c == -1) break; @@ -94,6 +107,10 @@ main (int argc, char *argv[]) encoding = optarg; break; + case 'p': + password = optarg; + break; + case 'O': output_format = optarg; break; @@ -128,6 +145,16 @@ main (int argc, char *argv[]) output_format = dot + 1; } + if (encrypted_sys_file_open (&enc, input_filename) > 0) + { + if (strcmp (output_format, "sav") && strcmp (output_format, "sys")) + error (1, 0, _("can only convert encrypted data file to sav or sys " + "format")); + + decrypt_sav_file (enc, input_filename, output_filename, password); + goto exit; + } + input_fh = fh_create_file (NULL, input_filename, fh_default_properties ()); reader = any_reader_open (input_fh, encoding, &dict); if (reader == NULL) @@ -179,12 +206,58 @@ main (int argc, char *argv[]) if (!casewriter_destroy (writer)) error (1, 0, _("%s: error writing output file"), output_filename); +exit: fh_done (); i18n_done (); return 0; } +static void +decrypt_sav_file (struct encrypted_sys_file *enc, + const char *input_filename, + const char *output_filename, + const char *password) +{ + FILE *out; + int err; + + if (password == NULL) + { + password = getpass ("password: "); + if (password == NULL) + exit (1); + } + + if (!encrypted_sys_file_unlock (enc, password)) + error (1, 0, _("sorry, wrong password")); + + out = fn_open (output_filename, "wb"); + if (out == NULL) + error (1, errno, ("%s: error opening output file"), output_filename); + + for (;;) + { + uint8_t buffer[1024]; + size_t n; + + n = encrypted_sys_file_read (enc, buffer, sizeof buffer); + if (n == 0) + break; + + if (fwrite (buffer, 1, n, out) != n) + error (1, errno, ("%s: write error"), output_filename); + } + + err = encrypted_sys_file_close (enc); + if (err) + error (1, err, ("%s: read error"), input_filename); + + if (fflush (out) == EOF) + error (1, errno, ("%s: write error"), output_filename); + fn_close (output_filename, out); +} + static void usage (void) { @@ -204,6 +277,7 @@ Options:\n\ is one of the extensions listed above\n\ -e, --encoding=CHARSET override encoding of input data file\n\ -c MAXCASES limit number of cases to copy (default is all cases)\n\ + -p PASSWORD password for encrypted .sav files\n\ --help display this help and exit\n\ --version output version information and exit\n", program_name, program_name);