Accept LF, CR LF, and LF as new-line sequences in data files.
Until now, PSPP has used the host operating system's idea of the
new-line sequence when reading data files and other text files.
This means that, when a file with CR LF line ends is read on an OS
that uses LF as new-line (e.g. an MS-DOS file on Unix), each line
appears to have a CR at the the end. This commit fixes the
problem, by normalizing the new-line sequence at time of reading.
This commit eliminates a performance optimization from
ds_read_line(), because the getdelim() function that it used cannot
be made to stop reading at one of two different delimiters. If
this causes a real performance regression, then the getndelim2
function from gnulib could be used to restore the optimization.
Also adds a test to make sure that it works.
Thanks to RĂ©mi Dewitte <remi@gide.net> for pointing out the problem
and providing an initial patch (which solved the problem in a
completely different way from this commit).