-@node System and Portable Files
-@chapter System Files and Portable Files
+@node System and Portable File IO
+@chapter System and Portable File I/O
The commands in this chapter read, write, and examine system files and
portable files.
* GET:: Read from a system file.
* GET DATA:: Read from foreign files.
* IMPORT:: Read from a portable file.
-* MATCH FILES:: Merge system files.
* SAVE:: Write to a system file.
* SYSFILE INFO:: Display system file dictionary.
* XEXPORT:: Write to a portable file, as a transformation.
system file are considered. Variables with the same name but different
types (numeric, string) will cause an error message. Otherwise, the
system file variables' attributes will replace those in their matching
-active file variables, as described below.
+active file variables:
+@itemize @bullet
+@item
If a system file variable has a variable label, then it will replace the
active file variable's variable label. If the system file variable does
not have a variable label, then the active file variable's variable
-label, if any, will be retained.
+label, if any, will be retained.
+
+@item
+If the system file variable has custom attributes (@pxref{VARIABLE
+ATTRIBUTE}), then those attributes replace the active file variable's
+custom attributes. If the system file variable does not have custom
+attributes, then the active file variable's custom attributes, if any,
+will be retained.
+@item
If the active file variable is numeric or short string, then value
labels and missing values, if any, will be copied to the active file
variable. If the system file variable does not have value labels or
missing values, then those in the active file variable, if any, will not
be disturbed.
+@end itemize
+
+In addition to properties of variables, some properties of the active
+file dictionary as a whole are updated:
-Finally, weighting of the active file is updated (@pxref{WEIGHT}). If
-the active file has a weighting variable, and the system file does not,
-or if the weighting variable in the system file does not exist in the
-active file, then the active file weighting variable, if any, is
-retained. Otherwise, the weighting variable in the system file becomes
-the active file weighting variable.
+@itemize @bullet
+@item
+If the system file has custom attributes (@pxref{DATAFILE ATTRIBUTE}),
+then those attributes replace the active file variable's custom
+attributes.
+
+@item
+If the active file has a weighting variable (@pxref{WEIGHT}), and the
+system file does not, or if the weighting variable in the system file
+does not exist in the active file, then the active file weighting
+variable, if any, is retained. Otherwise, the weighting variable in
+the system file becomes the active file weighting variable.
+@end itemize
@cmd{APPLY DICTIONARY} takes effect immediately. It does not read the
active
@display
GET DATA
- /TYPE=@{GNM,TXT@}
- /FILE=@{'file-name',file_handle@}
+ /TYPE=@{GNM,PSQL,TXT@}
@dots{}additional subcommands depending on TYPE@dots{}
@end display
@item GNM
Spreadsheet files created by Gnumeric (@url{http://gnumeric.org}).
+@item PSQL
+Relations from PostgreSQL databases (@url{http://postgresql.org}).
+
@item TXT
Textual data files in columnar and delimited formats.
@end table
-The FILE subcommand is mandatory for all implemented file types.
-Specify the file to be read as a string file name or (for textual data
-only) a file handle (@pxref{File Handles}).
-
Each supported file type has additional subcommands, explained in
separate sections below.
@menu
* GET DATA /TYPE=GNM::
+* GET DATA /TYPE=PSQL::
* GET DATA /TYPE=TXT::
@end menu
determined from the length of the string it contains, unless the
ASSUMEDVARWIDTH subcommand is given.
+
+The FILE subcommand is mandatory. Specify the name of the file
+to be read.
+
The SHEET subcommand specifies the sheet within the spreadsheet file to read.
There are two forms of the SHEET subcommand.
In the first form,
If omitted, the default value is determined from the length of the
string in the first spreadsheet cell for each variable.
+
+@node GET DATA /TYPE=PSQL
+@subsection Postgres Database Queries
+
+@display
+GET DATA /TYPE=PSQL
+ /CONNECT=@{connection info@}
+ /SQL=@{query@}
+ [/ASSUMEDVARWIDTH=n]
+ [/UNENCRYPTED]
+ [/BSIZE=n].
+@end display
+
+@cindex postgres
+@cindex databases
+
+The PSQL type is used to import data from a postgres database server.
+The server may be located locally or remotely.
+Variables are automatically created based on the table column names
+or the names specified in the SQL query.
+Postgres data types of high precision, will loose precision when
+imported into PSPP.
+Not all the postgres data types are able to be represented in PSPP.
+If a datum cannot be represented a warning will be issued and that
+datum will be set to SYSMIS.
+
+The CONNECT subcommand is mandatory.
+It is a string specifying the parameters of the database server from
+which the data should be fetched.
+The format of the string is given in the postgres manual
+@url{http://www.postgresql.org/docs/8.0/static/libpq.html#LIBPQ-CONNECT}.
+
+The SQL subcommand is mandatory.
+It must be a valid SQL string to retrieve data from the database.
+
+The ASSUMEDVARWIDTH subcommand specifies the maximum width of string
+variables read from the database.
+If omitted, the default value is determined from the length of the
+string in the first value read for each variable.
+
+The UNENCRYPTED subcommand allows data to be retrieved over an insecure
+connection.
+If the connection is not encrypted, and the UNENCRYPTED subcommand is not
+given, then an error will occur.
+Whether or not the connection is
+encrypted depends upon the underlying psql library and the
+capabilities of the database server.
+
+The BSIZE subcommand serves only to optimise the speed of data transfer.
+It specifies an upper limit on
+number of cases to fetch from the database at once.
+The default value is 4096.
+If your SQL statement fetches a large number of cases but only a small number of
+variables, then the data transfer may be faster if you increase this value.
+Conversely, if the number of variables is large, or if the machine on which
+PSPP is running has only a
+small amount of memory, then a smaller value will be better.
+
+
+The following syntax is an example:
+@example
+GET DATA /TYPE=PSQL
+ /CONNECT='host=example.com port=5432 dbname=product user=fred passwd=xxxx'
+ /SQL='select * from manufacturer'.
+@end example
+
+
@node GET DATA /TYPE=TXT
@subsection Textual Data Files
@cindex text files
@cindex data files
When TYPE=TXT is specified, GET DATA reads data in a delimited or
-fixed columnar format, much like DATA LIST (@pxref{DATA LIST}). The
-FILE subcommand must be specified indicate the name or the file handle
-of the file to be read.
+fixed columnar format, much like DATA LIST (@pxref{DATA LIST}).
+
+The FILE subcommand is mandatory. Specify the file to be read as
+a string file name or (for textual data
+only) a file handle (@pxref{File Handles}).
The ARRANGEMENT subcommand determines the file's basic format.
DELIMITED, the default setting, specifies that fields in the input
[/IMPORTCASE=@{ALL,FIRST max_cases,PERCENT percent@}]
/DELIMITERS="delimiters"
- [/QUALIFIER="quote"]
+ [/QUALIFIER="quotes" [/ESCAPE]]
[/DELCASE=@{LINE,VARIABLES n_variables@}]
/VARIABLES=del_var [del_var]@dots{}
where each del_var takes the form:
which each field appears on a separate line, specify the empty string
for DELIMITERS.
-The optional QUALIFIER subcommand names a character that can be used
-to quote values within fields in the input. A field that begins with
-the specified quote character ends at the next match quote.
-Intervening delimiters become part of the field, instead of
-terminating it.
+The optional QUALIFIER subcommand names one or more characters that
+can be used to quote values within fields in the input. A field that
+begins with one of the specified quote characters ends at the next
+matching quote. Intervening delimiters become part of the field,
+instead of terminating it. The ability to specify more than one quote
+character is a PSPP extension.
+
+By default, a character specified on QUALIFIER cannot itself be
+embedded within a field that it quotes, because the quote character
+always terminates the quoted field. With ESCAPE, however, a doubled
+quote character within a quoted field inserts a single instance of the
+quote into the field. For example, if @samp{'} is specified on
+QUALIFIER, then without ESCAPE @code{'a''b'} specifies a pair of
+fields that contain @samp{a} and @samp{b}, but with ESCAPE it
+specifies a single field that contains @samp{a'b}. ESCAPE is a PSPP
+extension.
The DELCASE subcommand controls how data may be broken across lines in
the data file. With LINE, the default setting, each line must contain
Consider the following information on animals in a pet store:
@example
-"Pet Name", "Age", "Color", "Date Received", "Price", "Needs Walking", "Type"
+'Pet''s Name', "Age", "Color", "Date Received", "Price", "Height", "Type"
, (Years), , , (Dollars), ,
-"Rover", 4.5, Brown, "12 Feb 2004", 80, True, "Dog"
-"Charlie", , Gold, "5 Apr 2007", 12.3, False, "Fish"
-"Molly", 2, Black, "12 Dec 2006", 25, False, "Cat"
-"Gilly", , White, "10 Apr 2007", 10, False, "Guinea Pig"
+"Rover", 4.5, Brown, "12 Feb 2004", 80, '1''4"', "Dog"
+"Charlie", , Gold, "5 Apr 2007", 12.3, "3""", "Fish"
+"Molly", 2, Black, "12 Dec 2006", 25, '5"', "Cat"
+"Gilly", , White, "10 Apr 2007", 10, "3""", "Guinea Pig"
@end example
@noindent
@c If you change this example, change the regression test in
@c tests/command/get-data-txt-examples.sh to match.
@example
-GET DATA /TYPE=TXT /FILE='pets.data' /DELIMITERS=', ' /QUALIFIER='"'
+GET DATA /TYPE=TXT /FILE='pets.data' /DELIMITERS=', ' /QUALIFIER='''"' /ESCAPE
/FIRSTCASE=3
/VARIABLES=name A10
age F3.1
color A5
received EDATE10
price F5.2
- needs_walking A5
- type A10.
+ height a5
+ type a10.
@end example
@node GET DATA /TYPE=TXT /ARRANGEMENT=FIXED
Use of @cmd{IMPORT} to read a system file or scratch file is a PSPP
extension.
-@node MATCH FILES
-@section MATCH FILES
-@vindex MATCH FILES
-
-@display
-MATCH FILES
- /@{FILE,TABLE@}=@{*,'file-name'@}
- /RENAME=(src_names=target_names)@dots{}
- /IN=var_name
-
- /BY=var_list
- /DROP=var_list
- /KEEP=var_list
- /FIRST=var_name
- /LAST=var_name
- /MAP
-@end display
-
-@cmd{MATCH FILES} merges one or more system, portable, or scratch files,
-optionally
-including the active file. Cases with the same values for BY
-variables are combined into a single case. Cases with different
-values are output in order. Thus, multiple sorted files are
-combined into a single sorted file based on the value of the BY
-variables. The results of the merge become the new active file.
-
-Specify FILE with a system, portable, or scratch file as a file name
-string or file handle
-(@pxref{File Handles}), or with an asterisk (@samp{*}) to
-indicate the current active file. The files specified on FILE are
-merged together based on the BY variables, or combined case-by-case if
-BY is not specified.
-
-Specify TABLE with a file to use it as a @dfn{table
-lookup file}. Cases in table lookup files are not used up after
-they've been used once. This means that data in table lookup files can
-correspond to any number of cases in FILE files. Table lookup files
-correspond to lookup tables in traditional relational database systems.
-If a table lookup file contains more than one case with a given set of
-BY variables, only the first case is used.
-
-Any number of FILE and TABLE subcommands may be specified.
-Ordinarily, at least two FILE subcommands, or one FILE and at least
-one TABLE, should be specified. Each instance of FILE or TABLE can be
-followed by any sequence of RENAME subcommands. These have the same
-form and meaning as the corresponding subcommands of @cmd{GET}
-(@pxref{GET}), but apply only to variables in the given file.
-
-Each FILE or TABLE may optionally be followed by an IN subcommand,
-which creates a numeric variable with the specified name and format
-F1.0. The IN variable takes value 1 in a case if the given file
-contributed a row to the merged file, 0 otherwise. The DROP, KEEP,
-and RENAME subcommands do not affect IN variables.
-
-When more than one FILE or TABLE contains a variable with a given
-name, those variables must all have the same type (numeric or string)
-and, for string variables, the same width. This rules applies to
-variable names after renaming with RENAME; thus, RENAME can be used to
-resolve conflicts.
-
-FILE and TABLE must be specified at the beginning of the command, with
-any RENAME or IN specifications immediately after the corresponding
-FILE or TABLE. These subcommands are followed by BY, DROP, KEEP,
-FIRST, LAST, and MAP.
-
-The BY subcommand specifies a list of variables that are used to match
-cases from each of the files. When TABLE or IN is used, BY is
-required; otherwise, it is optional. When BY is specified, all the
-files named on FILE and TABLE subcommands must be sorted in ascending
-order of the BY variables. Variables belonging to files that are not
-present for the current case are set to the system-missing value for
-numeric variables or spaces for string variables.
-
-The DROP and KEEP subcommands allow variables to be dropped from or
-reordered within the new active file. These subcommands have the same
-form and meaning as the corresponding subcommands of @cmd{GET}
-(@pxref{GET}). They apply to the new active file as a whole, not to
-individual input files. The variable names specified on DROP and KEEP
-are those after any renaming with RENAME.
-
-The optional FIRST and LAST subcommands name variables that @cmd{MATCH
-FILES} adds to the active file. The new variables are numeric with
-print and write format F1.0. The value of the FIRST variable is 1 in
-the first case with a given set of values for the BY variables, and 0
-in other cases. Similarly, the LAST variable is 1 in the last case
-with a given of BY values, and 0 in other cases.
-
-@cmd{MATCH FILES} may not be specified following @cmd{TEMPORARY}
-(@pxref{TEMPORARY}) if the active file is used as an input source.
-
-Use of portable or scratch files on @cmd{MATCH FILES} is a PSPP
-extension.
-
@node SAVE
@section SAVE
@vindex SAVE
@end itemize
@xref{SAVE}, for more information.
-@setfilename ignored
projects / pspp-builds.git / blobdiff