1 @node Invoking gnulib-tool
2 @chapter Invoking gnulib-tool
4 @c Copyright (C) 2005-2008 Free Software Foundation, Inc.
6 @c Permission is granted to copy, distribute and/or modify this document
7 @c under the terms of the GNU Free Documentation License, Version 1.2 or
8 @c any later version published by the Free Software Foundation; with no
9 @c Invariant Sections, with no Front-Cover Texts, and with no Back-Cover
10 @c Texts. A copy of the license is included in the ``GNU Free
11 @c Documentation License'' file as part of this distribution.
14 @cindex invoking @command{gnulib-tool}
16 The @command{gnulib-tool} command is the recommended way to import
17 Gnulib modules. It is possible to borrow Gnulib modules in a package
18 without using @command{gnulib-tool}, relying only on the
19 meta-information stored in the @file{modules/*} files, but with a
20 growing number of modules this becomes tedious. @command{gnulib-tool}
21 simplifies the management of source files, @file{Makefile.am}s and
22 @file{configure.ac} in packages incorporating Gnulib modules.
24 Run @samp{gnulib-tool --help} for information. To get familiar with
25 @command{gnulib-tool} without affecting your sources, you can also try
26 some commands with the option @samp{--dry-run}; then
27 @code{gnulib-tool} will only report which actions it would perform in
28 a real run without changing anything.
31 * Initial import:: First import of Gnulib modules.
32 * Modified imports:: Changing the import specification.
33 * Simple update:: Tracking Gnulib development.
34 * Source changes:: Impact of Gnulib on your source files.
35 * gettextize and autopoint:: Caveat: @code{gettextize} and @code{autopoint} users!
36 * Localization:: Handling Gnulib's own message translations.
37 * VCS Issues:: Integration with Version Control Systems.
42 @section Initial import
43 @cindex initial import
45 Gnulib assumes your project uses Autoconf and Automake. Invoking
46 @samp{gnulib-tool --import} will copy source files, create a
47 @file{Makefile.am} to build them, generate a file @file{gnulib-comp.m4} with
48 Autoconf M4 macro declarations used by @file{configure.ac}, and generate
49 a file @file{gnulib-cache.m4} containing the cached specification of how
52 Our example will be a library that uses Autoconf, Automake and
53 Libtool. It calls @code{strdup}, and you wish to use gnulib to make
54 the package portable to C89 and C99 (which don't have @code{strdup}).
57 ~/src/libfoo$ gnulib-tool --import strdup
58 Module list with included dependencies:
72 Creating directory ./lib
73 Creating directory ./m4
74 Copying file lib/dummy.c
75 Copying file lib/strdup.c
76 Copying file lib/string.in.h
77 Copying file m4/absolute-header.m4
78 Copying file m4/extensions.m4
79 Copying file m4/gnulib-common.m4
80 Copying file m4/gnulib-tool.m4
81 Copying file m4/strdup.m4
82 Copying file m4/string_h.m4
83 Creating lib/Makefile.am
84 Creating m4/gnulib-cache.m4
85 Creating m4/gnulib-comp.m4
88 You may need to add #include directives for the following .h files.
92 - add "lib/Makefile" to AC_CONFIG_FILES in ./configure.ac,
93 - mention "lib" in SUBDIRS in Makefile.am,
94 - mention "-I m4" in ACLOCAL_AMFLAGS in Makefile.am,
95 - invoke gl_EARLY in ./configure.ac, right after AC_PROG_CC,
96 - invoke gl_INIT in ./configure.ac.
100 By default, the source code is copied into @file{lib/} and the M4
101 macros in @file{m4/}. You can override these paths by using
102 @code{--source-base=DIRECTORY} and @code{--m4-base=DIRECTORY}. Some
103 modules also provide other files necessary for building. These files
104 are copied into the directory specified by @samp{AC_CONFIG_AUX_DIR} in
105 @file{configure.ac} or by the @code{--aux-dir=DIRECTORY} option. If
106 neither is specified, the current directory is assumed.
108 @code{gnulib-tool} can make symbolic links instead of copying the
109 source files. The option to specify for this is @samp{--symlink}, or
110 @samp{-s} for short. This can be useful to save a few kilobytes of disk
111 space. But it is likely to introduce bugs when @code{gnulib} is updated;
112 it is more reliable to use @samp{gnulib-tool --update} (see below)
113 to update to newer versions of @code{gnulib}. Furthermore it requires
114 extra effort to create self-contained tarballs, and it may disturb some
115 mechanism the maintainer applies to the sources. For these reasons,
116 this option is generally discouraged.
118 @code{gnulib-tool} will overwrite any pre-existing files, in
119 particular @file{Makefile.am}. Unfortunately, separating the
120 generated @file{Makefile.am} content (for building the gnulib library)
121 into a separate file, say @file{gnulib.mk}, that could be included
122 by your handwritten @file{Makefile.am} is not possible, due to how
123 variable assignments are handled by Automake.
125 Consequently, it is a good idea to choose directories that are not
126 already used by your projects, to separate gnulib imported files from
127 your own files. This approach is also useful if you want to avoid
128 conflicts between other tools (e.g., @code{gettextize} that also copy
129 M4 files into your package. Simon Josefsson successfully uses a source
130 base of @file{gl/}, and a M4 base of @file{gl/m4/}, in several
133 After the @samp{--import} option on the command line comes the list of
134 Gnulib modules that you want to incorporate in your package. The names
135 of the modules coincide with the filenames in Gnulib's @file{modules/}
138 Some Gnulib modules depend on other Gnulib modules. @code{gnulib-tool}
139 will automatically add the needed modules as well; you need not list
140 them explicitly. @code{gnulib-tool} will also memorize which dependent
141 modules it has added, so that when someday a dependency is dropped, the
142 implicitly added module is dropped as well (unless you have explicitly
143 requested that module).
145 If you want to cut a dependency, i.e., not add a module although one of
146 your requested modules depends on it, you may use the option
147 @samp{--avoid=@var{module}} to do so. Multiple uses of this option are
148 possible. Of course, you will then need to implement the same interface
149 as the removed module.
151 A few manual steps are required to finish the initial import.
152 @code{gnulib-tool} printed a summary of these steps.
154 First, you must ensure Autoconf can find the macro definitions in
155 @file{gnulib-comp.m4}. Use the @code{ACLOCAL_AMFLAGS} specifier in
156 your top-level @file{Makefile.am} file, as in:
159 ACLOCAL_AMFLAGS = -I m4
162 You are now ready to call the M4 macros in @code{gnulib-comp.m4} from
163 @file{configure.ac}. The macro @code{gl_EARLY} must be called as soon
164 as possible after verifying that the C compiler is working.
165 Typically, this is immediately after @code{AC_PROG_CC}, as in:
174 The core part of the gnulib checks are done by the macro
175 @code{gl_INIT}. Place it further down in the file, typically where
176 you normally check for header files or functions. It must come after
177 other checks which may affect the compiler invocation, such as
178 @code{AC_MINIX}. For example:
187 @code{gl_INIT} will in turn call the macros related with the
188 gnulib functions, be it specific gnulib macros, like @code{gl_FUNC_ALLOCA}
189 or autoconf or automake macros like @code{AC_FUNC_ALLOCA} or
190 @code{AM_FUNC_GETLINE}. So there is no need to call those macros yourself
191 when you use the corresponding gnulib modules.
193 You must also make sure that the gnulib library is built. Add the
194 @code{Makefile} in the gnulib source base directory to
195 @code{AC_CONFIG_FILES}, as in:
198 AC_CONFIG_FILES(... lib/Makefile ...)
201 You must also make sure that @code{make} will recurse into the gnulib
202 directory. To achieve this, add the gnulib source base directory to a
203 @code{SUBDIRS} Makefile.am statement, as in:
209 or if you, more likely, already have a few entries in @code{SUBDIRS},
210 you can add something like:
216 Finally, you have to add compiler and linker flags in the appropriate
217 source directories, so that you can make use of the gnulib library.
218 Since some modules (@samp{getopt}, for example) may copy files into
219 the build directory, @file{top_builddir/lib} is needed as well
220 as @file{top_srcdir/lib}. For example:
224 AM_CPPFLAGS = -I$(top_builddir)/lib -I$(top_srcdir)/lib
230 Don't forget to @code{#include} the various header files. In this
231 example, you would need to make sure that @samp{#include <string.h>}
232 is evaluated when compiling all source code files, that want to make
233 use of @code{strdup}.
235 In the usual case where Autoconf is creating a @file{config.h} file,
236 you should include @file{config.h} first, before any other include
237 file. That way, for example, if @file{config.h} defines
238 @samp{restrict} to be the empty string on a pre-C99 host, or a macro
239 like @samp{_FILE_OFFSET_BITS} that affects the layout of data
240 structures, the definition is consistent for all include files.
241 Also, on some platforms macros like @samp{_FILE_OFFSET_BITS} and
242 @samp{_GNU_SOURCE} may be ineffective, or may have only a limited
243 effect, if defined after the first system header file is included.
245 Finally, note that you can not use @code{AC_LIBOBJ} or
246 @code{AC_REPLACE_FUNCS} in your @file{configure.ac} and expect the
247 resulting object files to be automatically added to @file{lib/libgnu.a}.
248 This is because your @code{AC_LIBOBJ} and @code{AC_REPLACE_FUNCS} invocations
249 from @file{configure.ac} augment a variable @code{@@LIBOBJS@@} (and/or
250 @code{@@LTLIBOBJS@@} if using Libtool), whereas @file{lib/libgnu.a}
251 is built from the contents of a different variable, usually
252 @code{@@gl_LIBOBJS@@} (or @code{@@gl_LTLIBOBJS@@} if using Libtool).
255 @node Modified imports
256 @section Modified imports
258 You can at any moment decide to use Gnulib differently than the last time.
260 If you only want to use more Gnulib modules, simply invoke
261 @command{gnulib-tool --import @var{new-modules}}. @code{gnulib-tool}
262 remembers which modules were used last time. The list of modules that
263 you pass after @samp{--import} is @emph{added} to the previous list of
266 For most changes, such as added or removed modules, or even different
267 choices of @samp{--lib}, @samp{--source-base} or @samp{--aux-dir}, there
268 are two ways to perform the change.
270 The standard way is to modify manually the file @file{gnulib-cache.m4}
271 in the M4 macros directory, then launch @samp{gnulib-tool --import}.
273 The other way is to call @command{gnulib-tool} again, with the changed
274 command-line options. Note that this doesn't let you remove modules,
275 because as you just learned, the list of modules is always cumulated.
276 Also this way is often impractical, because you don't remember the way
277 you invoked @code{gnulib-tool} last time.
279 The only change for which this doesn't work is a change of the
280 @samp{--m4-base} directory. Because, when you pass a different value of
281 @samp{--m4-base}, @code{gnulib-tool} will not find the previous
282 @file{gnulib-cache.m4} file any more... A possible solution is to manually
283 copy the @file{gnulib-cache.m4} into the new M4 macro directory.
285 In the @file{gnulib-cache.m4}, the macros have the following meaning:
288 The argument is a space separated list of the requested modules, not including
292 The argument is a space separated list of modules that should not be used,
293 even if they occur as dependencies. Corresponds to the @samp{--avoid}
294 command line argument.
297 The argument is the relative file name of the directory containing the gnulib
298 source files (mostly *.c and *.h files). Corresponds to the
299 @samp{--source-base} command line argument.
302 The argument is the relative file name of the directory containing the gnulib
303 M4 macros (*.m4 files). Corresponds to the @samp{--m4-base} command line
307 The argument is the relative file name of the directory containing the gnulib
308 unit test files. Corresponds to the @samp{--tests-base} command line argument.
311 The argument is the name of the library to be created. Corresponds to the
312 @samp{--lib} command line argument.
315 The presence of this macro without arguments corresponds to the @samp{--lgpl}
316 command line argument. The presence of this macro with an argument (whose
317 value must be 2 or 3) corresponds to the @samp{--lgpl=@var{arg}} command line
321 The presence of this macro corresponds to the @samp{--libtool} command line
322 argument and to the absence of the @samp{--no-libtool} command line argument.
323 It takes no arguments.
325 @item gl_MACRO_PREFIX
326 The argument is the prefix to use for macros in the @file{gnulib-comp.m4}
327 file. Corresponds to the @samp{--macro-prefix} command line argument.
332 @section Simple update
334 When you want to update to a more recent version of Gnulib, without
335 changing the list of modules or other parameters, a simple call
339 $ gnulib-tool --import
343 This will create, update or remove files, as needed.
345 Note: From time to time, changes are made in Gnulib that are not backward
346 compatible. When updating to a more recent Gnulib, you should consult
347 Gnulib's @file{NEWS} file to check whether the incompatible changes affect
352 @section Changing your sources for use with Gnulib
354 Gnulib contains some header file overrides. This means that when building
355 on systems with deficient header files in @file{/usr/include/}, it may create
356 files named @file{string.h}, @file{stdlib.h}, @file{stdint.h} or similar in
357 the build directory. In the other source directories of your package you
358 will usually pass @samp{-I} options to the compiler, so that these Gnulib
359 substitutes are visible and take precedence over the files in
360 @file{/usr/include/}.
362 These Gnulib substitute header files rely on @file{<config.h>} being
363 already included. Furthermore @file{<config.h>} must be the first include
364 in every compilation unit. This means that to @emph{all your source files}
365 and likely also to @emph{all your tests source files} you need to add an
366 @samp{#include <config.h>} at the top. Which source files are affected?
367 Exactly those whose compilation includes a @samp{-I} option that refers to
368 the Gnulib library directory.
370 This is annoying, but inevitable: On many systems, @file{<config.h>} is
371 used to set system dependent flags (such as @code{_GNU_SOURCE} on GNU systems),
372 and these flags have no effect after any system header file has been included.
375 @node gettextize and autopoint
376 @section Caveat: @code{gettextize} and @code{autopoint} users
378 @cindex gettextize, caveat
379 @cindex autopoint, caveat
380 The programs @code{gettextize} and @code{autopoint}, part of
381 GNU @code{gettext}, import or update the internationalization infrastructure.
382 Some of this infrastructure, namely ca.@: 20 autoconf macro files and the
383 @file{config.rpath} file, is also contained in Gnulib and may be imported
384 by @code{gnulib-tool}. The use of @code{gettextize} or @code{autopoint}
385 will therefore overwrite some of the files that @code{gnulib-tool} has
386 imported, and vice versa.
388 Avoiding to use @code{gettextize} (manually, as package maintainer) or
389 @code{autopoint} (as part of a script like @code{autoreconf} or
390 @code{autogen.sh}) is not the solution: These programs also import the
391 infrastructure in the @file{po/} and optionally in the @file{intl/} directory.
393 The copies of the conflicting files in Gnulib are more up-to-date than
394 the copies brought in by @code{gettextize} and @code{autopoint}. When a
395 new @code{gettext} release is made, the copies of the files in Gnulib will
396 be updated immediately.
398 The solution is therefore:
402 When you run @code{gettextize}, always use the @code{gettextize} from the
403 newest GNU gettext release found on @url{http://ftp.gnu.org/gnu/gettext/},
404 and invoke @code{gnulib-tool} afterwards.
407 When a script of yours run @code{autopoint}, invoke @code{gnulib-tool}
411 If you get an error message like
412 @code{*** error: gettext infrastructure mismatch:
413 using a Makefile.in.in from gettext version ...
414 but the autoconf macros are from gettext version ...},
415 it means that a new GNU gettext release was made, and its autoconf macros
416 were integrated into Gnulib and now mismatch the @file{po/} infrastructure.
417 In this case, fetch and install the new GNU gettext release and run
418 @code{gettextize} followed by @code{gnulib-tool}.
423 @section Handling Gnulib's own message translations
425 Gnulib provides some functions that emit translatable messages using GNU
426 @code{gettext}. The @samp{gnulib} domain at the
427 @url{http://translationproject.org/, Translation Project} collects
428 translations of these messages, which you should incorporate into your
431 There are two basic ways to achieve this. The first, and older, method
432 is to list all the source files you use from Gnulib in your own
433 @file{po/POTFILES.in} file. This will cause all the relevant
434 translatable strings to be included in your POT file. When you send
435 this POT file to the Translation Project, translators will normally fill
436 in the translations of the Gnulib strings from their ``translation
437 memory'', and send you back updated PO files.
439 However, this process is error-prone: you might forget to list some
440 source files, or the translator might not be using a translation memory
441 and provide a different translation than another translator, or the
442 translation might not be kept in sync between Gnulib and your package.
443 It is also slow and causes substantial extra work, because a human
444 translator must be in the loop for each language and you will need to
445 incorporate their work on request.
447 For these reasons, a new method was designed and is now recommended. If
448 you pass the @code{--po-base=@var{directory}} and @code{--po-domain=@var{domain}}
449 options to @code{gnulib-tool}, then @code{gnulib-tool} will create a
450 separate directory with its own @file{POTFILES.in}, and fetch current
451 translations directly from the Translation Project (using
452 @command{rsync} or @command{wget}, whichever is available).
453 The POT file in this directory will be called
454 @file{@var{domain}-gnulib.pot}, depending on the @var{domain} you gave to the
455 @code{--po-domain} option (typically the same as the package name).
456 This causes these translations to reside in a separate message domain,
457 so that they do not clash either with the translations for the main part
458 of your package nor with those of other packages on the system that use
459 possibly different versions of Gnulib.
460 When you use these options, the functions in Gnulib are built
461 in such a way that they will always use this domain regardless of the
462 default domain set by @code{textdomain}.
464 In order to use this method, you must -- in each program that might use
465 Gnulib code -- add an extra line to the part of the program that
466 initializes locale-dependent behavior. Where you would normally write
471 setlocale (LC_ALL, "");
472 bindtextdomain (PACKAGE, LOCALEDIR);
473 textdomain (PACKAGE);
478 you should add an additional @code{bindtextdomain} call to inform
479 gettext of where the MO files for the extra message domain may be found:
483 bindtextdomain (PACKAGE "-gnulib", LOCALEDIR);
487 (This example assumes that the @var{domain} that you specified
488 to @code{gnulib-tool} is the same as the value of the @code{PACKAGE}
491 Since you do not change the @code{textdomain} call, the default message
492 domain for your program remains the same and your own use of @code{gettext}
493 functions will not be affected.
497 @section Issues with Version Control Systems
499 If a project stores its source files in a version control system (VCS),
500 such as CVS, SVN, or Git, one needs to decide which files to commit.
502 All files created by @code{gnulib-tool}, except @file{gnulib-cache.m4},
503 should be treated like generated source files, like for example a
504 @file{parser.c} file is generated from @file{parser.y}.
509 In projects which commit all source files, whether generated or not, into
510 their VCS, the @code{gnulib-tool} generated files should all be committed.
512 Gnulib also contains files generated by @command{make} (and removed by
513 @code{make clean}), using information determined by @command{configure}.
514 They should not be checked into the VCS, but instead added to
515 @file{.gitignore} or @file{.cvsignore}.
516 When you have a Gnulib source file of the form @file{lib/foo.in.h}, the
517 corresponding @file{lib/foo.h} is such a file.
520 In projects which customarily omit from their VCS all files that are generated
521 from other source files, all these files and directories would not be
522 added into the VCS. The only file that must be added to the VCS is
523 @file{gnulib-cache.m4} in the M4 macros directory. Also, the script for
524 restoring files not in the VCS, customarily called @file{autogen.sh} or
525 @file{bootstrap.sh}, will typically contain the statement for restoring
529 $ gnulib-tool --update
532 The @samp{--update} option operates much like the @samp{--import} option,
533 but it does not offer the possibility to change the way Gnulib is used.
534 Also it does not report in the ChangeLogs the files that it had to add
535 because they were missing.