* src/stat.c (USE_STATVFS): Adjust definition so that it is enabled
also on AIX 7.x systems that provide statvfs64 and no statvfs.
[USE_STATVFS && ! STAT_STATVFS && STAT_STATVFS64] (STATFS): Define
to statvfs64 in that precise case.
* m4/stat-prog.m4 (cu_PREREQ_STAT_PROG): Adjust the condition
here to match the new one in stat.c, to keep them in sync.
Reported by Bruno Haible. For details, see
http://article.gmane.org/gmane.comp.gnu.coreutils.general/1668
* tests/init.cfg (require_mkfs_PATH_): New function to test whether mkfs
is in PATH, otherwise adding /sbin to PATH. Needed for distributions
(OpenSuSE, Solaris) in which sudo does not include /sbin in PATH.
* tests/cp/cp-a-selinux: Use require_mkfs_PATH_.
* tests/cp/cp-mv-enotsup-xattr: Likewise.
* tests/cp/sparse-fiemap: Likewise.
* tests/mkdir/writable-under-readonly: Likewise.
* tests/rm/read-only: Likewise.
* NEWS (Improvements): Mention the cp-vs-NFSv4-ACL improvement we
have inherited via gnulib. For details, see
http://article.gmane.org/gmane.comp.lib.gnulib.bugs/28100
and the following messages in that thread.
* gnulib: Update to latest, for numerous ACL-related improvements.
* tests/init.sh: Relax check for diff -u support.
Rather than checking for GNU diff via --version, simply check
for support for -u itself. Useful at least on OpenBSD 4.9.
* tests/require-perl: Remove file.
* tests/Makefile.am (EXTRA_DIST): Remove it from this list.
* tests/init.cfg (require_perl_): New function.
* tests/misc/pwd-long: Use the new function, not the file.
* tests/ls/nameless-uid: Likewise.
* tests/misc/sum-sysv: Likewise.
* tests/misc/cut: Repeat each test using a multibyte locale,
if the configure-time test found such a locale.
Adjust the tests so that they also accept a slightly
different diagnostic that is specific to the MB-patched cut.
* HACKING (Add tests): Mention the variables and default values.
* README-release (Pre-release testing): Mention that setting
the SHELL variable may be required.
Suggested by Bruno Haible.
* tests/split/l-chunk: Don't use the `test "$var"`
idiom to test that var is set to something as
that's not supported by all shells.
The new style matches the usage is the rest of
the test in any case.
Reported by Bruno Haible on AIX 6.1 and 7.1
Without this change, we'd get use-of-uninit value warnings
and harder-to-diagnose failure down the road.
* tests/misc/pwd-long (normalize_to_cwd_relative): Diagnose stat
failure. This failed on AIX 6.1 and 7.1. Reported by Bruno Haible.
* tests/misc/printf-surprise: Also accept a strerror-style string
after the usual 'printf: write error:' diagnostic prefix.
Otherwise, this test would fail on HP-UX 11.
Reported by Bruno Haible.
* src/stdbuf.c (main): Pass the path of `stdbuf` rather than
the command it's running to the search function. This is
significant on platforms without /proc/self/exe
Reported by Bruno Haible
... which is not available on some platforms,
and the replacement currently requires linking
with threading libraries.
* src/split.c (closeout): Remove the call to strsignal()
which is largely redundant anyway as sig2str()
is already used to map number to name in the error.
Reported by Bruno Haible on AIX 6.1 and 7.1
Thanks to an improvement in gnulib's parse-datetime module,
commands like this now succeed (output manually indented):
$ ./date -u -d 2004-02-29T16:21:42.33+07:00 +%FT%T.%N%z
2004-02-29T09:21:42.330000000+0000
* tests/misc/date: Add a test to exercise the new-in-gnulib
parsing of ISO8601-with-"T" dates.
* NEWS (New features): Mention it.
* gnulib: Update, to pull in this parse-datetime improvement.
* gl/lib/heap.c: Move declaration to the top of scope.
Reported by Rob McMahon and Wolfgang Steinwender
in relation to "Sun WorkShop 6 update 2 C 5.3 Patch 111679-12
2003/05/18" and "GCC 2.95.3" respectively.
This effectively reverts the unreleased commit 5a647a05
* src/timeout.c (main): Don't propagate signals from the monitored
process, as on Linux /proc/sys/kernel/core_pattern could still
handle them and cause false reports against `timeout`
Lesson: do not include details like "4 million" in a file name.
* tests/rm/many-dir-entries-vs-OOM: Renamed from ...
* tests/rm/4-million-entry-dir: ...this.
* tests/Makefile.am (TESTS): Reflect renaming.
* tests/rm/4-million-entry-dir: Create only 200,000 files, rather
than 4 million. The latter was overkill, and was too likely to
fail due to inode exhaustion. Not everyone is using btrfs yet.
Now that this test doesn't take so long, label it as merely
"expensive", rather than "very expensive". Thanks to
Bernhard Voelker for pointing out the risk of inode exhaustion.
Ever since the LAST_PAGE functionality was added
in commit ed0923a1, 1996-12-05,
"Apply big patch (patch-20) from Roland Huebner"
it was ignored when -t or -T were specified or
when -l <= 10.
* src/pr.c (print_page): Increment the current page here.
(print_header): Don't increment the current page here.
* tests/pr/pr-tests: Add a test case.
* NEWS: Mention the fix
Reported at http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9347
For details, see the gnulib commit,
http://git.sv.gnu.org/cgit/gnulib.git/commit/?id=47cb657e
* tests/rm/4-million-entry-dir: New test.
* tests/Makefile.am (TESTS): Add it.
* NEWS (Bug fixes): Mention it.
* gnulib: Update to latest to get the required fts fixes.
* src/head.c (elide_tail_bytes_pipe): Remove the empty statement, ";"
after a jump label; it is needed only when a declaration follows.
* src/kill.c (main): Likewise.
* src/od.c (main): Likewise.
* src/paste.c (collapse_escapes): Likewise.
* src/printf.c (print_formatted): Likewise.
This problem was discovered when trying to build git coreutils on
Solaris 8 sparcv9 with Sun C 5.8: the 3rd argument to mbrlen was
of type int * (a pointer to the gnulib replacement mbstate_t) but
the system mbrlen wants the system mbstate_t *.
* bootstrap.conf (gnulib_modules): Add mbrlen.
* src/pathchk.c (mbrlen, mbstate_t) [!(HAVE_MBRLEN && HAVE_MBSTATE_T)]:
Remove macros, which are wrong now that gnulib supplies
replacements for mbstate_t and mbrlen.
* bootstrap: Sync from gnulib. This removes an obsolescent
gettext.m4 patch, along with some other changes that do not
seem to affect coreutils.
* bootstrap.conf (gnulib_modules): Use gettext, not gettext-h.
Current gnulib gettext seems to work without needing special hacking.
* configure.ac (AM_GNU_GETTEXT_VERSION): Now 0.18.1, not 0.17.
* gnulib: Update to latest.
* bootstrap.conf (gnulib_modules): Add largefile.
This is useful to Mac OS X 10.5 users if/when configure
is generated using autoconf prior to v2.68-80-gdb2f2e0.
* gnulib: Update to latest.
* tests/du/inaccessible-cwd: Skip this test on systems like HP-UX 11.31
that lack both the *at functions and the /proc/self/fd-based support
we might have used to emulate them. Reported by Bruno Haible in
http://debbugs.gnu.org/8846
* src/join (g_names): New global (was main's "names").
(main): Update all uses of "names".
(line_no[2]): New globals.
(get_line): Increment after reading each line.
(check_order): Print the standard "file name:line_no: " prefix
as well as the offending line when reporting disorder.
Here is a sample old/new comparison:
-join: file 1 is not in sorted order
+join: in:4: is not sorted: contents-of-line-4
* tests/misc/join: Change the two affected tests to expect
the new diagnostic.
Add new tests for more coverage: mismatch in file 2,
two diagnostics, zero-length out-of-order line.
* NEWS (Improvements): Mention it.
Suggested by David Gast in http://debbugs.gnu.org/9236
This is especially important for an error-handling shell function
like this that is actually called only rarely.
* cfg.mk (sc_prohibit_framework_failure): New rule, to prevent
use of the old name.
These changes were mostly mechanical, made by running the
following command:
git grep -lw framework_failure | grep -v ChangeLog \
| xargs perl -pi -e 's/\b(framework_failure)\b/${1}_/'
and then editing init.cfg and `tests/cp/cp-a-selinux' by hand.
* tests/init.cfg (framework_failure): Remove, `framework_failure_'
from init.sh should be used instead in the tests.
Remove now-obsolete "FIXME" comment.
(is_local_dir_, require_strace_, require_membership_in_two_groups_,
require_sparse_support_, skip_if_mcstransd_is_running_,
mkfifo_or_skip_) Use `framework_failure_', not `framework_failure'.
* Many test scripts: Likewise.
* tests/init.cfg (framework_failure, getlimits_): Use 'fatal_'
instead of 'error_'.
(error_): Delete, it's not used anymore (and one is anyway
advised to use 'fatal_' instead).
Update heading comments.
* tests/shell-or-perl (error_): Renamed ...
(fatal_): ... to this, for consistency. Also, add a useful
comment.
* bootstrap.conf (gnulib_modules): Explicitly use the fclose module.
This is required, now that gnulib's close module no longer depends
on fclose. Without this module, we would be exposed to fclose bugs
affecting at least glibc 2.13 and FreeBSD.
Reported by Bernhard Voelker. Suggested by Eric Blake.
* src/copy.c (create_hard_link): A new function refactored
from existing code.
(copy_internal): Call the new function from all 3 locations
that create hard links.
* tests/cp/same-file: Amend to match the adjusted diagnostic.
* tests/cp/preserve-link: Add test cases for when a missing
link in the destination tree is encountered first and second.
Also add cases for old and new separate files in the destination
tree, both to make the clobbering behavior explicit, and to
test any changes in this area in future.
* src/copy.c (copy_internal): Adjust formatting style to conform with
guidelines in HACKING: put braces around two one-line "else" blocks.
* tests/cp/existing-perm-dir: Use $(...), not `...`, and
stat rather than ls+cut to get the mode string.
mode=$(stat --p=%A dst/dir)
* src/copy.c (copy_internal): If we don't create the directory,
then we cannot have omitted permissions. Problem and trivial
fix reported by Eric Lammerts.
* tests/Makefile.am (TESTS): Add cp/existing-perm-dir.
* tests/cp/existing-perm-dir: New file.
* src/copy.c (copy_internal): With --update (-u), this function would
return early once it found that the destination is not older than the
source, *without* recording the source-dev/ino--to--dest_name mapping.
That mapping is required in order to preserve src hard links in the
destination tree, so when using cp with --update and --preserve=links
(perhaps via -p or -a), cp could fail to preserve one hard link
per inode when at least one of the hard-linked names already exists
in the destination tree.
Reported by Odd Harry Mannsverk in http://debbugs.gnu.org/8419.
* tests/cp/preserve-link: New file. Exercise the flaw/fix.
* tests/Makefile.am (TESTS): Add it.
* NEWS (Bug fixes): Mention it.
* src/timeout.c (settimeout): A new function to convert
from a floating point duration and call alarm() or
timer_settime() if that's available.
(parse_duration): Return a double rather than unsigned int.
(usage): Mention floating point is supported.
(main): Pass the double to settimeout() rather than
calling alarm() directly with the parsed int.
(cleanup): Likewise.
* doc/coreutils.texi (timeout invocation): Say floating point timeouts
now supported, and mention the caveat with resolution.
* bootstrap.conf: Include the timer-time gnulib module.
* tests/misc/timeout-parameters: Add a test with nanoseconds.
* NEWS: Mention the improvement.
This change derives from improvements to gnulib's fsusage module.
* NEWS (Improvements): df now supports disk partitions larger than
4 TiB on MacOS X 10.5 or newer and on AIX 5.2 or newer.
Alphabetize entries.
* gnulib: Update to latest.
* m4/jm-macros.m4 (coreutils_MACROS): Use fdatasync only if declared.
MacOS X 10.7 has an fdatasync that is not declared, and is rumored to
be ineffective. (Bug#9141)
The following dropped the space from the first field
printf "1234567 \t1\n" | unexpand -a
Note POSIX says that spaces should not precede tabs.
Also a single trailing space should not be converted
if the next field starts with non blank characters.
So we enforce those rules too, with this change.
* src/unexpand.c (unexpand): Implement as per POSIX rules.
* tests/misc/unexpand: Add tests, and adjust existing
tests as per POSIX rules.
* NEWS: Mention the fix.
Reported by Hallvard B Furuseth
Treat fractions as a request to round up to the next representable
value, and treat out-of-range values as maximal ones. This is
consistent with how "sleep" works. And this way, "timeout
999999999999999999d FOO" and "timeout 4.5 foo" are more likely to
do what the user wants.
* src/timeout.c: Include c-strtod.h and xstrtod.h, not xstrtol.h.
(apply_time_suffix): Change it to the way sleep.c's time_suffix
does things. Maybe this function (identical in both programs,
other than its name) should be moved to a library?
(parse_duration): Return a maximal value on overflow. Return
unsigned int, not unsigned long. Allow fractions, which round
up to the next integer value.
* tests/misc/timeout-parameters: Adjust tests to match new behavior.
Add a very large number.
* src/csplit.c (interrupt_handler): Reset signal to SIG_DFL
after deleting the files, so that a second interrupt won't
prematurely terminate cleanup.
(main): Don't use SA_NODEFER | SA_RESETHAND, as that might
allow premature termination of cleanup. Also, this ports better
to platforms like NonStop, which don't ahve SA_RESETHAND.
* src/stat.c (print_stat): Use ST_BLKSIZE() rather than
accessing st_blksize directly, which is not present on
NonStop at least. Reported by Joachim Schmitz.
* src/fiemap.h (struct fiemap): Adjust the previous change
to the fiemap_extents array, which would also require changes
to the sizeof calculations in extent_scan_read().
Instead, only declare the fiemap_extents zero length array
on linux, which is the only platform that references this member.
This avoids a compilation failure on systems that don't support
this non standard construct. We don't use the equivalent C99
flexible array construct so as to have maximum portability.
* src/extent-scan.c: Cleanup. Remove a redundant #ifndef.
* src/mktemp.c: maint: avoid warning by using the comma operator
rather than an always-true conditional (as suggested by Eric Blake).
Reported by Joachim Schmitz in http://debbugs.gnu.org/9064.
This information has already been added to the Texinfo manual, but was
missing from the --help output.
* src/wc.c (usage): As above, for --help.
Reported by Vincent Lefevre in http://bugs.debian.org/395430.
* cfg.mk (sc_prohibit_man_see_also_period): Prohibit a period at
the end of the first line after a "SEE ALSO" marker in man/*.x.
With this, we shouldn't have to make any more changes like those
in today's commit, f2dabd68.
man-pages(7) explicitly says about SEE ALSO sections: "Do not terminate
this with a period." This is also in line with all the other man pages
in coreutils.
* man/cat.x: Remove period at the end of the SEE ALSO section.
* man/tac.x: Likewise.
* m4/jm-macros.m4: Define HAVE_SETRLIMIT.
* src/timeout.c: If the child exited with a signal,
raise that signal to the timeout process itself,
so that callers may also see the signal status.
Use setrlimit to disable core dumps for the timeout
process, which would be generated by some signals.
* src/timeout.c (cleanup): Send signals directly to the child
in case it has started its own process group (like a cascaded
timeout command would for example).
* test/misc/timeout-group: Add a test case.
* NEWS: Mention the fix.
Or more accurately, commands not started from the shell prompt,
that are interactive, or need to receive Ctrl-C etc. from the terminal.
* doc/coreutils.texi (timeout invocation): Document --foreground.
* src/timeout.c (main): Set the foreground flag and don't create
a separate group.
(cleanup): Only send a signal directly to the monitored command
when the foreground flag is set.
(usage): Describe --foreground.
* tests/misc/timeout-group: Add a new test.
* tests/Makefile.am: Reference new test.
NEWS: Mention the new option.
Reported by Shay Shimony
Analysis by Alan Curry
Fix suggested by Paul Eggert
This fact was already noted in the Texinfo manual, but not in the
output of --help.
* src/cp.c (usage): As above, for --help.
Reported by Jari Aalto in http://bugs.debian.org/294327.
Use this new option with --check when the input is expected to
consist solely of checksum lines. With only --check, an invalid
line evokes a warning, but the program can still exit successfully.
With --strict, any invalid line makes the program exit non-zero.
* src/md5sum.c (strict, STRICT_OPTION): Declare/define.
(long_options): Add "strict".
(usage): Describe --strict.
(digest_check): Count improperly_formatted lines, too, and use
that number and the global "strict" to determine the return value.
(main): Handle STRICT_OPTION.
Reject --strict without --check.
* doc/coreutils.texi: Describe it.
* NEWS (New features): Mention it.
* src/date.c (usage): As above, for --help.
* doc/coreutils.texi (Time conversion specifiers): Likewise.
Reported by Britton Leo Kerin in http://bugs.debian.org/115833.
* bootstrap.conf (gnulib_modules): Remove pathmax.
* src/system.h: Don't include "pathmax.h".
(PATH_MAX) [!PATH_MAX]: Define to 8192. Defining it to a constant
is preferable to using a definition from pathmax.h that might expand
to pathconf ("/", _PC_PATH_MAX). Prompted by discussion leading to:
http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/27183/focus=27269
* tests/ls/stat-free-color: The system may perform additional stat
calls upon loading (seen on OpenSuSE-11.4). Count only the number
of stat calls compared to --help.
This also reduces back to "1" the number of expected calls,
effectively reverting part of 2011-06-01 commit, ccf2d9a4.
* tests/init.sh (warn_): Use "sed 1q" in place of "head -1".
The latter is officially obsolete but more portable than "head -n1".
Reported by Bernhard Voelker.
* tests/shell-or-perl: Prefer the `read' builtin over `grep' to
look at the shebang line of test scripts. Since `read' is a
special builtin, it might abort the whole program upon failures,
so add extra sanity checks, verifying that the test script exists
and is readable, before trying to read from it.
This change implements a more correct and idiomatic use of the
features of the Automake-provided 'parallel-tests' harness.
Moreover, this change is required in order for the testsuite to
continue to work with the new testsuite harness that is planned
to be introduced in Automake 1.12 (which, as of the writing date,
is still under development and in alpha state).
* tests/shell-or-perl: New auxiliary script.
* tests/Makefile.am (EXTRA_DIST): Distribute it.
* tests/check.mk (TESTS_ENVIRONMENT): Remove definition of the
`shell_or_perl_' shell function, whose code has been moved in
the new script above (with a few improvements and extensions).
Do not use it to run the test scripts.
(LOG_COMPILER): New, properly invoking `shell-or-perl'.
* src/Makefile.am (pkglibexec_PROGRAMS): Rename from pkglib_PROGRAMS.
The latter is invalid. Without this change, automake
v1.11-373-g9ca6326 and newer (on master) would fail with this:
`pkglibdir' is not a legitimate directory for `PROGRAMS'
This changes the default installation directory of libstdbuf.so from
$prefix/lib/coreutils/ to
$prefix/libexec/coreutils/
* src/stdbuf.c (set_LD_PRELOAD): Search in PKGLIBEXECDIR, not PKGLIBDIR,
since that's where we install libstdbuf.so.
Do not search in "", the system default search path.
Ensure that English diagnostics are emitted even when using
French sorting rules.
* tests/misc/sort-debug-keys: Unset LC_ALL and set LC_COLLATE,
LC_CTYPE and LC_NUMERIC to the fr_FR.UTF-8 locale, while setting
LC_MESSAGES=C. Reported by Stefano Lattarini.
Using a .diff is much more maintainable. Otherwise, changes in
the gnulib module description file may not be noticed and merged
promptly and may even result in subtle errors. Luckily, this time,
the failure to propagate gnulib's changes to modules/tempname resulted
only in an obvious link failure.
* gl/modules/tempname: Remove file.
* gl/modules/tempname.diff: Use a .diff file instead.
* gnulib: Update submodule to latest.
* tests/init.cfg (skip_test_): Remove function.
Use skip_ in place of skip_test_ everywhere else.
* cfg.mk (sc_prohibit_skip_): Remove rule.
* tests/**: Use skip_, not skip_test_, everywhere.
* tests/init.sh (warn_): Use printf, not echo. The latter would
misbehave when given strings containing a backslash or starting
with e.g., -n. James Youngman suggested setting IFS.
Running "make check" normally prints a diagnostic to the outermost
stderr (usually a tty) to explain why a test is skipped. It did this
by redirecting FD 9 to stderr (via "exec 9>&2") before invoking the
shell script. Shell scripts write skip-explanation to FD 9 via
init.sh's skip_ function. However, with ksh and HP-UX's /bin/sh,
the effects of "exec 9>&2" are canceled upon fork-and-exec, so we
would get a "Bad file number" diagnostic and no skip explanation on
those systems.
* tests/check.mk (TESTS_ENVIRONMENT): Redirect more portably, via
"$(SHELL) 9>&2", rather than the prior "exec 9>&2; $(SHELL) ..."
Actually, we use "shell_or_perl_ 9>&2", to make this effective
also for the perl-based tests.
* tests/init.sh (stderr_fileno_): Update the advice in comments.
See http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/22488
for lots of discussion. Stefano Lattarini suggested the solution
of putting "9>&2" after the command. Reported by Bruno Haible.
* gnulib: Update to latest.
* src/system.h: Definitions of ST_* macros have moved into the
gnulib module stat-size (specifically, the header file
stat-size.h), so remove them from here.
* src/truncate.c: Include stat-size.h.
* src/stat.c: Likewise.
* src/shred.c: Likewise.
* src/ls.c: Likewise.
* src/du.c: Likewise.
* src/ioblksize.h: New file. Move definition of io_blksize out of
system.h so that system.h does not have to include stat-size.h.
* src/cat.c: Include ioblksize.h.
* src/split.c: Likewise.
* src/copy.c: Include both stat-size.h and ioblksize.h.
* src/Makefile.am (noinst_HEADERS): Add ioblksize.h.
* tests/dd/nocache: Relax the test, as the system
may return various errors from posix_fadvise().
HPUX 11.31 returns ENOTTY for example.
Reported by Bruno Haible
* src/date.c (usage): Add examples for TZ handling,
and "seconds since epoch" parsing, neither of which
was mentioned in the man page until now.
* THANKS.in: Add Rick.
Suggested by Rick Stanley.
* configure.ac: Require autoconf-2.64, which is nearly two years old.
* src/system.h (emit_ancillary_info): Use PACKAGE_URL, now that we
require autoconf-2.64.
* tests/ls/stat-free-color: This test recently began to fail on
rawhide because dynamic library start-up code now stats "/selinux",
making the total number of calls 2 rather than the prior 1.
Create two more dangling symlinks, so that any erroneous stat-
or lstat-calling code will get at least those three.
I think it would be better to exit through the error() and not
to call the exit() after the error(). This way we can get rid of
one function call (and curly brackets).
* src/yes.c (main): Exit through the error(), remove exit() call
after error().
* src/tail.c (start_bytes): Increase *READ_POS (not READ_POS)
by the number of bytes read. This is a real bug that happens
to have no consequence in practice. First, this code is exercised
only when tailing-forever a non-regular file by bytes, and with a
start-relative offset, e.g., "mkfifo f; tail -f -c +3 f", but even
then, the invalid READ_POS value does not influence how tail works.
It is stored in the File_spec.size member, but that member is not
used at all in tail_forever_inotify, and in tail_forever, it is
used only when the File_spec refers to a regular file.
* src/chown-core.c (describe_change): Output the
original owner if possible.
(user_group_str): Handle the case when neither
owner or group are passed.
* NEWS: Mention the change in behavior.
* src/chown_core.c (describe_change): Accept the ownership of
the original file and output that when not changing.
This is significant when --from is specified as then
the original and specified ownership may be different.
(user_group_str): A new helper function refactored from
describe_change().
(change_file_owner): Pass the original user and group
strings to describe_change().
* test/chown/basic: Add a test case.
* NEWS: Mention the fix.
* configure.ac (WARN_CFLAGS): Don't turn off -Wstrict-overflow.
(GNULIB_WARN_CFLAGS): Remove -Wstrict-overflow from the list of
warning options used in lib/.
Normally I find that -Wstrict-overflow produces too many false
positives, but considering that it warns of the bug reported in
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=33498, I now think
it is worthwhile. The lesser of two evils.
Thanks to Daniel Veillard for showing me the gcc bug report.
* src/factor.c (factor_using_pollard_rho): Change type of "i"
to unsigned to avoid warning from gcc's -Wstrict-overflow.
* src/expr.c: Use an unsigned intermediate.
* src/dircolors.c (main): Reorder operations to avoid the risk of
pointer overflow.
* src/tr.c (squeeze_filter): Change NOT_A_CHAR from an anonymous
"enum" to an "int", to avoid this warning:
tr.c:1624:10: error: assuming signed overflow does not occur when
simplifying conditional to constant [-Werror=strict-overflow]
* src/pr.c (main): Make index "i" unsigned.
* cfg.mk (sc_preprocessor_indentation): New test, from libvirt.
Exempt 3 files from new cppi test.
* gl/lib/randread.c: Adjust cpp indentation to comply.
* src/extent-scan.c (extent_need_sync): Likewise.
* src/shred.c (incname): Add an assertion to tell static analyzers
that we know this particular use of strchr never returns NULL.
Finish incomplete sentence in function-describing comment.
* src/split.c (lines_chunk_split): Don't use ignore_error() which
is redundant and confusing when not running with --filter.
(lines_rr): Likewise.
(ofile_open): Likewise. Add a comment to clarify that
filters aren't restarted under file descriptor pressure.
* src/split.c (main): Exit with a diagnostic if --filter
is specified along with a specific chunk number.
* test/split/filter: Ensure this combination fails.
* src/split.c (bytes_split): Stop reading when we
can no longer write to a child process.
(lines_rr): Likewise.
(lines_bytes_split): No change is made here since
input is bounded by the original file size.
* test/split/filter: Add test cases.
src/split.c (main): Don't unblock SIGPIPE before cleanup,
as then any pending signals will be sent and cause
the main split process to exit with a non zero status (141).
* test/split/filter: Add a test for this case.
* src/split.c (lines_bytes_chunk): Handle the edge case
where the file is truncated as we read.
* tests/misc/split-lchunk: Cleanup; no functional change.
* src/chmod.c (describe_change): Pass in the original mode,
and output this in the messages.
* tests/chmod/c-option: Adjust as per the new message.
* THANKS.in: Remove the now auto-generated name.
* NEWS: Mention the change in behavior.
Signed-off-by: Pádraig Brady <P@draigBrady.com>
* cfg.mk: Include via "-include", to accommodate new tight-scope rule.
(sc_check-AUTHORS): Change the name of the rule in src/Makefile.am
to _sc_check-AUTHORS, so it doesn't conflict with this one when
this file is included into the sub-make's context.
* src/Makefile.am (_sc_check-AUTHORS): Rename from sc_check-AUTHORS.
* gnulib: Update to latest.
* src/touch.c (main): Avoid even the hint of possibility that
we'd dereference NULL upon localtime failure. Coverity reported
the potential, but it appears not to be possible, since posixtime
rejects any time for which the subsequent localtime would return NULL.
See http://thread.gmane.org/gmane.comp.gnu.coreutils.general/1253
* src/split.c (lines_chunk_split): Ensure that data is only
written to stdout when k specified. Also ensure that
extra files are not created when there is more data available
than reported in the file size.
* tests/misc/split-lchunk: Verify that split -n l/k/n doesn't
generate any files, and that -n l/n always generates n files.
* NEWS: Mention the fix.
* src/timeout.c (usage): Add a space to be consistent
with other uses of "(the default)" in the documentation.
Signed-off-by: Pádraig Brady <P@draigBrady.com>
* tests/misc/tac-continue: Fix typo in usually-skipped test:
s/mkfifo_or_skip/mkfifo_or_skip_/ (i.e., append "_").
This test is usually skipped, because I'm probably the only
one to set the FULL_PARTITION_TMPDIR envvar, and recently the
one I'd been using ceased to exist, so this test was skipped
even for me. Good argument for making this a root-only test
and creating a full partition just for this test case.
* NEWS: "an misleading"
* src/expr.c: "a integer
* src/ptx.c (find_occurs_in_text): "a end"
* src/shred.c (do_wipefd): "a infinite"
* src/sort.c (SUBTHREAD_LINES_HEURISTIC): "an dual-core"
(compare_random): "an checksum"
* cfg.mk (old_NEWS_hash): Update, since the typo was in old news.
* src/printf.c (STRTOX): Don't access memory after a
string containing a single quote character.
* tests/misc/printf: Add tests for various combinations
of single quote characters combined with a numeric format.
* THANKS.in: Add bug reporter.
* NEWS: Mention the fix.
Reported-by: Paul Marinescu <paul.marinescu@imperial.ac.uk>
* gl/lib/randperm.c (randperm_new): When the number of items
to return H, is much smaller than the total number of items N,
use a hash to represent the sparse permutations of the set N.
This is currently enabled for N > 128K and N/H > 32.
* tests/misc/shuf: Ensure shuf can quickly return 2 numbers
from a large range.
* gl/modules/randperm: Depend on hash.
* NEWS: Mention the change.
skip_test_ emits its diagnostic both to FD 9 (tty), and to
FD 2 (usually the log file), whereas init.sh's skip_ emits
only to FD 9. Without that, the log is slightly less useful.
* tests/cp/fiemap-2: Use skip_test_, not skip_.
* tests/cp/fiemap-perf: Likewise.
* tests/du/bigtime: Likewise.
* tests/du/files0-from-dir: Likewise.
* tests/du/move-dir-while-traversing: Likewise.
* tests/init.sh: Likewise.
* tests/misc/sort-stale-thread-mem: Likewise.
* tests/misc/stat-nanoseconds: Likewise.
* tests/mv/i-3: Likewise.
* tests/mv/sticky-to-xpart: Likewise.
* tests/split/filter: Likewise.
Prompted by a report from Pádraig Brady.
Even on a system with d_type support, the default use of --color
makes ls stat every file in order to be able to honor settings like
EXEC, STICKY, ORPHAN, SETUID, etc., because those settings require
information that is not provided by dirent.d_type. However, if
for a potentially large performance gain, you are willing to disable
those settings, you can now make ls --color give type-related coloring
and perform no stat calls at all (other than the unavoidable call-per-
command-line argument). Before this change, even with all of those
attributes disabled, ls --color would still stat every directory.
Now, we're down to the minimum of one stat call per command-line arg.
* src/ls.c (gobble_file): With --color, don't stat a
non-command-line-specified directory when no directory-coloring
attribute is enabled.
* tests/init.cfg (require_dirent_d_type_): New function.
* tests/d_type-check: New script, mostly from Pádraig Brady.
* tests/Makefile.am (EXTRA_DIST): Add it.
* tests/ls/stat-free-color: New test.
* tests/Makefile.am (TESTS): Add it.
* doc/coreutils.texi (General output formatting): Describe how
to use dircolors to make ls --color refrain from calling stat
on a d_type-enabled file system.
Prompted by a query from Josef Bacik.
gnulib's group-member module now ensures that the group_member
function is declared in <unistd.h>, just like it is glibc.
* lib/euidaccess-stat.c: Remove inclusion of "group-member.h".
* src/chgrp.c: Likewise.
* configure.ac: Rather than disabling -Wmissing-field-initializers,
use the fact that gnulib now disables it automatically when required
(on versions of GCC older than 4.7).
* src/system.h: Remove the no longer needed DECLARE_ZEROED_AGGREGATE.
* src/ls.c: Likewise.
* src/pathchk.c: Likewise.
* src/shred.c: Likewise.
* src/stty.c: Likewise.
* src/wc.c: Likewise.
* src/split.c: Include <signal.h>, <sys/wait.h> and "sig2str.h".
(FILTER_OPTION): New anonymous enum member.
(filter_command, filter_pid): New globals.
(open_pipes, open_pipes_alloc, n_open_pipes): Likewise.
(oldblocked, newblocked): Likewise.
(longopts): Add "filter".
(usage): Document --filter.
(create): Extend to create a pipe and fork "sh -c CMD".
(closeout): Adapt to close a pipe and wait for child process.
(cwrite): Call closeout, not just close.
(lines_chunk_split): FIXME
(bytes_chunk_extract): FIXME
(opid, ofile_open, lines_rr, main): FIXME
(ignorable): New function, to encapsulate EPIPE test.
* src/sort.c (key_warn): `sort -k2,1n --debug` would output
warnings about being both "zero width" and "spanning multiple fields".
Suppress the latter one.
* tests/misc/sort-debug-warn: Add a couple of test cases.
* src/copy.c (copy_reg): Fix my typo (mis-applied patch).
The patch by Jeff Liu was fine, but I mis-applied it
and introduced a compilation error in commit efa479c1.
2011-05-03 Jim Meyering <meyering@redhat.com>
Running "GZIP=-9 ./bootstrap" would fail right away, because the
tool-version-checking code would treat the upper-cased program name
as an environment variable name and if that has a value use the
result as the application name. That works fine for automake,
autoconf, etc. but not for gzip.
* bootstrap (check_versions): Do not treat $GZIP as a program name.
If defined at all, it is supposed to list gzip options.
Reported by Alan Curry in http://debbugs.gnu.org/8609
* tests/Makefile.am (TESTS): Remove misc/pwd-unreadable-parent.
This test was misleading and useless (was always skipped).
Inspired by a report from Bruno Haible: http://debbugs.gnu.org/8570
* tests/misc/pwd-unreadable-parent: Remove file.
* tests/CuSkip.pm (skip): New file/module/function, to help
the perl test scripts "skip" a test consistently, emitting
a diagnostic both into the log file and into the outermost
stderr stream that is more likely to be seen by a human.
* tests/check.mk (TESTS_ENVIRONMENT): Add -MCuSkip.
* tests/misc/date-next-dow: Use CuSkip::skip in place of warn+exit-77.
* tests/misc/tty-eof: Likewise.
* tests/misc/uniq: Likewise.
* tests/rm/fail-eperm: Likewise.
* tests/misc/md5sum-newline: Likewise. Also, s/program_name/ME/.
* tests/misc/ls-misc (setuid_setup, main): Likewise.
* tests/misc/pwd-long: Likewise, and add -I"$abs_srcdir" -MCuSkip
to the $PERL invocation command.
Inspired by a request from Bruno Haible regarding misc/tty-eof:
http://debbugs.gnu.org/8570
* src/dd.c (O_NOCACHE): Undefine. This symbol is defined
via AIX's <fcntl.h>, yet used as an enum name in dd.c.
Reported by Gary V. Vaughan in http://debbugs.gnu.org/8555
* NEWS (Portability): Mention this.
* src/tail.c (tail_forever_inotify): Just as without inotify,
tail --follow=name now terminates when the last tailed-by-name file
is unlinked or moved aside. This bug was introduced on 2009-06-15
via commit ae494d4b, "tail: use inotify if it is available".
Reported by Tim Underwood in
http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/22286
* NEWS (Bug fixes): Mention this.
* tests/tail-2/follow-name: Test for this.
* tests/Makefile.am (TESTS): Add it.
* src/copy.c (copy_reg): Upon btrfs clone failure, print not just
the destination file name, but also the source file name.
That may be useful upon failure of a cross-device clone attempt.
Otherwise, this would fail (albeit rarely) on a "make -j24 check" run.
* tests/tail-2/pipe-f2: Increase timeout from 1 second to 10,
to avoid false positive failure.
* bootstrap.conf (gnulib_modules): Include full-read and full-write
explicitly. Before, we'd get them via safe-read, but with newer
gnulib, that is no longer enough: link failure due to undefined
references to full_write.
* tests/cp/sparse-fiemap: Parenthesize ternary expression used
as an argument to awk's printf. Otherwise, gawk 3.0.1 and the
one from debian stable's original-awk would get a syntax error.
Reported by Dennis Clarke.
Copyright note: tiny change
* tests/cp/sparse-fiemap: When this test was run as root on an ext3
file system, (ext3 had known problems), it would trickily create and
mount a loopback ext4 file system and use that instead. However, due
to a bug in 2.6.39-rc1..rc3, this loopback test (when run in another
loopback FS) exposed a bug with 1k-blocksize ext4 whereby non-NUL
data would be read from a hole. For details, see this:
http://thread.gmane.org/gmane.comp.file-systems.ext4/24495
to avoid the expense of extent_copy's unconditional use of
FIEMAP_FLAG_SYNC.
* src/copy.c (copy_reg): Do not attempt extent_copy on a file
that appears to have no holes.
* NEWS (Changes in behavior): Document this. At first I labeled this
as a bug fix, but that would be inaccurate, considering there is no
documentation of FIEMAP semantics, nor even consensus among kernel
FS developers. Here's hoping SEEK_HOLE/SEEK_DATA support will soon
make it into the linux kernel.
* src/copy.c (HAVE_STRUCT_STAT_ST_BLOCKS): Define to 0 if undefined,
so we can use it in the return expression, here:
(is_probably_sparse): New function, factored out of...
(copy_reg): ...here. Use the new function.
* src/copy.c (extent_copy): Do not treat "unwritten extents" specially.
Otherwise, with a release-candidate 2.6.39-rc3 kernel, XFS or ext4,
when using gold as your linker, and if you forget to run "make check",
you could end up installing files full of zeros instead of the expected
binaries. For a lot of discussion, see
http://thread.gmane.org/gmane.comp.file-systems.xfs.general/37895
* tests/cp/fiemap-empty: Disable this test.
* src/extent-scan.c (extent_need_sync): Always return true,
to make the sole caller always use FIEMAP_FLAG_SYNC.
This will doubtless have an undesirable performance impact,
but we'll mitigate that shortly, by using extent_copy only on
files with holes.
* tests/filefrag-extent-compare: Don't check the length of the
last extent, as this was seen to vary on XFS, where it leaves
trailing blocks allocated for performance reasons.
* tests/cp/fiemap-empty: Though not seen as an issue in practise,
try to avoid possible issues with the allocator in file systems,
by requesting to allocate a power of 2.
* NEWS: Slightly obfuscate a line to avoid a false-positive
doubled-word ("is-is") match.
Fix a grammar error in news for 8.2.
* cfg.mk (old_NEWS_hash): Resync.
and to avoid a false-positive "TO to" in new doubled word check.
* src/install.c (change_timestamps): Rename parameters for
readability. Make the comment match the code.
* extent-scan.h (extent_scan_free): Init the pointer to NULL,
and reset the count to 0, so that we can realloc the buffer.
* src/extent-scan.c (extent_scan_init): Likewise.
(extent_scan_read): Loop over multiple fiemap scans, so we handle
mergeable extents that span across fiemap scan boundaries. Once
we have enough unique extents, return so as to minimize memory use.
Running the new fiemap-empty test uses 600MB of disk space via
fallocate, and in so doing caused failure in unrelated tests that
were running in parallel on a small file system. Rather than
simply running fallocate (which allocates the space, inducing
disk full when it fails), skip the test if there is less than
800MB of free space, as computed via stat and awk.
* tests/init.cfg (require_file_system_bytes_free_): New function.
* tests/cp/fiemap-empty: Use it.
* tests/misc/help-version: Sleep only ~30s, not 10m.
The latter was a problem when somehow that sleep process would
hang around and thereby prevent (for up to 10m) a normal unmount
of the temporary partition in which I'd run the tests.
* tests/cp/preserve-gid: Simply chmod a+rx instead.
That is safer, in case the nameless UID actually has an account,
and might take advantage of root running a program in a directory
under its control
* src/Makefile.am (sc_tight_scope): Adjust rule to use an eval-based
trap-setting for-loop rather than 4x hard-coded 128+N constants.
Also catch SIGQUIT (3). Tweak comments.
* src/extent-scan.c (extent_need_sync): Require sync also for 2.6.38.
Without this, part of the cp/fiemap-empty test would fail both on
F15-to-be (2.6.38.1-6.fc15.x86_64) and rawhide. For details, see
http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/22190
When running the erroneous command, cp -rl A D D, and depending on the
structure of directories A and D and the file system type (because that
changes order of dir. entry traversal), cp would sometimes fail to
detect that D was being copied into D, and would create D/D/D/D/D/...
until it hit PATH_MAX or exhausted some resource.
I noticed this via the occasional failure of the cp/into-self test
when run using a ZFS file system. It is occasional because the bug
is dependent on the order in which directory entries are traversed,
and that is apparently indeterminate with ZFS.
Technically, with the current recursive implementation, there is no
risk of an infinite loop, due to stack limitations, but with an
eventual fts-based implementation, it might have iterated until
disk space or inodes are exhausted.
* src/copy.c (copy_dir): Avoid copy-into-self interminable loop on
systems with large PATH_MAX. On other systems, diagnose the copy-into-
self error consistently. Handle the parameter,
first_dir_created_per_command_line_arg, correctly when there are two
or more sub-directories.
* src/copy.c (extent_copy): Treat an allocated but empty extent
much like a hole. I.E. don't read data we know is going to be NUL.
Also we convert the empty extent to a hole only when SPARSE_ALWAYS
so that the source and dest have the same allocation. This will
be improved soon, when we use fallocate() to do the allocation.
* tests/cp/fiemap-empty: A new test for efficiency and correctness
of copying empty extents.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the change in behavior.
* src/extent-scan.c (extent_scan_read): Add a more stringent check
for OFF_T overflow, to ensure subsequent code is immune.
Detect overlapping extents and adjust, so as files always copied.
Detection using a single scan with fallback to a standard copy
was thought too expensive in memory or time.
* NEWS: Mention the fix
This bug was introduced in commit ca9e212c, 2009-09-24,
"cp, mv: use linkat to guarantee semantics", which
inadvertently disabled the creation of hardlinks to symlinks.
However rather than implementing the intention of that commit
and relying on gnulib linkat emulation, we'll revert to the
previous emulation as that maintains ownership and timestamps.
* src/copy.c (copy_internal): Use our existing hardlink to
symlink emulation when link() might dereference the symlink.
Also ensure that we copy the timestamps of the original symlink
when we use the emulation.
* tests/cp/link-symlink: Add a test to ensure timestamps copied.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the fix.
Reported by Ruediger Meier
* tests/tail-2/inotify-rotate: Wait 50% longer for grep to succeed.
Without this change, this test would fail consistently when using
"make -j25 check" with F15 in a virtio- and spinning-rust-backed
virtual machine.
* src/extent-scan.h (struct extent_scan): Add the fm_flags member to
pass to the fiemap scan.
* src/extent-scan.c (extent_need_sync): A new function used to
detect Linux kernels before 2.6.38.
(extent_scan_init): Add FIEMAP_FLAG_SYNC when needed.
* tests/cp/sparse-fiemap: Adjust comment.
* NEWS: Mention the change in behavior.
Indirectly suggested by Mike Frysinger
* tests/init.cfg (require_selinux_): Skip the test also when
/proc/filesystems does not list selinuxfs.
Add comments.
* cfg.mk (exclude_file_name_regexp--sc_file_system): Exempt
tests/init.cfg, with its use of /proc/filesystems.
Based on the patch by Mathieu Bridon in http://debbugs.gnu.org/8359.
More discussion in http://bugzilla.redhat.com/573111
* src/find-mount-point.h: Move "*" to where it belongs.
Move "const", too.
* src/find-mount-point.c: Move "const" to conform.
* src/Makefile.am (sc_tight_scope): Allow `*'s before the function name.
Use perl's -l option and drop the \n after (and quotes around) $1.
* src/df.c (alloc_table_row): A new function to allocate storage
for a row of strings.
(print_table): A new function to interate over all stored strings in
the table, and apply alignment honoring the max width of each column.
(get_header): Renamed from print_header, and adjusted accordingly.
(get_dev): Renamed from show_dev. Also we no longer wrap longer
device names over two lines, which can be an unexpected issue for
scripts parsing the output from df.
(get_disk): s/show_/get_/
(get_point): Likewise.
(get_entry): Likewise.
(get_all_entries): Likewise.
* NEWS: Mention the change.
Make GNU coreutils' test recognize "==" as a synonym for "=".
This is already the case in GNU coreutils' expr, bash, ksh,
busybox ash, FreeBSD-current /bin/sh and /bin/test, and
OpenBSD's /bin/sh.
Before, env test a '==' a would fail with this diagnostic:
"test: ==: binary operator expected". Now, it succeeds.
* src/test.c: Accept "==" as a synonym for "=".
* doc/coreutils.texi (String tests): Document it.
Reported as http://debbugs.gnu.org/8263
Also see http://austingroupbugs.net/view.php?id=375
* tests/cp/preserve-gid: Ensure that every process under test uses
the cp binary we've just built. Before this fix, with a restrictive
umask or build-dir permissions, the UID-changing tests would end up
using whatever cp happened to be available through $PATH
Analysis by arbogast.cedric@gmail.com in http://debbugs.gnu.org/8292.
The names in THANKS are generated from two sources: the hard-coded
list, THANKS.in, and the names of committers from the git log.
When a contributor on the hard-coded list commits a change,
we remove their now-redundant name from THANKS.in.
* THANKS.in: Remove a now-duplicate name.
* tests/filefrag-extent-compare: Merge adjacent extents in
each list before processing, so we correctly account for
split extents in either list.
* tests/cp/sparse-fiemap: Remove the explicit syncing,
which was only changing the way extents were arranged,
and thus working around the extent comparison issue
that was seen on ext4 loop back.
* gl/lib/mbsalign.c (rpl_wcswidth): Remove this in favor
of the equivalent wcswidth replacement in gnulib.
* bootstrap.conf: Depend on the wcswidth module.
Suggested by Bruno Haible.
* gnulib: Update to latest, to address http://debbugs.gnu.org/8230.
When built on Solaris 9 and run on Solaris 10, touch would segfault.
Reported by Ben Walton.
* bootstrap: Update from gnulib.
* tests/init.sh: Likewise.
* NEWS (Bug fixes): Mention this.
* src/sort.c (SUBTHREAD_LINES_HEURISTIC): Do not spawn a new thread
for every 4 lines. Increase this from 4 to 128K. 128K lines seems
appropriate for a 5-year-old dual-core laptop, but it is too low for
some common combinations of short lines and/or newer systems.
* NEWS (Bug fixes): Mention it.
* src/extent-scan.c (extent_scan_read): Merge adjacent extents
that vary only in size, so that we may process them more efficiently.
This will be especially useful when we introduce fallocate()
so that we don't reproduce fragmentation in the destination.
* src/dd.c (warn_partial_read): New static var.
(iread): Diagnose partial reads if needed.
(iwrite): Don't diagnose them here; not needed any more.
(scanargs): Determine whether partial reads should be diagnosted.
* src/dd.c (FFS_MASK): A new macro (Find First Set) refactored
from the following enum as it's now used twice.
(usage): Mention the new 'nocache' flag.
(cache_round): A new function to help ignore requests
to drop cache, that are less than page_size.
(invalidate_cache): A new function to call posix_fadvise()
with the appropriate offset and length. Note we don't
use fdadvise() so we can detect errors when count=0.
(dd_copy): Call invalidate_cache() for the portions read.
(iwrite): Likewise for the portions written.
(main): Call invalidate_cache for page_size slop or
for full file when count=0.
* cfg.mk (sc_dd_O_FLAGS): Adjust to pass.
* doc/coreutils.texi (dd invocation): Describe the 'nocache' flag,
and give some examples of how it can be used.
* tests/dd/nocache: A new test.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the new feature.
* doc/coreutils.texi (dd invocation): Clarify that bs= can
cause parital reads to be immediately written to output.
* src/dd.c (usage): Hint that bs= can cause partial writes.
See http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8171
* src/dd.c (usage): Move 'sync' up with other data transformation
options. Having it alongside 'fsync' and 'fdatasync' is
particularly confusing. Also the double line description of
the 'sync' option, serves as a visual break from the "flag"
type options that follow.
* doc/coreutils.texi (dd invocation): Apply the same grouping
as above, by splitting the "conv=" table in two.
An alternative to this is to auto enable iflag=fullblock
when oflag=direct and bs= is specified.
It was thought better though, to warn about the specific issue,
and give full control of dd's options to the user.
* src/dd.c (iwrite): Warn, when we write after having
disabled O_DIRECT.
See https://bugzilla.redhat.com/show_bug.cgi?id=614605
* src/du.c (main): Fail on AI_ERR_READ error, rather than merely
diagnosing and continuing. Based on a patch by Stefan Vargyas.
Also move the handling of AI_ERR_EOF into the case stmt.
Do not report ferror/fclose(stdin) failure when we've
already diagnosed e.g., failure to read the DIR, above.
Bug introduced by 2008-11-24 commit 031e2fb5, "du: read and
process --files0-from= input a name at a time,".
* src/wc.c: Handle read failure as with du: do not exit
immediately, but rather go on to print any total and to clean-up.
As above, move the handling of AI_ERR_EOF into the case stmt.
* tests/du/files0-from-dir: New file, to test both du and wc.
* tests/Makefile.am (TESTS): Add it.
* NEWS (Bug fixes): Mention it.
* src/wc.c (main): Diagnose failed argv_iter_init_* failure,
rather than falling through and dereferencing NULL.
Bug introduced by 2008-11-25 commit c2e56e0d,
"wc: read and process --files0-from= input a name at a time,".
* NEWS (Bug fixes): Mention it.
* tests/cp/sparse-fiemap: Move the PERL check to the top,
since we don't test anything without it. In the loop,
don't use skip_test_ as it exits the test completely.
There was an execution path by which "libstdbuf" could be used after
being freed, but that would happen only if there were no libstdbuf.so
alongside the stdbuf program and there had been an installation error
leading to absence of the file, PKGLIBDIR/libstdbuf.so.
* src/stdbuf.c (set_LD_PRELOAD): Rearrange loop to make it perfectly
clear that there is no possibility of use-after-free.
Steve Grubb reported this possible use-after-free of "libstdbuf".
* src/uptime.c (print_uptime): Omit unnecessary "#if defined
HAVE_GETLOADAVG || defined C_GETLOADAVG". This #if is always
true, and removing it will help us simplify the gnulib getloadavg
module.
* tests/cp/sparse-fiemap: Check for fiemap support against a file
rather than a directory to enable tests on BTRFS for example.
Explicity disable the test on ext3 or file systems where we
can't determine the type.
* tests/cp/fiemap-perf: Likewise. Also disable the test on older
BTRFS (like in Fedora 14), where extents are returned for holes.
* tests/init.cfg: Comment that BTRFS only supports fiemap
for regular files.
Don't depend on heuristics to detect sparse files
if fiemap is available. Also don't scan for new
holes unless --sparse=always has been specified.
* src/copy.c (extent_copy): Pass the user specified
sparse mode, and handle as described above.
Do not run a sub-make to set up the environment for each
and every test script. Instead, run it just once and store
the result in a file.
* tests/check.mk (built_programs): Remove definition.
(.built-programs): New rule to create the temporary file.
(CLEANFILES): Arrange to remove it.
(TESTS_ENVIRONMENT): Simply cat .built-programs, rather than
running the sub-make.
* .gitignore: Ignore it.
* tests/cp/fiemap-2: Enable the fiemap check for files, which
will enable the test for files on ext3.
* tests/cp/fiemap-perf: Comment why we're not enabling for ext3.
* tests/cp/sparse-fiemap: Ditto. Also sync the files before
doing a fiemap which was needed for ext4 loop back at least.
Add a comment that FIEMAP_FLAG_SYNC is ineffective, thus
requiring the explicit syncs.
* tests/fiemap-capable: A new python script to determine
if a specified path supports fiemap.
* tests/init.cfg (fiemap_capable_): Use the new python script.
* tests/Makefile.am (EXTRA_DIST): Include the new python script.
This is required for patch, and hence is about to move to gnulib.
* gl/lib/di-set.c (di_set_lookup): New function.
* gl/lib/di-set.h: Declare it.
* gl/tests/test-di-set.c (main): Exercise it.
The bug was introduced on 2004-12-04 via commit 7380cf79.
* src/cut.c (set_fields): When computing the maximum range endpoint,
take into consideration the start of any unbounded range, like "999-".
* NEWS (Bug fixes): Mention it.
* tests/misc/cut (big-unbounded-b,c,f): Add tests.
Reported by Paul Marinescu in http://debbugs.gnu.org/7993
The bug was introduced on 2004-12-04 via commit 7380cf79.
Upon failed lseek, sparse_copy_finalize would mistakenly return true.
Admittedly, that is very unlikely, since that particular lseek
is attempted only if the preceding call to sparse_copy induced
a hole at EOF (via lseek on the destination FD). However, now
that sparse_copy has an output parameter, N_READ, there is no
longer any reason to call lseek (fd, 0, SEEK_CUR), so...
* src/copy.c (sparse_copy_finalize): Remove the function.
(copy_reg): Call ftruncate with n_read, rather than
sparse_copy_finalize with its now-unnecessary lseek.
Lasse Collin spotted the bug in sparse_copy_finalize.
This case was overlooked in commit bdde34f9, 2010-08-05,
"sort: tune and refactor --debug code, and fix minor underlining bug"
* src/sort.c (debug_key): Don't adjust the key end when
it's before the key start.
* tests/misc/sort-debug-keys: Add a test case.
* tests/cp/fiemap-perf: Correct erroneous added test.
Since nonexistent names were used, the final test ended up
being "test =", which would always "succeed".
* src/copy.c (write_zeros): This bug caused 4 or 8 bytes to
be written at a time which is very inefficient. One could
trigger the issue with `cp --sparse=never sparse non-sparse`
on a file system that supports fiemap.
The recent FIEMAP-related changes made it so the unusual case of
copying a sparse file to a non-regular destination (e.g., a pipe)
would erroneously write one byte too many to that destination.
That happened because extent_copy assumed that it could use lseek
to obtain the number of bytes written to the output file descriptor.
That was valid only for regular files.
* src/copy.c (sparse_copy): Add a parameter, to be used by extent_copy,
but not by reg_copy. Adjust callers.
(extent_copy): Maintain new local, dest_pos, using new arg, n_read.
Don't call lseek on dest_fd; use new var, dest_pos, instead.
(copy_reg): Add unused arg.
* tests/cp/fiemap-perf: Copy block-comparing code from sparse-fiemap.
* tests/cp/sparse-fiemap: The same test was here, alongside a much
more involved test. Remove it, now that it is in its own file.
Don't use "#ifdef EINTR". dd.c has been doing that since 2004.
* src/copy.c (sparse_copy): Remove #ifdef...#endif around EINTR use.
* src/tee.c (tee_files): Remove #ifdef...#endif around EINTR use.
If we need it, add something like this in system.h:
/* When EINTR is not defined, define it to an improbable value
so that each use does not have to be #ifdef'd. */
#ifndef EINTR
# define EINTR 999988
#endif
* src/copy.c (extent_copy): Before this change, extent_copy would fail
to create holes, thus breaking --sparse=auto and --sparse=always.
I.e., copying a large enough file of all zeros, cp --sparse=always
should introduce a hole, but with extent_copy, it would not.
we're going to have to use it from within extent_copy, too.
* src/copy.c (sparse_copy): New function, factored out of...
(copy_reg): ...here.
Remove now-unused locals.
* src/copy.c (extent_copy): Don't let what should have been a
temporary reduction of buf_size (to handle a short ext_len) become
permanent and thus impact the performance of all further iterations.
* src/copy.c (copy_reg): Move use of extent_scan to just *after*
we allocate the main copying buffer, so we can...
(extent_scan): Take a new parameter, BUF, and use that rather
than allocating a private buffer. Update caller.
* src/copy.c (copy_reg): Rename a variable to make more sense from
caller's perspective: s/require_normal_copy/normal_copy_required/.
This is an output-only variable, and the original name could make
it look like an input (or i&o) variable.
so that we benefit from using extents also when reading a sparse
input file with --sparse=never.
* src/copy.c (copy_reg): Remove erroneous test of "make_holes"
so that we call extent_copy also when make_holes is false.
Otherwise, what's the point of that parameter?
* src/extent-scan.h [struct extent_scan]: Rename member:
s/hit_last_extent/hit_final_extent/. "final" is clearer,
since "last" can be interpreted as "preceding".
Rename extent-scan functions to start with extent_scan_.
* src/Makefile.am (copy_sources): Also distribute extent-scan.h.
* src/extent-scan.c: Don't include error.h or quote.h. Neither is used.
* src/copy.c: shorten a comment to fit in 80 columns
* src/extent-scan.c, src/extent-scan.h: Correct formatting.
* src/copy.c (write_zeros): Add comments.
(extent_copy): Move decls of "ok" and "i" down to scope where used.
Adjust comments.
Rename local: s/holes_len/hole_size/
Print a diagnostic upon failure to write zeros.
Changes:
========
1. fix write_zeros() per Jim's comments.
2. remove char const *fname from struct extent_scan.
3. change the signature of open_extent_scan() from
"void open_extent_scan(struct extent_scan **scan)" to
"void open_extent_scan(struct extent_scan *scan)" to avoid having
to malloc the extent_scan variable; instead save it on the stack.
4. move close_extent_scan() from a function defined in extent-scan.c
to extent-scan.h as a macro definition, but it does nothing for now,
since initial extent scan defined at stack.
5. add a macro "free_extents_info()" defined at extent-scan.h to
release the memory allocated to extent info which should be called
combine with get_extents_info(), it just one line, so IMHO, define
it as macro should be ok.
* src/extent-scan.c: New file; functions to read "extents".
* src/extent-scan.h: Header file of extent-scan.c.
* src/Makefile.am: Reference it and link it to copy_source.
* src/copy.c: Use the new functions and avoid double-free.
* src/copy.c (fiemap_copy): Ensure that our fiemap buffer
is large enough and well-aligned.
Replace "0LL" with equivalent "0" as 3rd argument to lseek.
* src/copy.c (fiemap_copy): Rename from fiemap_copy_ok.
Add/improve comments.
Remove local, "fail".
(fiemap_copy): Do not require caller to set
"normal_copy_required" before calling fiemap_copy.
Report ioctl failure if it's the 2nd or subsequent call.
* tests/cp/sparse-fiemap: Ensure that a file with many extents
(more than fit in copy.c's internal buffer) is copied properly.
Don't require root access if current partition is btrfs or xfs.
Use init.sh, not test-lib.sh.
* tests/filefrag-extent-compare: New file.
* tests/cp/sparse-fiemap: Add a new test for FIEMAP-copy against a
loopbacked ext4 partition.
* tests/Makefile.am (sparse-fiemap): Reference the new test.
* src/fiemap.h: Add fiemap.h for fiemap ioctl(2) support. Copied
from linux's include/linux/fiemap.h, with minor formatting changes.
* src/copy.c (copy_reg): Now, when `cp' is invoked with --sparse=[WHEN],
we will try to do FIEMAP-copy if the underlying file system
supports it, and fall back to a normal copy if it fails.
* doc/coreutils.texi (cut invocation): Remove the tr -s '[:blank:]'
example, as it doesn't handle leading and trailing blanks. Add `awk`
examples for common field processing operations often asked about.
Also document a `join` hack, to achieve the same thing. Note the
join options are ordered so as to be compatible with other systems.
This allows one to use join as a field extractor like:
join -a1 -o 1.3,1.1 - /dev/null
* src/join.c (join): Don't flag unpairable lines when
one of the files is empty.
* tests/misc/join: Add a new test for empty input, and adjust
a previous test that was only checking against empty input.
* doc/coreutils.texi (join invocation): Document the change.
* NEWS: Likewise.
Lines with a different number of fields than the first line,
will be truncated or padded.
* src/join.c (prfields): A new function refactored from prjoin(),
to output all but the join field.
(prjoin): Don't swap line1 and line2 when line1 is blank
so that the padding is applied to the right place.
(main): Handle the -o 'auto' option.
* tests/misc/join: Add 6 new cases to test the auto format.
* NEWS: Mention the change in behavior.
Suggestion from Assaf Gordon
Those were useful when tests might have been run in the same
directory and in parallel. Now, each test is run in a newly-
created empty directory.
* tests/cp/backup-1: Remove obsolete uses of "$$".
* tests/cp/same-file: Likewise.
* tests/dd/misc: Likewise.
* tests/mv/part-symlink: Likewise.
* tests/mv/to-symlink: Likewise.
* tests/touch/fail-diag: Likewise.
There was a non-negligible delay after running a single test.
Now, you'll know why when you see this test's name.
* tests/check.mk (vc_exe_in_TESTS): Don't @-hide commands.
Use $(AM_V_GEN) instead.
* tests/du/move-dir-while-traversing: Ignoring SIGTSTP is enough;
don't also attempt to ignore SIGSTOP, it cannot be handled or ignored.
Spotted by Andreas Schwab.
* tests/du/move-dir-while-traversing: Create an even larger tree
to avoid a false-positive failure due to du terminating before
the rename is triggered.
* src/tail.c (fremote): Do not print a diagnostic when
fstatfs (pipe_FD, &buf) fails, as it now does on linux-2.3.38.
This avoids the spurious failure of tests/misc/tail's f-pipe-1
test, when running in input-from-pipe mode.
The previous gnulib submodule reference was *still* to a
non-public commit. My submodule had a stray commit, so
the reference was always to a local merge commit.
Reported by Rob Vermaas.
* src/uniq.c (find_field): Remove the byte-skipping loop altogether.
Instead, perform the simple calculation. This results in a 10%
performance improvement for large byte offsets.
* NEWS (Bug fixes): Mention it.
* src/uniq.c (find_field): Stop processing loop when end of line
is reached. Before this fix, 'uniq -f 10000000000 /etc/passwd'
would run for a very long time.
* doc/coreutils.texi (tr's Character sets): Document how a 9-bit
octal value is interpreted. tr does not ignore the ninth bit.
(echo invocation, printf invocation): Document that any ninth
bit in \OOO is ignored. (http://debbugs.gnu.org/7574)
* bootstrap.conf: Add the read-file module
* src/ptx.c: Replace the original code which would
needlessly read SIZE_MAX bytes of files larger than this.
* src/shuf.c: Replace the original code.
* src/system.h: Note where it should be included, and
make ordering check portable to GLIBC > 2
* src/copy.c: Move <sys/ioctl.h> along with other system headers
as is done elsewhere.
* src/install.c: Move <sys/wait.h> along with other system headers
as is done elsewhere.
* src/ptx.c: Include <regex.h> rather than "regex.h" as
is done elsewhere. Note <regex.h> is kept after "system.h"
as per commit dba300a0.
* HACKING: Remove mention of "indent-tabs-mode: nil", since
we've remove all of those directives. No longer needed.
Remove dated (pre-emacs-23) reference regarding WhiteSpace mode.
* NEWS (Bug fixes): Mention it.
* src/du.c (prev_level): Move declaration "up" to file-scope global.
(du_files): Reset prev_level to 0 upon abnormal fts_read termination.
Reported by Johathan Nieder in http://bugs.debian.org/609049
Also, improve a diagnostic.
* tests/du/move-dir-while-traversing: Test for the above.
* tests/Makefile.am (TESTS): Add it.
Before this change, we had a tendency to manually list each
contributor's name in THANKS. Now, each commit "Author" is
included in the generated THANKS file automatically, and most
of the old THANKS file is now a template, THANKS.in.
We'll still have to manually list the names of people who report
problems without a usable patch.
* THANKS.in: New file, derived from THANKS, but removing names of
those who are listed as git log 'Author:'s.
* THANKS: Remove file.
* thanks-gen: New file.
* Makefile.am (THANKS): New rule.
(EXTRA_DIST): Add .mailmap, THANKS.in and thanks-gen.
* .gitignore: Add THANKS and THANKS-to-translators.
* .mailmap: Unify on single address and name-spelling per contributor.
Commit 041c9c47 traded the 'gettext' module for the lighter 'gettext-h'
module, so as to not require the latest gettext release (we only need
the latest release if we ship gettext as a dependent library, but
coreutils has long preferred to use it as an external library).
But that commit overlooked two places necessary to allow the use of
gettext 0.17.
This does not force you to downgrade (using gettext 0.18.1.1 is still
just fine), nor does it affect tarballs (once a tarball is built
with a given gettext version, it can be built on other machines
regardless of what gettext version is present).
* bootstrap.conf (buildreq): Relax prerequisite.
* configure.ac (AM_GNU_GETTEXT_VERSION): Likewise.
* src/split.c (set_suffix_length): Only auto-calculate
the suffix length when the number of files is specified.
* tests/misc/split-a: Add a case to trigger the bug,
and exercise the suffix length auto-calculation.
* NEWS: Mention the fix.
Reported by Dmitry V. Levin and Sergey Vlasov at
https://bugzilla.altlinux.org/show_bug.cgi?id=24841
* src/csplit.c (create_output_file): Detect overflow when the
file counter wraps around, and exit with a diagnostic. Formerly
the code silently wrapped around and wrote to the wrong file,
losing output data.
* src/getlimits.c (decimal_ascii_add): Remove, replacing with ...
(decimal_absval_add_one): New function, with different signature,
which does not assume ASCII. All callers changed.
(print_int): Remove assumptions that integers fit in 206 bits, and
that characters are ASCII. These assumptions are portable in
practice but are easy to remove here.
* tests/mv/i-3: Adjust comment to match just-changed code.
Spotted by Pádraig Brady.
* tests/init.cfg (retry_delay_): Correct spelling of function name
in usage example.
* gnulib-tests/Makefile.am (test_xvasprintf_CFLAGS):
(test_lock_CFLAGS, test_tls_CFLAGS): Avoid a syntax error when
$(WERROR_CFLAGS) expands to more than one token.
* gnulib-tests/Makefile.am (test_xvasprintf_CFLAGS):
(test_lock_CFLAGS, test_tls_CFLAGS): Do not append GCC-specific
flags like -Wno-format-security unless the GCC-specific flag
-Werror is also specified. This avoids a "make check" failure on
Solaris when using Sun C 5.8.
This prevents a compilation failure on Solaris 8, GCC 4.4.2, with
"configure --enable-gcc-warnings".
* src/who.c (MAXHOSTNAMELEN): Remove; no longer needed.
* src/pinky.c: Likewise.
* tests/init.sh (setup_): Initialize fail=0 before invoking mktempd_.
Ensure that IFS is defined initially.
(mktempd_): Remove fail=0 initialization; no longer needed.
* src/cp.c (do_copy): When -T is specified, initialize
the NEW_DST and SB variables, which are checked when
running: cp -T --force --backup file file
* tests/cp/backup-1: Add the -T case
* src/sort.c (main): If --parallel isn't specified,
restrict the number of threads to 8 by default.
If the --parallel option is specified, then
allow any number of threads to be set, independent
of the number of processors on the system.
* doc/coreutils.texi (sort invocation): Document the changes
to determining the number of threads to use.
Mention the memory overhead when using multiple threads.
* tests/misc/sort-spinlock-abuse: Allow single core
systems that support pthreads.
* tests/misc/sort-stale-thread-mem: Likewise.
* tests/misc/sort-unique-segv: Likewise.
* NEWS: Mention the change in behaviour.
Without this change, tests/misc/sort-compress-hang would consume
more than 10,000 process slots on my RHEL 5.5 x86-64 server,
making it likely for other applications to fail due to lack of
process slots. With this change, the same benchmark causes 'sort'
to consume at most 19 process slots. The change also improved
wall-clock time by 2% and user+system time by 14% on that benchmark.
* NEWS: Document this.
* src/sort.c (MAX_PROCS_BEFORE_REAP): Remove.
(reap_exited): Renamed from reap_some; this is a more accurate name,
since "some" incorrectly implies that it reaps at least one process.
All uses changed.
(reap_some): New function: it *does* reap at least one process.
(pipe_fork): Do not allow more than NMERGE + 2 subprocesses.
(mergefps, sort): Omit check for exited processes: no longer needed,
and anyway the code consumed too much CPU per line when 2 < nprocs.
* NEWS: Document this.
* src/sort.c (UNCOMPRESSED, UNREAPED, REAPED): New constants.
(struct tempnode): New member 'state', to hold these constants.
The pid member is now undefined if state == UNCOMPRESSED.
(struct sortfile): Replace member 'pid' with member 'temp'.
(uintptr): Remove.
(proctab_hasher, proctab_comparator, register_proc, delete_proc):
Proctab entries are now struct tempnode *, not pid_t, to handle
the case where multiple tempnode objects correspond to the same
pid. This avoids a race condition that can cause a hang.
(register_proc): Arg is now struct tempnode *, not pid_t. All
callers changed.
(delete_proc): Set tempnode state to REAPED.
(create_temp_file): No need to set pid member here; it's now
done when the pid is known.
(maybe_create_temp, create_temp): Remove PPID arg. Return struct
tempnode *, not char *. All callers changed.
(maybe_create_temp): Set node state to UNCOMPRESSED or UNREAPED.
No need to set node->pid to 0.
(open_temp): Replace NAME and PID args with a single TEMP arg.
All callers changed. Wait only for unreaped children.
(zaptemp): Wait for decompressor to finish before removing its
temporary-file input. This avoids .nfsXXXX hassles with NFS
and fixes a race (leading to a hang) regardless of NFS.
(open_input_files): Adjust to new way of dealing with temp files
and their subprocesses.
* tests/Makefile.am (TESTS): Add misc/sort-compress-hang.
* tests/misc/sort-compress-hang: New file.
* NEWS: Document this.
* src/sort.c (avoid_trashing_input): The previous fix to this
function didn't fix all the problems with this code. Replace it
with something simpler: just copy the input file. This doesn't
change the number of files, so return void instead of the updated
file count. Caller changed.
* tests/misc/sort-merge-fdlimit: Test for the bug.
Those options are useful only on systems that lack inotify support
and in the unusual event that a system with inotify support must
resort to polling.
* src/tail.c (usage): Note that the --max-unchanged-stats=N and
--sleep-interval=N options are rarely useful on systems with
inotify support.
* doc/coreutils.texi (tail invocation): Likewise.
* src/sort.c (avoid_trashing_input): Fix a typo that could cause a
buffer overrun in theory. In practice this is extremely unlikely,
as it requires running out of file descriptors in a small merge,
presumably because some other process is hogging all the OS's file
descriptors.
* tests/check.mk (TESTS_ENVIRONMENT): Default TMPDIR to /tmp,
rather than to the working directory; this is more common in
practice, which makes the tests more real-worldish; and it is
often faster. Also, it avoids some problems with NFS cleanups.
* tests/misc/sort-compress: Remove unnecessary code setting TMPDIR.
* tests/misc/sort-compress-proc: Likewise. Do the final sleep
only if TMPDIR is relative, which should be rarely given the
change to TESTS_ENVIRONMENT.
* src/sort.c (uintptr): New type.
(enum procstate, struct procnode, update_proc): Remove.
(proctab_hasher, proctab_comparator, register_proc, wait_proc):
(reap_some): The proctab is now simply a hash of process-IDs
rather than of pointers to objects with reference counts and
states; this is smaller and faster and easier to understand.
(nprocs): Now pid_t, not size_t, since one cannot have more than
PID_MAX children.
(reap): If the argument is -1, wait; if 0 (a new value), do not.
Delete pid from proctab as needed. Ignore children that are not
in proctab, as they are from the program that exec'ed us and are
irrelevant to our success or failure.
(delete_proc, reap_all): New functions.
(open_temp): Register the child.
(sort): Clean up all children afterwards; without this patch,
'sort' sometimes missed failures in children due to race conditions.
* tests/Makefile.am (TESTS): Add misc/sort-compress-proc.
* tests/misc/sort-compress-proc: New file, to test for the
bugs fixed above.
* tests/misc/sort-stale-thread-mem: Don't initialize fail=0 here;
that is done in init.sh. This avoids a syntax-check failure.
Invoke "Exit $fail" at end, too.
Mark as a very expensive test.
This change does not fix the actual bug. That was done by commit
c9db0ac6, "sort: preallocate merge tree nodes to heap". The fix
was to store each "node" structure on the heap, not on the stack.
Otherwise, a node from one thread's stack could be used in another
thread after the first thread had expired (via pthread_join).
This bug was very hard to trigger when using spinlocks, but
easier once we began using mutexes.
* NEWS (Bug fixes): Mention it.
For details, see http://debbugs.gnu.org/7597.
* src/sort.c (specify_nthreads, merge_tree_init, init_node):
(queue_init, sortlines, struct thread_args, sort, main):
Use size_t, not unsigned long int, for thread counts, since thread
counts are now used to compute sizes.
(specify_nthreads): Check for size_t overflow.
(merge_tree_init, sort): Shorten name of local variable, for
readability.
(merge_tree_init): Move constants next to each other in product,
so that the constant folding is easier to see.
(init_node): Now static. Add 'restrict' only where it might
be helpful for compiler optimization.
(queue_init): 2nd arg is now nthreads, not "reserve", which is
a bit harder to follow. All uses changed.
(struct thread_args): Rename lo_child to is_lo_child, so that
it's obvious to the reader when we're talking about this boolean
as opposed to the new lo_child member of the other structure.
All uses changed.
(sort): Remove unused local variable end_node.
(main): Don't allow large thread counts to cause undefined behavior
later, due to integer overflow.
* src/sort.c: (merge_tree_init) New function. Allocates memory for
merge tree nodes.
(merge_tree_destory) New function.
(init_node) New function.
(sortlines) Refactor node creation code to init_node. Remove now
superfluous arguments. All callers changed.
(sort) Initialize/destory merge tree. Refactor root node creation
to merge_tree_init.
Running a command like this on a multi-core system
sort < big-file | less
would peg all processors at near 100% utilization.
* src/sort.c: (struct merge_node) Change member lock to mutex.
All uses changed.
* tests/Makefile.am (XFAIL_TESTS): Remove definition, now that
this test passes once again. I.e., the sort-spinlock-abuse test
no longer fails.
* NEWS (Bug reports): Mention this.
Reported by DJ Lucas in http://debbugs.gnu.org/7489.
When -n l/N is used and long lines are present that both
span partitions and multiple buffers, one would get
inconsistent chunk sizes.
* src/split.c (main): Add a new undocumented ---io-blksize option
to support full testing with varied buffer sizes.
(cwrite): Refactor most handling of --elide-empty to here.
(bytes_split): Remove handling of --elide-empty.
(lines_chunk_split): Likewise. The specific issue here
was the first handling of elide_empty_files interfered
with the replenishing of the input buffer.
* test/misc/split-lchunk: Add -e and the new ---io-blksize
combinations to the test.
* tests/misc/sort-spinlock-abuse: On a busy system, with only 12
pauses of length 0.1 seconds, the buggy (busy-spinlock blocked)
sort would fail to accumulate 1 second of CPU time, and hence
would mistakenly pass. Increase from 12 to 50.
* src/sort.c (struct thread_args, sortlines_thread, sortlines, sort):
Rename "merge_queue" to "queue", for consistency with other functions
that just use the name "queue" for these things.
* src/sort.c (mergelines_node): Return void, not size_t. All
callers changed. Change *node->dest here, not in caller.
Do not change node->dest: it's not needed and could cause problems
on (mostly theoretical) hosts that do not allow adding integers to
null pointers.
(queue_check_insert_parent): Omit MERGED parameter; no longer needed.
All callers changed.
* src/sort.c (queue_check_insert, queue_check_insert_parent): Make
the queue arg first, for consistency with other functions such as
queue_insert that put the queue arg first. Rename from
check_insert and update_parent, respectively. All callers
changed.
* src/sort.c (struct merge_node): 'lock' is now the actual lock,
not a pointer to the lock; there's no need for indirection here.
Make 'level' unsigned int instead of size_t, since it is a
bit-shift count; also, move it next to a bool so that it's more
likely to take less space. All uses changed.
(sortlines, sort): Spell out initialization instead of using an
initializer. This makes the initializer a bit easier to understand,
and avoids unnecessary stores into the spin lock.
This problem was observed on RHEL 5.5 x86-64 when running as a
client of a NetApp FAS2050.
* tests/cp/cp-mv-backup: Don't leave a file descriptor open to
a file in a directory that will be cleaned up with "rm -rf".
Under NFS, when the rm unlinks that file, it is instead renamed
to .nfsXXXX and then rm cannot remove the parent directory,
and the test fails.
* tests/cp/same-file: Likewise.
* src/sort.c (MAX_MERGE): Avoid integer overflow when on a machine
with (say) 32-bit int and 64-bit size_t and when level == 15.
Without this fix, on such a machine with 32768 or more processors,
the level computation could overflow on large input, and this
would result in division by zero.
* src/sort.c (write_unique): Save the entire "struct line", not
just a pointer to one. Otherwise, with a multi-thread run,
sometimes, with some inputs, fillbuf would would win a race
and clobber a "saved->text" pointer in one thread just before
it was dereferenced in a comparison in another thread.
* NEWS (Bug fixes): Mention it.
* README-valgrind: Include the "noinst" programs in
those wrapped by valgrind. Update $PATH in check.mk
rather than Makefile.am. Make wrapper scripts
work when suppressions not setup. Keep lines < 80 chars.
This patch was written by Jim Meyering and myself.
* src/copy.c (copy_reg): Turn EISDIR to ENOTDIR to improve the
quality of diagnostics for commands like "cp a nosuch/". Reported
by Марк Коренберг and Alan Curry in the thread starting at:
http://lists.gnu.org/archive/html/bug-coreutils/2010-11/msg00178.html
* THANKS: Update.
* tests/mv/trailing-slash: Add a test.
* src/split.c (usage, long_options, main): New options --number,
--unbuffered, --elide-empty-files.
(set_suffix_length): New function to auto increase suffix length
to handle a specified number of files.
(create): New function. Refactored from cwrite() and ofile_open().
(bytes_split): Add max_files argument to support byte chunking.
(lines_chunk_split): New function. Split file into chunks of lines.
(bytes_chunk_extract): New function. Extract a chunk of file.
(of_info): New struct. Used by functions lines_rr and ofile_open
to keep track of file descriptors associated with output files.
(ofile_open): New function. Shuffle file descriptors when there
are more output files than available file descriptors.
(lines_rr): New function to distribute lines round-robin to files.
(chunk_parse): New function. Parses K/N syntax.
* tests/misc/split-bchunk: New test for byte chunking.
* tests/misc/split-lchunk: New test for line delimited chunking.
* tests/misc/split-rchunk: New test for round-robin chunking.
* tests/Makefile.am: Reference new tests.
* tests/misc/split-fail: Add failure scenarios for new options.
* tests/misc/split-l: Fix a typo. s/ln/split/.
* doc/coreutils.texi (split invocation): Document --number.
* NEWS: Mention the new feature.
* .mailmap: Map new email address for shortlog.
Signed-off-by: Pádraig Brady <P@draigBrady.com>
* NEWS: Describe patch.
* bootstrap.conf (gnulib_modules): Add ftoastr.
* src/od.c: Include ftoastr.h, not float.h.
(FLT_DIG, DBL_DIG): Remove. No need to verify LDBL_DIG.
(FMT_BYTES_ALLOCATED): No need to worry about floating point now,
since this format is no longer used for floating point.
(PRINT_FIELDS): New macro, with most of the guts of the old PRINT_TYPE.
(PRINT_TYPE): Rewrite to use PRINT_FIELDS.
(PRINT_FLOATTYPE): New macro. This uses the new functions from
ftoastr.
(print_float, print_double, print_long_double): Reimplement
using PRINT_FLOATTYPE.
(decode_one_format): Calculate field widths based on ftoastr-supplied
macros.
* tests/Makefile.am (TESTS): Add misc/od-float.
* tests/misc/od-float: New file.
Initially, I did this,
git grep -l srcdir/test-lib.sh|xargs perl -p0i -e '~180-byte script'
but that line would have been much longer than the maximum permitted
by coreutils' commit hook, and wasn't readable besides, so here's a
more readable version:
lhs=$(printf '%s\\n' \
'if test "$VERBOSE" = yes; then' \
' set -x' \
' touch --version' \
'fi' \
'' \
'. $srcdir/test-lib.sh' \
| sed 's/\$/\\\$/g;s/touch/(\\w+)/')
rhs=$(printf '%s\\n' \
'. "${srcdir=.}/init.sh"; path_prepend_ ../src' \
'test "$VERBOSE" = yes && FIXME --version' \
| sed 's/\$/\\\$/g;s/FIXME/\$1/')
git grep -l srcdir/test-lib.sh|xargs perl -p0i -e "s,$lhs,$rhs,"
src/truncate.c (main): Use a bool to store if an error occurred,
rather than an int, to protect against overflow.
(do_ftruncate): Likewise. Also change 0/false to mean failure
rather than success.
... when configured with the --enable-gcc-warnings option.
This follows on from commit 34ef0a01, 2010-10-14,
"sort: fix unportable cast of unsigned char * -> char *"
* configure.ac: -Wall implicitly enables this warning
so remove the explicit disabling.
* gl/modules/fadvise-tests: Add the module previously missed
in commit 63b5e816, 2010-07-14, "fadvise: new module ...".
* gl/tests/test-fadvise.c: Add a comment as to why we don't
check return values.
* doc/coreutils.texi (stat invocation): Add sentence-ending period.
* NEWS: Correct stat change description: s/floating point //.
* cfg.mk (old_NEWS_hash): Update, to match this NEWS change.
Also, do the following to avoid "make syntax-check" failure
induced by new rules.
* .x-sc_bindtextdomain: Exempt files with an #ifdef'd "main".
* Makefile.am: Add this file.
The implementation of variable-precision time stamps relied
on heuristics that made the output subtly nondeterministic,
or at least hard to reproduce:
http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/21531/focus=21538
So, for now at least, we're removing that feature.
* bootstrap.conf (gnulib_modules): Remove fstimeprec.
* gl/lib/fstimeprec.c, gl/lib/fstimeprec.h: Remove files.
* gl/modules/fstimeprec, gl/modules/fstimeprec-tests: Likewise.
* gl/tests/test-fstimeprec.c: Remove file.
* doc/coreutils.texi (csplit invocation): Say that %d and %i are
aliases for %u.
* src/csplit.c (FLAG_THOUSANDS, FLAG_ALTERNATIVE): New constants.
(get_format_flags): Now take char const * and int * and return
size_t. It now stores info about the flags instead of merely
scanning them. Also, it handles '0' correctly. Drop support for
the undocumented '+' and ' ' flags since the value is unsigned.
Add support for the (undocumented) "'" flag. All uses changed.
(get_format_width, get_format_prec): Remove.
(check_format_conv_type): Renamed from get_format_conv_type, with
a different signature. It now converts the format to one that is
compatible with unsigned int, and checks flags. All uses changed.
(max_out): Have snprintf compute the number of bytes needed rather
than attempting to do it ourselves (which doesn't work portably
with outlandish formats such as %4294967296d).
(check_format_conv_type, main): Check for overflow in size
calculations. Don't assume size_t fits in unsigned int.
* tests/misc/csplit: Check for proper handling of flags, with
%0#6.3x. Coreutils 8.6 mishandles this somewhat-weird example.
* src/csplit.c (free_buffer): Also free the line offsets buffers
(remove_line): Also free the containing structure
* tests/misc/csplit-heap: A new test to trigger with leaks of
this magnitude.
* tests/Makefile.am: Reference the new test
* NEWS: Mention the fix
Reported by David Hofstee
Without this fix, seq 1000 | csplit - /./ '{*}' would write
the NUL-terminated file name, xx1000, into a buffer of size 6.
* src/csplit.c (main): Use properly sized file name buffer.
* NEWS (Bug fixes): Mention it.
* tests/misc/csplit-1000: New test to trigger the bug.
* tests/Makefile.am (TESTS): Add misc/csplit-1000.
* tests/mv/i-3: Adjust so that the symlink is resolved
before redirecting to the background command, as otherwise
the stdin descriptor passed to the command will fail the
isatty() or ttyname() test.
because dash-0.5.6-2.fc14.x86_64 would also be disqualified.
This reverts part of yesterday's commit 6c058b2d, "tests: avoid
failure due to bug in FreeBSD 8.1's /bin/sh". Note that the offending
aspect of those shells is not officially a bug, since "local" is not
specified by POSIX. However, it is very unintuitive that prepending
"local" to an existing, standards-compliant assignment would evoke
such a fundamental change in semantics.
* tests/init.sh: Remove snippet requiring sane "local" support.
Upstream dash bug report:
http://thread.gmane.org/gmane.comp.shells.dash/419
* src/stat.c (digits, printf_flags): New static vars.
(make_format): New function.
(out_string, out_int, out_uint, out_uint_o, out_uint_x):
(out_minus_zero): Use it to avoid undefined behavior when invoking
printf.
(print_it): Check for invalid conversion specifications such as
%..X and %1-X, which would otherwise rely on undefined behavior
when invoking printf.
* tests/misc/stat-nanoseconds: Check that the "I" printf flag
doesn't mess up in the C locale, as it formerly did on non-GNU
hosts.
* tests/init.sh: Arrange not to accept FreeBSD 8.1's /bin/sh, since
it fails this test: /bin/sh -c 'f(){ local s=$IFS; test -n "$s"; }; f'
* tests/init.cfg (sanitize_path_): Stop-gap measure to work around
a bug in FreeBSD 8.1's /bin/sh. We'll un-do this change once all
300+ tests use init.sh.
* NEWS: Document this.
* doc/coreutils.texi (stat invocation): Likewise.
* gl/lib/fstimeprec.c, gl/lib/fstimeprec.h, gl/modules/fstimeprec:
* gl/modules/fstimeprec-tests, gl/tests/test-fstimeprec.c:
New files.
* bootstrap.conf (gnulib_modules): Add fstimeprec.
* src/stat.c: Include fstimeprec.h. Don't include xstrtol.h.
(decimal_point, decimal_point_len): New static vars.
(main): Initialize them.
(epoch_sec, out_ns): Remove.
(out_int, out_uint): Now returns whatever printf returned.
(out_minus_zero, out_epoch_secs): New functions.
(print_stat): Use out_epoch_sec instead of out_ns and epoch_sec.
(print_stat, print_it, usage): Remove the %:X-style formats.
* tests/misc/stat-nanoseconds: Set TZ=UTC0 to avoid problems
with weird time zones. Use a time stamp near the Epoch so that we
don't have to worry about leap seconds. Redo test cases to match
new behavior.
* tests/touch/60-seconds: Change %Y.%:Y to %.9Y, to adjust to
new behavior.
This reverts part of the recent commit 9069af45,
"stat: print timestamps to full resolution", which made %X, %Y, %Z
print floating point numbers. We prefer to retain portability of
%X, %Y and %Z uses, while still providing access to full-resolution
time stamps via modified format strings. Also make the new
%W consistent.
* src/stat.c: Include "xstrtol.h".
(print_it): Accept a new %...:[XYZ] format directive,
e.g., %:X, to print the nanoseconds portion of the corresponding time.
For example, %3.3:Y prints the zero-padded, truncated, milliseconds
part of the time of last modification.
(print_it): Update print_func signature to match.
(neg_to_zero): New helper function.
(epoch_time): Remove function; replace with...
(epoch_sec): New function; use timetostr.
(out_ns): New function. Use "09" only when no other modifier
is specified.
(print_statfs): Change type of "m" to unsigned int,
now that it must accommodate values larger than 255.
(print_stat): Likewise.
Map :X to a code of 'X' + 256. Likewise for Y, Z and W.
(usage): Update.
* tests/touch/60-seconds: Use %Y.%:Y in place of %Y.
* tests/misc/stat-nanoseconds: New file.
* tests/Makefile.am (TESTS): Add it.
* NEWS (Changes in behavior): Mention this.
With improvements by Pádraig Brady.
Thanks to Andreas Schwab for raising the issue.
* tests/ls/stat-free-symlinks: strace currently outputs an
informational message about arch difference to stdout.
Therefore we need to strip that before comparison.
* doc/coreutils.texi (cp invocation): Change the description slightly
so as users might not immediately discount using this option.
Mention that --reflink is overridden by the other linking options and
--attributes-only, and give an example where this might be useful.
* src/copy.c (copy_internal): Bypass the reflink if
--attributes-only is specifed.
* tests/cp/reflink-perm: Ensure both --reflink modes are
overridden by --attributes-only.
* NEWS: Mention the change in behavior.
Reported by Jim Meyering.
src/tail.c (main): As an optimization, don't bother checking
for stdin or remote files, when ---disable-inotify is specified.
To improve the fix in commit 61b77891, set the disable_inotify
flag when we fall back to polling, so that we recheck remote files.
NEWS: Mention the fix
* src/du.c (show_date): Fix call to fputs with a buffer that
contains some uninitialized data.
* tests/Makefile.am (TESTS): Add du/big-timestamp.
* tests/du/bigtime: New file, which checks for the bug.
And remove the now-superfluous totals from the other two warnings,
so the plurals will also work in other languages than English.
* src/md5sum.c (digest_check): Change as above.
* tests/misc/md5sum (check-quiet2): Adjust accordingly.
* gnulib: Update to latest for warning-free tests.
* gnulib-tests/Makefile.am (AM_CFLAGS): Uncomment $(WERROR_CFLAGS),
now that these tests are warning-free.
* tests/misc/sort-float: On systems with DBL_MIN < LDBL_MIN,
this test would fail because the expected output was not sorted.
Detect that case, and if needed, reverse those two values.
* src/fold.c (fold_file): Apply fadvise to istream, not stdin.
This bug would have inhibited the fadvise optimization when not
reading from standard input.
* src/sort.c (key_warnings): Local buffer should be of size
INT_BUFSIZE_BOUND (uintmax_t), not INT_BUFSIZE_BOUND (sword).
This bug was discovered by running 'make check' on a 32-bit
Solaris 8 sparc host, using Sun cc. I saw several other instances
of invoking umaxtostr on a buffer declared to be of size
INT_BUFSIZE_BOUND (VAR), and these instances should at some point
be replaced by INT_BUFSIZE_BOUND (uintmax_t) too, as that's a
less error-prone style.
* tests/misc/ls-misc (push_ls_colors): Don't assume LS_COLORS
is set. This part of the fix is by Jim Meyering.
(sl-dangle2, sl-dangle3, sl-dangle4, sl-dangle5): Don't assume
that newly-created files will have time stamps in the past. They
might not, due to clock skew, if the file systems are remote.
http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/21322/focus=21346
* src/install.c (extra_mode): Don't assign ~S_IRWXUGO & ~S_IFMT
to a mode_t variable, as the number might be too big to fit.
Solaris 10 cc warns about this, and the C standard says it
has undefined behavior.
* gl/lib/mbsalign.c: Include <stdint.h> for SIZE_MAX.
Use SIZE_MAX rather than (size_t) -1, to avoid warning for
"size_t x = -1" on Solaris 10 cc.
* gl/modules/mbsalign (Depends-on): Add stdint.
* src/sort.c (fold_toupper): Change this back from char to
unsigned char, fixing a portability issue introduced in commit
59e2e55d0f dated February 26, as the
C Standard doesn't let you convert from unsigned char * to char *
without a cast, and the (in theory more portable) style here is to
convert char values, not pointer values.
(getmonth): Convert char to unsigned char when needed for
comparison.
* configure.ac (GNULIB_WARN_CFLAGS): Define new variable, for use
in gnulib-tests.
* gnulib-tests/Makefile.am: Disable specific -W___ options,
but only for a few specific offending tests:
(test_xvasprintf_CFLAGS): Define.
(test_lock_CFLAGS, test_tls_CFLAGS): Define.
* src/tail.c (tail_forever_inotify): Handle the case where
tail --follow=name with inotify, is not able to add a watch on
a specified directory. This may happen due to inotify resource
limits or if the directory is currently missing or inaccessible.
In all these cases, revert to polling which will try to reopen
the file later. Note inotify returns ENOSPC when it runs out
of resources, and instead we report a particular error message,
lest users think one of their file systems is full.
(main): Document another caveat with using inotify, where we
currently don't recheck directories recreated after the
initial watch is setup.
* tests/tail-2/F-vs-rename: Fix the endless loop triggered by
the above issue.
* tests/tail-2/inotify-hash-abuse: Likewise.
* tests/tail-2/wait: Don't fail in the resource exhaustion case.
* tests/tail-2/F-vs-missing: A new test for this failure mode
which was until now just triggered on older buggy linux kernels
which returned ENOSPC constantly from inotify_add_watch().
* NEWS: Mention the fix.
The bug was introduced with commit 23f6d41f, 19-02-2003.
* src/split.c (bytes_split, lines_split, line_bytes_split):
Correctly check the return from full_read().
* tests/misc/split-fail: Ensure split fails when
it can't read its input.
* NEWS: Mention the fix.
* src/stat.c (print_statfs, usage): Drop %C, since it applies to
files, not file systems.
(out_file_context): Match style of other out_* functions.
(print_stat): Update caller.
* doc/coreutils.texi (stat invocation): Document %C.
* NEWS: Document the change.
* src/stat.c (default_format): Don't print SELinux context
when in file system (-f) mode, as the context is associated
with the file, not the file system.
Fix logic inversion, so that in terse mode, %C is included
only when is_selinux_enabled and not vice versa.
This touches all new tests added since commit b062bbd9, when
init.cfg was added. Older tests still exist that could use
conversion, but we can address those later.
* tests/sample-test: Restore hint for --version output.
* tests/misc/stat-birthtime: Include --version if verbose.
* tests/misc/tr-case-class: Likewise.
* tests/misc/stat-mount: Likewise.
* tests/misc/sort-unique: Likewise.
* tests/misc/sort-benchmark-random: Likewise.
Suggested by Jim Meyering.
Yes, this patch intentionally leaks the results of default_format(),
since it is called only twice, and since the results are in scope
until main() exits. Not worth the extra code to pacify valgrind.
* src/stat.c (main): Hoist default format computation out of loop.
(do_statfs, do_stat): Move default format generation...
(default_format): ...into new function. Allocate the result in
pieces, rather than repeating mostly-similar chunks. Allow
translation of verbose format. Pass a second format to do_stat,
for the one aspect of the default format that is conditional on
file type.
* NEWS: Document the translation aspect.
* src/stat.c (epoch_time): New function.
(print_stat): Use it for %[WXYZ].
* NEWS: Document this.
* tests/touch/60-seconds: Adjust test to match.
* tests/misc/stat-birthtime: Likewise.
This module may be moved to gnulib at some stage,
so keep it C89 compatible.
* gl/lib/mbsalign.c (mbsalign): Declare variables at start of scope
(ambsalign): Likewise
* gl/lib/mbsalign.h: Remove trailing comma from enum
Reported by Andrei Suhan
* NEWS (Bug fixes): Mention the du-exclude--vs--cycle-dir fix.
Reported by Graham Cobb in http://bugs.debian.org/598438,
that bug was fixed by the 2010-07-24 commit, 77428214f,
"du: tune, and fix some -L bugs with dangling or cyclic symlinks"
This valid translation spec aborted:
LC_ALL=en_US.iso-8859-1 tr '[:upper:]- ' '[:lower:]_'
This invalid translation spec aborted:
LC_ALL=en_US.iso-8859-1 tr '[:upper:] ' '[:lower:]'
This was caused by commit 6efd1046, 05-01-2008,
"Avoid tr case-conversion failure in some locales"
This misaligned conversion spec was allowed:
LC_ALL=C tr 'A-Y[:lower:]' 'a-z[:upper:]'
This was caused by commit af5d0c36, 21-10-2007,
"tr: do not reject an unmatched [:lower:] or [:upper:] in SET1"
This misaligned spec was allowed by extending the class:
LC_ALL=C tr '[:upper:] ' '[:lower:]'
* src/tr.c (validate_case_classes): A new function to check
alignment of case conversion classes. Also it adjusts the
length of the sets so that locales with different numbers of
upper and lower case characters, don't cause issues.
(string2_extend): Disallow extending the case conversion
class as in the above example. That is locale dependent
and most likely not what the user wants.
(validate): Do the simple test for "restricted" char classes
earlier, so we don't redundantly do more expensive validation.
(main): Remove the case class validation, and simplify.
* tests/misc/tr-case-class: A new test to test the various
alignment and locale issues, associated with case conversion.
* tests/misc/tr: Move case conversion tests to new tr-case-class.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the fixes.
* src/sort.c (sortlines, sort): Use pthread_spin_destroy when a
spin lock is no longer used. This isn't needed on GNU/Linux or
Solaris, but POSIX says it may free up resources on some platforms.
* tests/misc/sort-debug-keys: Don't depend on
printf supporting \xhh format, which isn't supported
by dash for example. Also change from double quoted
strings to single quoted, when we don't need any
variable interpolation.
* tests/misc/stat-mount: Don't try to correlate the
mount points output by df and stat, as they're similar,
but sometimes different in the presence of bind mounts.
* doc/coretuils.texi (stat invocation): Clarify the
bind mount difference between stat and df.
With it, we can remove the two sole tests of HAVE_TERMIOS_H.
* bootstrap.conf (gnulib_modules): Add termios.
* src/ls.c: Don't test HAVE_TERMIOS_H.
* src/stty.c: Likewise.
* m4/jm-macros.m4 (gl_CHECK_ALL_TYPES): Remove configure-time
test for termios.h.
* bootstrap.conf (gnulib_modules): Use calloc-gnu, malloc-gnu
and realloc-gnu modules, rather than calloc, malloc and realloc.
The shorter-named modules are now deprecated.
(obsolete_gnulib_modules): Remove.
(gnulib_modules): Remove raise, strbprk; they're obsolete, too.
Move strtod, strtol here, from obsolete_gnulib_modules.
* tests/init.cfg (retry_delay_): Describe
the backoff method used.
* tests/ls/readdir-mountpoint-inode: Add a timeout
to the stat call to eliminate the chance of hangups.
* tests/mv/i-3: Change the timeout required to pass
from 1 second to a range of .1s - 3.1s.
* tests/rm/dangling-symlink: Likewise.
All tests currently pass on NFS on Linux kernel 2.6.22 at least,
but some fail on 2.6.9, so we exclude those here.
* tests/init.cfg (is_local_dir_): A new function
returning if the specified directory is on a local file system.
(require_local_dir_): A new function to skip tests
if the current directory is not on a local file system.
* tests/cp/existing-perm-race: Skip if non local.
* tests/cp/file-perm-race: Likewise.
* tests/cp/parent-perm: Likewise.
* tests/cp/parent-perm-race: Likewise.
* tests/cp/preserve-2: Likewise.
* tests/mv/part-symlink: Likewise.
* tests/du/basic: Use refactored function.
* tests/install/basic-1: Likewise.
* tests/mkdir/p-3: Likewise.
* tests/dd/skip-seek-past-dev: Likewise.
* tests/du/slink: Likewise. Remove redundant test
for NFS file system.
* tests/misc/join: s/local/locale/.
Following on from commit f86bb696, 01-02-2010,
"join: make -t '' operate on the whole line".
Bypassing the delimiter search in this case,
gives about an 8% performance boost.
* src/join (xfields): Don't bother looking for '\n'
in the data, which we know won't be present.
* src/tac.c (main): Reading a line longer than 16KiB would cause
tac to realloc its primary buffer. Then, just before exit, tac
would mistakenly free the original (now free'd) buffer.
This bug was introduced by commit be6c13e7, "maint: always free a
buffer, to avoid even semblance of a leak".
* NEWS (Bug fixes): Mention it.
* tests/misc/tac (double-free): New test, to exercise this.
Reported by Salvo Tomaselli in <http://bugs.debian.org/594666>.
* src/find-mount-point.c: A new file refactoring
find_mount_point() out from df.c
* src/find-mount-point.h: Likewise.
* src/df.c: Use the new find-mount-point module.
* src/stat.c (print_stat): Handle the new %m format.
(find_bind_mount): A new function to
return the bind mount for a file if any.
(out_mount_mount): Print the bind mount for a file, or else
the standard mount point given by the find-mount-point module.
(usage): Document the %m format directive.
* src/Makefile.am: Reference the refactored find-mount-point.c
* po/POTFILES.in: Add find_mount_point.c to the translation list
* doc/coreutils.texi (stat invocation): Document %m,
and how it may differ from the mount point that df outputs.
* test/misc/stat-mount: A new test to correlate mount points
output from df and stat.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the new feature
* THANKS: Add the author
Signed-off-by: Pádraig Brady <P@draigBrady.com>
* src/df (show_point): Remove the optimization for comparing
the specified path with the device name, as this produces
inconsistent results in the presence of bind mounts. For bind
mounts, the device name is populated with the bind mount target.
* NEWS: Mention the change in behavior.
* .x-sc_program_name: Exclude all current and future
c files in gl/tests from this check
* gl/tests/test-di-set.c: Remove the hack to work around
the set_program_name syntax-check
* gl/tests/test-ino-map.c: Likewise
* gl/tests/test-rand-isaac.c: Likewise
* doc/coreutils.texi (md5sum invocation): Mention currently known
security problems. Don't recommend SHA-1 as alternative.
* man/md5sum.x (BUGS): Warn about the vulnerabilities and
reference the SHA-2 based alternatives.
Reported by Simon Josefsson
This change was prompted by the previous one: I audited the code
looking for similar examples. Too bad valgrind doesn't catch this.
* src/sort.c (check, mergefps): xrealloc -> free + xmalloc
* src/who.c (print_user): Likewise.
* src/sort.c (compare_random): Use free/xmalloc rather than
xrealloc, since the old buffer contents need not be preserved.
Also, don't fail if the guessed-sized malloc fails. Suggested by
Bruno Haible.
* tests/ls/readdir-mountpoint-inode: Check to see if skip_test_ is
called in a helper function via $() instead of mistakenly failing.
* THANKS: Update.
* src/sort.c (compare_random): Guess that the output will be
3X the input. This avoids the overhead of calling strxfrm
twice on typical implementations. Suggested by Bruno Haible.
* NEWS: Document this.
* src/sort.c (getmonth): Omit LEN arg, as MONTH is now null-terminated.
(compare_random): Don't null-terminate keys, as caller now does that.
(compare_version): Remove.
(debug_key): Null-terminate string for getmonth.
(keycompare): Support combining -R with any of -d, -f, -i, -V.
Also, support combining -V with any of -d, -i.
(check_ordering_compatibility): Allow newly-supported combinations.
* tests/misc/sort (02q, 02r, 02s): New tests, for new combinations.
(incompat2): Now test -nR, since -fR are now compatible.
Formerly, the 'compare' function and some of its subroutines had a
debugging flag, which caused them to output underlines. This
change refactors the code so that debugging output is
more-separated from the actual sorting. In the process, the
change fixes a minor error in the debugging output. The change
shortens the source code and executable size a tad, and improves
CPU performance by 2.4% on my platform with a simple benchmark (C
locale, line sorting, no debug).
* src/sort.c (long_double, strtold): Move back to prelude, since
they're now used by multiple functions again.
(unit_order): Move to file scope, since it's now used by two functions.
(find_unit_order, human_numcompare, numcompare, general_numcompare):
Remove endptr parameter. All callers changed.
(human_numcompare): Args are now const pointers.
(getmonth): Endptr is now non-const.
(key_numeric): Move up, since it's needed earlier.
(debug_key): Take a line and a key as argument, instead of having
the caller figure out where the field is.
(debug_line): New function.
(keycompare, compare): Omit debug parameter; debug output now done
elsewhere. All callers changed.
(write_line): Renamed from write_bytes; all callers changed.
Use debug_line (not 'compare') to output debug info.
Use a slightly faster check for whether output file is stdout.
(check): Don't do debugging output; it's not that useful here,
and it confuses the code.
(main): Check for incompatibility between -c and --debug.
Use standard diagnostic for incompatible options.
* tests/misc/sort-debug-keys: Fix test case: "--Mi-1" is not
a number, so its first character should not be underlined when
debugging a numeric sort.
* lib/Makefile.am (libcoreutils_a_SOURCES): Remove xmemxfrm.c,
xmemxfrm.h.
* lib/memxfrm.c, lib/memxfrm.h, lib/xmemxfrm.c, lib/xmemxfrm.h: Remove.
* m4/memxfrm.m4: Likewise.
* m4/prereq.m4 (gl_PREREQ): Remove gl_MEMXFRM.
* po/POTFILES.in: Remove lib/xmemxfrm.c.
* src/sort.c: Don't include xmemxfrm.h.
(cmp_hashes): Remove.
(xstrxfrm): New function.
(compare_random): If a line contains NULs, don't create a big
buffer that contains the strxfrm output of each string in the line.
Instead, accumulate checksums and differences as we go, so that
at any one time we have to store at most the output of a single
strxfrm call when processing the line. This removes the need for
an memxfrm function.
* src/sort.c (debug_width): New function, which does not stop
counting tabs at \0, and also invokes mbsnwidth. Stamp out strnlen!
(count_tabs): Remove.
(debug_key): Use debug_width instead of mbsnwidth and count_tabs.
* tests/misc/sort-debug-keys: Check that \0 and \t intermix.
* NEWS: Document changes to sort -h, which are now minor with
respect to the pre-July-30th version.
* doc/coreutils.texi (sort invocation): Likewise. The
documentation now describes how -h comparison is done rather than
being vague with border cases.
* src/sort.c (long_double, strtold): Move back to general_numcompare.
(LD, compute_human): Remove.
(find_unit_order): Remove THOU_SEP parameter, since thousands
separators are now allowed by all callers. Revert to previous
behavior of sorting by suffix, and returning the order rather than
2 * order + binary, since we no longer care whether binary powers
are being used. However, treat all zeros the same, instead of
sorting 0M before 0G; this is more consistent with the desired
behavior of sorting -1G before -1M.
* tests/misc/sort (h1, h3, h6): Adjust to match mostly-reverted
behavior. However, check that all zeros sort together.
* tests/misc/sort-debug-keys: Omit a "_", since the trailing "i"
in "1234Gi" is no longer part of the key.
* NEWS: Document changes to sort -h.
* doc/coreutils.texi (sort invocation): Likewise.
* src/sort.c (long_double, strtold): Move to prelude, since they're
now used by multiple functions.
(LD): New macro.
(struct keyfield.iec_present): Remove this member. All uses removed.
(check_mixed_SI_IEC): Remove. This code was busted in the presence
of multiple threads, as it had a race condition.
(find_unit_order): Remove arg KEY; add arg THOU_SEP; arg ENDPTR is
now char ** rather than char const **. Return an integer that
distinguishes decimal from binary powers. Parse the number
consistently with the intersection of strtold and strnumcmp.
Set *ENDPTR unconditionally.
(compute_human): New static function.
(human_numcompare): Remove arg KEY. Remove 'const' from other args.
Use strnumcmp if possible, but fall back on floating point if not.
(numcompare, general_numcompare): Arg EA is now char ** rather
than char const **.
(numcompare): Adjust to new find_unit_order signature and behavior.
(keycompare): Adjus to new human_numcompare signature.
* tests/misc/sort (h1, h3, h4, h6): Adjust to new behavior.
* tests/misc/sort-debug-keys: Likewise.
* src/sort.c (fillbuf): Don't append eol unless the line is nonempty.
This fixes a bug that was partly but not completely fixed by
the aadc67dfdb commit (dated July 15).
* tests/misc/sort (realloc-buf-2): New test, which catches this
bug on 64-bit hosts.
* src/sort.c (mergelines, queue_destroy, queue_init, queue_insert):
(queue_pop, write_unique, mergelines_node, check_insert):
(update_parent): No longer inline; these uses of "inline"
seemed unlikely to help performance much.
* gl/lib/heap.c (struct heap): Move this here...
* gl/lib/heap.h (struct heap): ... from here, as outside code no
longer needs to access any of these members.
* src/sort.c (queue_pop): Omit unnecessary unlock+lock after
pthread_cond_wait returns. Don't access "count" member of the
heap; any efficiency gains should be quite minor, the access
complicates this code, and "count" should be private anyway.
* src/sort.c (lock_node, unlock_node, queue_destroy, queue_init):
(queue_pop):
Omit 'restrict'; it shouldn't help here, as these functions have just
one pointer parameter and don't access static storage.
(queue_insert, check_insert, update_parent): Omit 'restrict', as
the pointer types differ, and are not char * or unsigned char *,
and therefore can't alias.
(write_unique): Omit 'restrict', as the pointer types are all
read-only.
(merge_loop, sortlines): Omit 'restrict', as any performance
advantages are extremely unlikely and it's not worth cluttering
the code for that.
(struct thread_args): Omit 'restrict': this seems to be incorrect.
It's unlikely for 'restrict' to be correct inside a typedef.
* src/sort.c (inittables, general_numcompare, compare_nodes):
(queue_init, queue_pop): Omit casts that are not needed, typically
because they are between void * and some other pointer type.
* src/du.c (process_file): Avoid recalculation of hashes
and of file-exclusion for directories. Do not descend into
the same directory more than once, unless -l is given; this is faster.
Calculate stat buffer lazily, since it
need not be computed at all for excluded files.
Count space if FTS_ERR, since stat buffer is always valid then.
No need for 'print' local variable.
(main): Use FTS_NOSTAT. Use FTS_TIGHT_CYCLE_CHECK only when not
hashing everything, since process_file finds cycles on its own
when hashing everything.
* tests/du/deref: Add test cases for -L bugs.
* tests/misc/sort-merge-fdlimit: This test was written assuming that
-R typically opens /dev/urandom, but that's no longer the case.
Redo test to specify a random source; this resurrects the point of
checking for file descriptor exhaustion. Also try plain -R, since
that implementation may change in the future too.
* gl/lib/rand-isaac.c: Remove the I/O; this belongs elsewhere.
Add support for ISAAC64. Port to hosts with padding bits.
Add self to author list. Include <limits.h>, for CHAR_BIT.
Don't include string.h, sys/time.h, unistd.h.
(min, just): New functions.
(IF32): New macros.
(ind, ISAAC_STEP, isaac_refill, mix, isaac_init, isaac_seed):
Add support for ISAAC64. Port to hosts with padding bits.
(ind): Now an inline function rather than a macro; no need for it
to be a macro with modern compilers.
(ISAAC_STEP): Renamed from isaac_step, since it's not function-like.
Don't bother to pass args that are always the same. All uses changed.
(ISAAC_STEP, ISAAC_SEED): Move to inside the only function body
that can use it.
(ISAAC_MIX): Renamed from isaac_mix, since it's now a macro and is
no longer function-like. Don't bother saving and restoring state;
no longer needed now that we're not a function. All uses changed.
(isaac_seed_start, isaac_seed_data, isaac_seed_finish): Remove.
(isaac_seed): Take just the one arg; the caller now sets s->m.
* gl/lib/rand-isaac.h: Use _GL_RAND_ISAAC_H to protect, instead
of RAND_ISAAC_H. Try out " #" rather than "# " for indenting.
(ISAAC_BITS_LOG, ISAAC_BITS): New macros.
(ISAAC_WORDS_LOG): Renamed from ISAAC_LOG.
(isaac_word): New type. All uses of uint32_t changed to isaac_word,
to support ISAAC64.
(struct isaac_state): Rename member MM to M, and make it public.
(isaac_seed, isaac_refill): Adjust to new API.
* gl/lib/randread.c: Include sys/time.h.
(get_nonce): New function, containing the nonce stuff that used
to be in rand-isaac.c but better belongs here.
(randread_new): Use it.
* gl/modules/randread (Depends-on): Add inline.
* gl/modules/randread-tests: New file.
* gl/tests/test-rand-isaac.c: New file.
* bootstrap.conf: Include the new module
* gl/lib/fadvise.c: Provide a simpler interface to posix_fadvise.
(fadvise): Provide hint to the whole file associated with a stream.
(fdadvise): Provide hint to the specific portion of a file
associated with a file descriptor.
* gl/lib/fadvise.h: Redefine POSIX_FADV_* to FADVISE_* enums.
* gl/modules/fadvise: New file.
* m4/jm-macros.m4: Remove the no longer needed posix_fadvise check.
* .x-sc_program_name: Exclude test-fadvise.c from this check.
* gl/tests/test-fadvise (main): New test program.
* gl/modules/fadvise-testss: A new index to reference the tests.
* src/sort.c (stream_open): Use the new interface.
* src/dd.c (iwrite): Likewise.
* configure.ac (optional_pkglib_progs): Only update
after the main programs have been selected, so that
libstdbuf.so can be excluded if stdbuf also is.
* gl/lib/rand-isaac.c (isaac_seed_start): New arg SEEDED.
(isaac_seed): New args FD and BYTES_BOUND. Read from FD if possible.
Don't bother with low-quality sources if FD has enough bytes.
* gl/lib/rand-isaac.h: New size_t arg for isaac_seed.
* gl/lib/randread.c: Include fcntl.h, unistd.h.
(NAME_OF_NONCE_DEVICE): New #define.
(nonce_device): New static var.
(randread_new): Use nonce device if available.
* src/sort.c (random_md5_state): New static var.
(random_md5_state_init): New function, to initialize random_md5_state.
(random_state, randread_source): Remove.
(cmp_hashes): Use random_md5_state rather than random_state.
Break ties using memcmp, not by getting more randomness.
If MD5 collisions turn into a problem in practice, we should
simply use a better checksum.
(main): If -R is given, call random_md5_state_init rather than
going single-threaded.
Programs like 'sort' were linking to -lrt in order to get
clock_gettime, but this was misguided: it wasted considerable
resources while gaining at most 10 bits of entropy. Almost nobody
needs the entropy, and there are better ways to get much better
entropy for people who do need it.
* gl/lib/rand-isaac.c (isaac_seed): Include <sys/time.h> not
"gethrxtime.h".
(isaac_seed): Use gettimeofday rather than gethrxtime.
* gl/modules/randread (Depends-on): Depend on gettimeofday
and not gethrxtime.
* src/Makefile.am (mktemp_LDADD, shred_LDADD, shuf_LDADD, sort_LDADD):
(tac_LDADD): Omit $(LIB_GETHRXTIME); no longer needed.
* src/sort.c (keycompare): Use xmemcoll0, as it avoids
a couple of stores.
(write_bytes): Leave the buffer the way we found it,
as it might be used again for a later comparison,
if -u is used.
Don't write NUL after the comparison buffers on each compare,
which increases performance by about 3% for short lines
on a pentium-m with gcc-4.4.1
* src/sort.c: (fillbuf): Delimit input items with NUL.
(write_bytes): Restore the item delimiter char which was
replaced with NUL in fillbuf().
* src/Makefile.am (printf_LDADD, seq_LDADD, sleep_LDADD, sort_LDADD):
(tail_LDADD, uptime_LDADD): Omit $(POW_LIB), as it's no longer
needed due to recent gnulib changes, where the strtod module no
longer uses the pow function. strtold needs pow only because it's
sometimes aliased to strtod. See
http://lists.gnu.org/archive/html/bug-gnulib/2010-07/msg00076.html
* NEWS: Add another blank line before the previous version.
(Bug fixes): Move to the start.
(Changes in behavior): Add the item about the du mem usage change
from the "New features" section.
* gl/lib/heap.c (heap_alloc): Use the fact that the xalloc
routines will not return NULL. Also remove the redundant
temporary variables.
(heap_insert): From Jim Meyering, use x2nrealloc() which
is simpler while handling overflow and increasing the
size more efficiently. This reallocation is currently
unused by coreutils in any case as it preallocates enough.
This patch is by Gene Auyeung, Chris Dickens, Chen Guo, and Mike
Nichols, based off of a patch by Paul Eggert, Glen Lenker, et. al.,
with a basic heap implementation based off of the GDSL heap,
originally by Nicolas Darnis.
The number of sorts done in parallel is limited to the number
of available processors by default, or can be further restricted
with the --parallel option.
On a dual-die, 8 core Intel Xeon, results show sorting with
8 threads is almost 4 times faster than using a single thread.
Timings when sorting a 96MB file:
THREADS TIME (s)
1 5.10
2 2.87
4 1.75
8 1.31
Single threaded sorting has also been improved,
especially for cheaper comparison operations:
COMMAND BEFORE (s) AFTER (s)
sort 8.822 8.716
sort -g 10.336 10.222
sort -n 3.077 2.961
LANG=C sort 2.169 2.066
* bootstrap.conf: Add heap, pthread.
* coreutils.texi (sort): Describe the new --parallel option.
* gl/lib/heap.c: New file. Very basic heap implementation.
* gl/lib/heap.h: New file.
* gl/modules/heap: New file.
* src/Makefile.am: Add LIB_PTHREAD.
* src/sort.c: Include heap.h, nproc.h, pthread.h.
(MAX_MERGE): New macro.
(SUBTHREAD_LINES_HEURISTIC, PARALLEL_OPTION): New constants.
(MERGE_END, MERGE_ROOT): New constants.
(struct merge_node): New struct.
(struct merge_node_queue): New struct.
(sortlines temp): Remove declaration.
(usage, long_options, main): New option, --parallel.
(specify_nthreads): New function.
(mergelines): New signature, to emphasize the fact that the HI area
must be part of the destination. All callers changed.
(sequential_sort): New function, renamed from sortlines. Merge in
the functionality of sortlines_temp.
(compare_nodes): New function.
(lock_node, unlock_node): New functions.
(queue_destroy): New function.
(queue_init): New function.
(queue_insert): New function.
(queue_pop): New function.
(write_unique): New function.
(mergelines_node): New function.
(check_insert): New function.
(update_parent): New function.
(merge_loop): New function.
(sortlines): Rewrite to support and use parallelism, with a new
signature. All callers changed.
(struct thread_args): New struct.
(sortlines_thread): New function.
(sortlines_temp): Remove.
(sort): New argument NTHREADS. All uses changed. Output moved to
mergelines_node.
(main): disable threading if we are sorting at random.
* tests/Makefile.am (TESTS): Add misc/sort-benchmark-random.
* tests/misc/sort-benchmark-random: New file.
Signed-off-by: Pádraig Brady <P@draigBrady.com>
* gl/lib/ino-map.c (ino_hash): Declare "i" as unsigned int.
Use an intermediate variable for the for-loop upper bound,
so it's a little more readable. Adjust comment.
* gl/lib/di-set.c (di_ent_hash): Likewise.
* src/dd.c (dd_copy): Use requested blocksize (not adjusted) in
diagnostic, to forestall user complaints that the numbers don't
match exactly. Report both exact and human-readable sizes, using
a message format that is consistent with both "BBBB bytes (N XB)
copied" in dd.c and "memory exhausted" in lib/xmalloc.c.
Problem reported by Jim Meyering in:
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=6524#74
* gl/lib/di-set.c (di_ent_hash): Rework so that the compiler does
not incorrectly warn about shifting by 64-bits in unreachable code.
* gl/lib/ino-map.c (ino_hash): Likewise.
* gl/lib/dev-map.c, gl/lib/dev-map.h, gl/modules/dev-map: Remove.
* gl/lib/ino-map.c, gl/lib/ino-map.h, gl/modules/ino-map: New files.
* gl/modules/dev-map-tests, gl/tests/test-dev-map.c: Remove.
* gl/modules/ino-map-tests, gl/tests/test-ino-map.c: New files.
* gl/lib/di-set.h (struct di_set): Renamed from struct di_set_state,
and now private. All uses changed.
(_ATTRIBUTE_NONNULL_): Don't assume C99.
(di_set_alloc): Renamed from di_set_init, with no size arg.
Now allocates the object rather than initializing it.
For now, this no longer takes an initial size; we can put this
back later if it is needed.
* gl/lib/di-set.c: Include hash.h, ino-map.h, and limits.h instead of
stdio.h, assert.h, stdint.h, sys/types.h (di-set.h includes that
now), sys/stat.h, and verify.h.
(N_DEV_BITS_4, N_INO_BITS_4, N_DEV_BITS_8, N_INO_BITS_8): Remove.
(struct dev_ino_4, struct dev_ino_8, struct dev_ino_full): Remove.
(enum di_mode): Remove.
(hashint): New typedef.
(HASHINT_MAX, LARGE_INO_MIN): New macros.
(struct di_ent): Now maps a dev_t to a inode set, instead of
containing a union.
(struct dev_map_ent): Remove.
(struct di_set): New type.
(is_encoded_ptr, decode_ptr, di_ent_create): Remove.
(di_ent_hash, di_ent_compare, di_ent_free, di_set_alloc, di_set_free):
(di_set_insert): Adjust to new representation.
(di_ino_hash, map_device, map_inode_number): New functions.
* gl/modules/di-set (Depends-on): Replace dev-map with ino-map.
Remove 'verify'.
* gl/tests/test-di-set.c: Adjust to the above changes to API.
* src/du.c (INITIAL_DI_SET_SIZE): Remove.
(hash_ins, main): Adjust to new di-set API.
Add comments and adjust interfaces to allow low-level failure
to propagate out to callers.
* src/stat.c (out_file_context): Return bool, not void,
so we can tell callers about failure.
(print_statfs, print_stat, print_it): Propagate failure to caller.
(do_statfs): Propagate print_it failure to caller.
(do_stat): Likewise.
I nearly forgot to update do_stat to propagate print_it failure,
and it compiled just fine in spite of that. To prevent possibility
of a repeat, I've marked each function that returns non-void with
ATTRIBUTE_WARN_UNUSED_RESULT.
* tests/init.cfg: Introduce a retry_delay_() function to
repeatedly call a test function that requires a delay.
This delay can now be shorter for the common case on fast
systems, but will double until a configurable limit it reached
before failing on slower systems.
* tests/dd/reblock: Use retry_delay_.
* tests/misc/cat-buf: Likewise.
* tests/misc/stdbuf: Likewise.
* tests/tail-2/F-vs-rename: Likewise.
* tests/tail-2/flush-initial: Likewise.
* tests/tail-2/tail-n0f: Likewise.
* tests/tail-2/wait: Likewise.
* test/dd/misc: Comment that delay is needed to trigger failure.
* src/du.c (INITIAL_DI_SET_SIZE): Increase to the prime just under
1024. This gives a speed-up of about 2% when processing a tree
containing 100,000 files, each with a link count greater than 1,
all pointing to files in some other tree.
When processing a hard-linked file, du must keep track of the file's
device and inode numbers in order to avoid counting its storage
more than once. When du would process many hard linked files --
as are created by some backup tools -- the amount of memory required
for the supporting data structure could become prohibitively large.
This patch takes advantage of the fact that the amount of information
in the numbers of the typical dev,inode pair is far less than even
32 bits, and hence usually fits in the space of a pointer, be it
32 or 64 bits wide. A typical du traversal examines files on no
more than a handful of distinct devices, so the device number can
be encoded in just a few bits. Similarly, few inode numbers use
all of the high bits in an ino_t. Before, we would represent the
dev,inode pair using a naive struct, and allocate space for each.
Thus, an entry in the hash table consisted of a pointer (to that
struct) and a "next" pointer. With this change, we encode the
dev,inode information and put those bits in place of the pointer,
and thus do away with the need to allocate additional space for
each dev,inode pair.
* src/du.c: Include "di-set.h".
Don't include "hash.h"; it's no longer used.
(INITIAL_DI_SET_SIZE): Define.
(di_set): New global, to replace "htab".
(entry_hash, entry_compare, hash_init): Remove functions.
(hash_ins): Use di-set functions, rather than ones from the hash module.
(main): Likewise.
* bootstrap.conf (gnulib_modules): Add the new di-set module.
* NEWS (New features): Mention it.
* NEWS: Mention this.
* src/du.c (hash_all): New static var.
(process_file): Use it.
(main): Set it.
* tests/du/hard-link: Add a couple of test cases to help make
sure this bug stays squashed.
* tests/du/files0-from: Adjust existing tests to reflect
change in semantics with duplicate arguments.
* src/copy.c (copy_attr): A new function which merges copy_attr_by_fd
and copy_attr_by_name. Also display all errors when --attributes-only
* src/copy.c (copy_reg): Skip copying the file contents if specified.
Refactor the SELinux error handling code a little and display all
SELinux errors when only copying attributes.
* src/copy.h (struct cp_options): Add a data_copy_required boolean
* src/cp.c (main): Default to copying data but don't if specified
* src/install.c: Default to copying data
* src/mv.c: Likewise
tests/cp/reflink-perm: Add a test to check that --attributes-only
does not copy data
* tests/cp/acl: Likewise. Also refactor to remove redundant
acl manipulation
* doc/coreutils.texi (cp invocation): Describe the new option
* NEWS: Mention the new feature
Previously we defaulted to "long-iso" format in locales without
specific format translations, like the en_* locales for example.
This reverts part of commit 6837183d, 08-11-2005, "ls ... acts like
--time-style='posix-long-iso' if the locale settings are messed up"
* src/ls.c (decode_switches): Only use the ISO format when specified.
* NEWS: Mention the change in behavior.
Reported by Daniel Qarras at http://bugzilla.redhat.com/525134
* tests/tail-2/inotify-rotate: Switch to new init.sh-based framework.
(grep_timeout): New function.
Use it in place of open-coded loops that might infloop.
This was prompted by my encountering an inexplicable, and so far
unreproducible, infloop in the code that was waiting for "b" to
appear in "out".
From there, they will be used by both test-lib.sh (as we phase it out)
and the newer init.sh, to which all tests will migrate.
* tests/test-lib.sh: Move most functions from here, ...
* tests/init.cfg: ...to here. New file.
* tests/Makefile.am (EXTRA_DIST): Add init.cfg.
* src/stat.c (main): Remove support for the --context (-Z) option.
In upstream releases this option has always been a no-op. It was
first ignored for compatibility, and since the June 2008 commit,
574f7614 (coreutils-7.0), its use has evoked a warning.
* NEWS (Changes in behavior): Mention it.
* src/comm.c (usage): Don't align example comments in --help output,
since the extra space (sequence of two spaces) there would be
interpreted by help2man and induce an unwanted line break
in the resulting man page. Reported by Jari Aalto.
* src/tail.c (xlseek): Give INT_BUFSIZE_BOUND a variable name,
not a type name.
* src/ls.c (gobble_file, format_user_or_group_width): Likewise.
* src/head.c (elide_tail_bytes_pipe): Likewise.
(elide_tail_lines_seekable, main): Likewise.
[This change is not complete -- there are doubtless other uses
that can be updated in the same way.]
sprintf is relatively heavy-weight.
* src/sort.c (key_warnings): Use umaxtostr and stpcpy rather
than sprintf.
Also, replace each INT_BUFSIZE_BOUND "type_name" argument
with the equivalent variable name. More maintainable that way.
* src/touch.c (main): Remove support for the deprecated, long-named
--file option, which is an alternate name for --reference (-r).
That option was undocumented with the arrival of --reference, in
the 1995-10-29 commit, 8b92864e1d. Since the 2009-02-09 commit,
ed85df444a, use of --file has elicited a warning. Not only was
this code due for removal, but the long-name-use-detecting code
was buggy in that it would use a stale or uninitialized "long_idx",
as reported by Robin H. Johnson in http://bugs.gentoo.org/322421.
* NEWS (Changes in behavior): Mention it.
* src/stat.c (alignof): Remove definition.
Instead, include "alignof.h", and sort the #include directives.
And get its definition from the gnulib module by that name:
* bootstrap.conf (gnulib_modules): Add alignof.
Previously we copied `dd` and suppressed error messages
when truncating neither regular files or shared mem objects.
This was valid for `dd`, as truncation is ancillary to copying
it may also do, but for `truncate` we should display all errors.
Also we used the st_size from non regular files which is undefined,
so we display an error when the user tries this.
* src/truncate (do_truncate): Error when referencing the size
of non regular files or non shared memory objects. Display all
errors returned by ftruncate().
(main): Error when referencing the size of non regular files or
non shared memory objects. Don't suppress error messages for
any file types that can't be opened for writing.
* tests/misc/truncate-dir-fail: Check that referencing the
size of a directory is not supported.
* tests/misc/truncate-fifo: Ensure the test doesn't hang
by using the `timeout` command. Don't test the return from
running ftruncate on the fifo as it's system dependent as
to whether this fails or not.
NEWS: Mention the change in behavior.
Reported by Jim Meyering.
* doc/coreutils.texi (truncate invocation): Mention that --reference
bases the --size rather than just setting it.
* src/truncate.c (usage): Likewise. Also remove the clause
describing --size and --reference as being mutually exclusive.
(do_truncate): Add an extra parameter to hold the size
of a referenced file, and use it if positive.
(main): Pass the size of a referenced file to do_truncate().
* tests/misc/truncate-parameters: Adjust for the new combinations.
* NEWS: Mention the change
Suggested by Richard W.M. Jones
* src/sort.c (key_warnings): Always warn about significant leading
blanks when character offsets are specified, unless they key is
possibly a line offset, i.e. of the form -k1.x,1.y. Also suppress
this warning if the user could be sorting right aligned indexes.
* tests/cp/cp-a-selinux: Initialize skip, to avoid a syntax error
in subsequent "test".
Remove redirect-to-/dev/null, now that output is always to a log file.
* src/sort.c (usage): Mention --debug can output warnings to stderr.
Also split the translatable string to aid translation.
(default_key_compare): A new function refactored from main(),
and now also called from the new key_warnings() function.
(key_to_opts): A new function refactored from incompatible_options(),
and now also called from the new key_warnings() function.
(key_numeric): A new function refactored to test if key is numeric.
(key_warnings): A new function to output warnings to stderr,
about questionable use of various options. Currently it warns
about zero length keys and ineffective global options.
(incompatible_options): Refactor out key_to_opts()
(main): Use key_init() to initialize gkey. Refactor out
default_key_compare(). Call key_warnings() in debug mode.
* doc/coreutils.texi (sort invocation): Mention that warnings
are output by --debug.
* tests/misc/sort-debug-warn: A new test for debug warnings.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the new feature
* src/sort (usage): Add description for --debug.
(write_bytes): Pass a line structure so it can subsequently
be passed to compare to highlight the keys when in debug mode.
Also transform TAB and NUL characters written to stdout so
that the highlighting in debug mode aligns correctly.
(human_numcompare): Pass an "endptr" so we can record the extent
of the number matched.
(general_numcompare): Likewise.
(find_unit_order): Likewise.
(getmonth): Likewise.
(numcompare): Likewise. Note we reuse find_unit_order() for this,
which is a good enough approximation, and means we don't need to
change the strnumcmp() interface.
(check_mixed_SI_IEC): Return whether iec_present, so that can be
used to set the "endptr" in find_unit_order. Also make the key
parameter optional, which will be the case from numcompare().
(count_tabs): A new function to determine how much to adjust
the mbswidth() values by (TABs don't have a width).
(mark_key): A new function to output the key highlighting to stdout.
(debug_key): A new function to determine the offset and width
of the key highlighting.
(key_compare): Pass the show_debug parameter so the key highlighting
is only displayed when explicitly called. For each key type, set
the length (lena) and whether leading blanks are auto skipped (skipb)
which are then used by debug_key() to highlight the portion of the
key used in the comparison.
(compare): Pass the show_debug parameter so the key highlighting
is only displayed when explicitly called. Call debug_key() to
highlight the last resort comparison.
(check): Output highlighting for disorder line to stdout.
(main): Process the --debug option and make it mutually exlusive
with the -o option as I don't see it useful there, even potentially
harmful if someone left a --debug in by mistake when updating a file.
Also restricting debug output to stdout, simplifies the logic
for dealing with temporary files.
* doc/coreutils.texi (sort invocation): Describe the --debug option,
and reference it from the --key description.
* tests/misc/sort-debug-keys: A new test for highlighting keys.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the new feature.
* cfg.mk (.re-defmac): Generate better regexps: allow white space
before the '#', and append a word-boundary requirement.
Without the latter, #define NULL_DEV ... would evoke a false-positive.
* cfg.mk (sc_always_defined_macros): Adjust its helpers not to depend
on the existence of ./lib. Instead, extract symbols directly from
gnulib/lib/*.in.h files.
using the new --mail-headers option to gnulib's announce-gen, and
the updated maint.mk rules to connect the pieces.
* README-release: Remove hard-coded To:, Cc: etc. parts, now
that they're emitted automatically into the announcement template.
* cfg.mk (announcement_Cc_): Override the default.
* gnulib: Update to latest, to get newer announce-gen and maint.mk.
* cfg.mk (gl_trap_): Define, using a loop and eval'd trap,
rather than repeated "trap" uses. Also handle "13", SIGPIPE.
(sc_always_defined_macros): Use it.
(sc_system_h_headers): Likewise.
* cfg.mk (gl_generated_headers_): Define.
(headers_with_interesting_macro_defs): Remove headers covered
by the above.
(.re-defmac): Extract symbol names from many more files.
(sc_always_defined_macros): Use VC_LIST_EXCEPT, not VC_LIST, so
that we can use the usual exception mechanism.
Test for $(gnulib_dir), not system.h.
* .x-sc_always_defined_macros: New file. Exempt src/seq.c.
* Makefile.am (syntax_check_exceptions): Add it here.
* src/dd.c (SA_NODEFER, SA_RESETHAND): Remove definitions,
now that gnulib guarantees they are defined in <signal.h>.
* src/ls.c (SA_RESTART): Likewise.
* src/timeout.c (WIFSIGNALED, WTERMSIG): Remove definitions,
now that gnulib guarantees they are defined in <sys/wait.h>.
* src/operand2sig.c: Likewise.
* src/kill.c: Likewise.
* src/sort.c (general_numcompare): Use long doubles unconditionally,
and strtold when available, to convert numbers with greater range and
precision. Performance was seen to be on par with standard doubles.
* doc/coreutils.texi (sort invocation): Amend the -g description to
mention long double rather than double, and strtold rather than strtod.
* src/getlimits.c (main): Output floating point limits for use in tests.
* tests/misc/sort-float: A new test to ensure sort is using long
doubles when possible, and that locale specific floats are handled.
* tests/Makefile.am: Reference the new test.
* tests/test-lib.sh (getlimits_): Normalize indenting.
* NEWS: Mention the new behaviour.
Reported by Nelson Beebe.
* doc/coreutils.texi (factor invocation): Don't say that "factoring
large prime numbers is hard". A pedant might ding you, since it's
trivial to factor a number that is known to be prime. Instead, say
that "factoring large numbers... is hard". Reported by Andreas Eder.
* bootstrap.conf (gnulib_modules): Add the following:
netinet_in, sys_ioctl, sys_wait, so that we can eliminate
the #if HAVE_<header>_H tests guarding their header inclusions.
The value of `$?' on entrance to signal handlers in shell scripts
cannot be relied upon, so set the exit code explicitly.
* cfg.mk (sc_always_defined_macros, sc_system_h_headers): Set
the exit code in signal handler explicitly to 128 + SIG<SIGNAL>.
* src/Makefile.am (sc_tight_scope): Likewise.
* tests/test-lib.sh: Likewise.
Necessary for cygwin. Technically, this patch is not correct,
in that it clobbers O_APPEND, but it is no different than any
other use of xfreopen to force binary mode, so all such uses
should be fixed at once in a later patch.
* src/base64.c (main): Open input in binary mode.
* THANKS: Update.
Reported by Yutaka Amanai.
* NEWS: Mention that cp and mv from the previous release did
not support preserving extended attributes (fixed in e489fd04).
Improve the grammar for the "cp capabilities" item.
This regression was introduced in commit 224a69b5, 2009-02-24,
"sort: Fix two bugs with determining the end of field".
The specific regression being that we include 1 field too many when
an end field is specified using obsolescent key syntax (+POS -POS).
* src/sort.c (struct keyfield): Clarify the description of the eword
member, as suggested by Alan Curry.
(main): When processing obsolescent format key specifications,
normalize eword to a zero based count when no specific end char is given
for an end field. This matches what's done when keys are specified with -k.
* tests/misc/sort: Add a few more tests for the obsolescent key formats,
with test 07i being the particular failure addressed by this change.
* THANKS: Add Alan Curry who precisely identified the issue.
* NEWS: Mention the fix.
Reported by Santiago Rodríguez
* src/copy.c (copy_reg): Copy xattrs _after_ setting file ownership
so that capabilities are not cleared when setting ownership.
* tests/cp/capability: A new root test.
* tests/Makefile.am (root_tests): Reference the new test.
* NEWS: Mention the fix.
* src/Makefile.am (kill_LDADD): Add $(LIBTHREAD) so that
we link with the appropriate libraries to provide Thread Local Storage
on platforms that replace strsignal (like AIX for example).
Tested-by: Daniel Richard G. <danielg@teragram.com>
* tests/ls/color-norm: Use the "time" output by `ls -l`
to check normal style. Previously we used the size from `ls -s`,
but the size of "empty" files can vary depending on whether
SELinux is enabled for example.
* tests/ls/capability: Adjust this test not to expect the no-op escape
sequence that was removed from all other tests by 2010-01-30 commit
5d43617e, "ls --color: don't emit a final no-op escape sequence".
* src/copy.c (copy_reg): Suppress SELinux ENOTSUP warnings consistently
between the destination being present or not. Previously we did
not suppress ENOTSUP messages when the destination was present.
(copy_internal): Use the same ENOTSUP supression method as
copy_reg() even though the issue was not seen in this case.
* tests/cp/cp-a-selinux: Add a test case for the issue and
group the other test cases in the file more coherently.
* tests/cp/cp-mv-enotsup-xattr: Do the same check for xattr
warnings, even though they did not have the issue.
The 2010-03-26 commit, 4c38625e, "doc: fix info on cp --preserve..."
was not entirely correct as cp --preserve=all does produce some
xattr warnings.
* src/copy.h: Update and clarify the comments for reduce_diagnostics
and require_preserve_{xattr,context}.
* doc/coreutils.texi (cp invocation): Update the -a and
--preserve=xattr,context options to say when and which
xattr warnings are output.
(mv invocation): Mention that some warnings are output
when preserving xattrs.
* doc/coreutils.texi (cp invocation): Mention that
"capabilities" are preserved when implemented using
extended attributes.
(mv invocation): Mention ACLs etc. are maintained
due to xattrs being copied.
* cfg.mk (detect_empty_lines_at_EOF_): Define.
(sc_prohibit_empty_lines_at_EOF): New rule.
* .x-sc_prohibit_empty_lines_at_EOF: New file. Exempt pr test inputs.
* Makefile.am (syntax_check_exceptions): Add it.
Pádraig Brady suggested to parse the output of tail -n1.
Now that even MinGW provides ftruncate, we know that all
reasonable portability targets provide this function.
Remove the workaround code. We nearly removed the gnulib
module three years ago:
http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/9203
and it is now officially "obsolete".
* bootstrap.conf (gnulib_modules): Remove ftruncate.
* src/copy.c (copy_reg): Remove use of HAVE_FTRUNCATE and its
no-longer-used workaround code.
* src/truncate.c: Remove a comment about handling missing ftruncate.
* m4/jm-macros.m4 (ARGMATCH_DIE): Use usage(EXIT_FAILURE), not usage(1).
* .x-sc_prohibit_magic_number_exit: Remove *.m4 exemption that was
masking the above.
* cfg.mk: Update to use new _sc_search_regexp interface. Run this:
perl -pi -e 's/\b_prohibit_regexp\b/_sc_search_regexp/;'
-e 's/\bmsg=/halt=/; s/\bre=/prohibit=/;' cfg.mk
and then adjust backslashes so they still line up.
Related to the 2010-03-25 commit, 88d4b346,
"timeout: use more standard option parsing".
* src/nice.c (main): Don't use parse_long_options()
which is a helper for commands that don't have any
long options specific to them.
* src/chroot.c (main): Likewise.
* tests/misc/nice-fail: Remove a case that now
passes due to us accepting multiple instances of the
--help and --version options.
* tests/misc/chroot-fail: Likewise.
* src/timeout.c (main): Don't use parse_long_options()
which is a helper for commands that don't have any
long options specific to them.
* tests/misc/timeout-parameters: Remove a case that now
passes due to us accepting multiple instances of the
--help and --version options.
* THANKS: Add the author.
Signed-off-by: Pádraig Brady <P@draigBrady.com>
The info docs have been inaccurate since 2009-04-17, commit 941bd482,
"mv: ignore xattr-preservation failure when not supported by filesystem"
* doc/coreutils.texi (cp invocation): Say that cp --preserve=all
does _not_ output errors when failing to copy xattrs.
* cfg.mk (_sed_remove_comments): Define, starting with gettext's
moopp sed code, but factoring it to be more understandable.
(sc_space_before_open_paren): Adapt.
Prompted by Bruno Haible's suggestion to use gettext's moopp code.
* gl/lib/mbsalign.c (mbsalign): Ensure the temporary destination buffer
is big enough, as it may need to be bigger than the source buffer
in the presence of single byte non printable chars.
* gl/tests/test-mbsalign.c (main): Add a test to trigger the issue.
* gl/lib/mbsalign.c (mbsalign): Support the MBA_UNIBYTE_FALLBACK
flag which reverts to unibyte mode if one can't allocate memory
or if there are invalid multibyte characters present.
Note memory is no longer dynamically allocated in unibyte mode so
one can assume that mbsalign() will not return an error if this
flag is present. Don't calculate twice, the number of spaces,
when centering. Suppress a signed/unsigned comparison warning.
(ambsalign): A new wrapper function to dynamically allocate
the minimum memory required to hold the aligned string.
* gl/lib/mbsalign.h: Add the MBA_UNIBYTE_FALLBACK flag and
also document others that may be implemented in future.
(ambsalign): A prototype for the new wrapper.
* gl/tests/test-mbsalign.c (main): New test program.
* gl/modules/mbsalign-tests: A new index to reference the tests.
* .x-sc_program_name: Exclude test-mbsalign.c from this check.
* src/rm.c (usage): Update wording to make two points more
apparent: undelete is not trivial, and partial recovery should be
a consideration factor in deciding whether rm is secure enough.
Initially suggested by Reuben Thomas.
Based on a report from Kim Hansen who wanted to
send a KILL signal to the monitored command
when `timeout` itself received a termination signal.
Rather than changing such a signal into a KILL,
we provide the more general mechanism of sending
the KILL after the specified grace period.
* src/timeout.c (cleanup): If a non zero kill delay
is specified, (re)set the alarm to that delay, after
which a KILL signal will be sent to the process group.
(usage): Mention the new option. Separate the description
of DURATION since it's now specified in 2 places.
Clarify that the duration is an integer.
(parse_duration): A new function refactored from main(),
since this logic is now called for two parameters.
(main): Parse the -k option.
* doc/coreutils.texi (timeout invocation): Describe the
new --kill-after option and use @display rather than
@table to show the duration suffixes. Clarify that
a duration of 0 disables the associated timeout.
* tests/misc/timeout-parameters: Check invalid --kill-after.
* tests/misc/timeout: Check a valid --kill-after works.
* NEWS: Mention the new feature.
Tell the system that we'll access input sequentially,
so that we more efficiently process uncached files in a few cases:
Reading from faster flash devices. E.g. 21 MB/s key:
NORMAL 31.6s (26.8 user)
SEQUENTIAL 27.7s
WILLNEED 27.7s
Processing in parallel with readahead when using a small 1M buffer:
NORMAL 24.7s (21.1 user)
SEQUENTIAL 22.7s
WILLNEED 25.6s
A small benefit when merging:
NORMAL 25.0s (16.9 user)
SEQUENTIAL 24.6s (16.6 user)
WILLNEED 38.4s (13.1 user)
Note WILLNEED is presented above for comparison to show it
has some unwanted characteristics due to its synchronous
prepopulation of the cache. It has a good benefit on a
mechanical disk @ 80MB/s and a multicore system with
competing processes:
NORMAL 14.73s
SEQUENTIAL 10.95s
WILLNEED 05.22s
However the scheduling differences causing this result
are probably best explicitly managed using `nice` etc.
* m4/jm-macros.m4 (coreutils_MACROS): check for posix_fadvise().
* src/sort.c (fadvise_input): A new function to apply
the POSIX_FADV_SEQUENTIAL hint to an input stream.
(stream_open): Call the above function for all input streams.
* tests/envvar-check (vars): Add LANGUAGE to the list of envvars
to unset. At least in glibc (as an extension to POSIX), its value
actually trumps LC_ALL:
$ LC_ALL=es_ES LANGUAGE=fr_FR.UTF-8 /bin/cat no-such
/bin/cat: no-such: Aucun fichier ou dossier de ce type
but only when the default locale is not C:
$ LC_ALL=C LANGUAGE=fr_FR.UTF-8 /bin/cat no-such
/bin/cat: no-such: No such file or directory
Prompted by a report from Mads Kiilerich.
* src/sort.c (char fold_toupper[]): Change to unsigned
so as the correct comparisons are made in getmonth().
This fixes unibyte locales where abbreviated months
have characters that are > 0x7F, but it also works for
multibyte locales with the caveat that multibyte characters
are matched case sensitively.
With this change, the following example sorts correctly:
$ echo -e "1 márta\n2 Feabhra" | LANG=ga_IE.utf8 sort -k2,2M
2 Feabhra
1 márta
* src/sort.c (inittables): Since we ignore blanks around months
in the input, don't include them when they're present in the locale.
With this change, the following example sorts correctly:
$ echo -e "1 2月\n2 1月" | LANG=ja_JP.utf8 sort -k2,2M
2 1月
1 2月
* tests/misc/sort-month: A new test to exercise the above cases.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the fix.
* src/expr.c (eval4, eval3): Clarify that expr expects integers,
and not the broader category of numbers.
* tests/misc/expr: Update test accordingly.
Suggested by Dan Jacobson.
* cfg.mk (sc_tight_scope): Pass the -s (silent) flag to `make`
so that it doesn't report about calling sub makes.
(sc_check-AUTHORS): Likewise.
(sc_strftime_check): Don't display stderr from `info`.
* src/Makefile.am (sc_tight_scope): Don't annotate with "GEN".
(sc_check-AUTHORS): Likewise.
Output the NORMAL attribute before non file name text.
This attribute will continue into file names that would
not otherwise be colored unless FILE is also set.
The regression was introduced with commit 483297d5, 28-02-2009,
"ls --color no longer outputs unnecessary escape sequences".
* src/ls.c (set_normal_color): A new function to output the
NORMAL attribute sequence if it's enabled.
(print_current_files): Output NORMAL before printing long format info.
(print_file_name_and_frills): Output NORMAL before printing file name.
(print_color_indicator): Reset the attributes before a file name with
attributes so that NORMAL attributes will not combine with them.
(print_name_with_quoting): Ensure attributes are reset after printing
the file name if NORMAL attributes were output.
* tests/ls/color-norm: A new test for NORMAL and FILE combinations.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the fix.
Reported in https://savannah.gnu.org/bugs/?26512
These checks were not being run as distcheck-hook targets
are only supported in the top-level Makefile. Instead
these tests are now run during a syntax-check.
* cfg.mk (sc_man_file_correlation): A new syntax check to
call the 2 existing tests to check the correlation between
the programs and man/*.[1x].
* man/Makefile.am (sc_man_file_correlation): Call the 2 existing
man page correlation tests.
(check-x-vs-1): Remove the "GEN" annotation as it's a bit verbose.
(check-programs-vs-x): Likewise.
* src/Makefile.am (all_programs.list): Exclude libstdbuf.so
from the list of programs. This issue was not noticed as
the checks were not actually being run.
* tests/tail-2/inotify-hash-abuse2: Explicitly kill the process
by using cleanup_() rather than using a timeout which may trigger
a failure on very slow systems (< 20 iterations of the loop per second).
* src/base64.c (usage): Don't capitalize the first character
in an --option description.
* src/stdbuf.c (usage): Likewise.
* src/truncate.c (usage): Likewise.
* cfg.mk (sc_option_desc_uppercase): A new syntax check to
stop this happening in future.
* man/Makefile.am (sc_option_desc_uppercase): Ensure all
man pages are generated, then search for erroneous uppercase chars.
* src/Makefile.am (all_programs): Ensure all
commands are built so that all man pages can be generated.
* src/join.c (usage): Mention "fields" rather than repeating "line"
so that it's more obvious that the fields are still parsed, and
thus -o is still honored for headers. Also remove an extraneous
'.' reported by Stéphane Raimbault.
* src/base64.c (usage): Remove extraneous blank line and order
the options alphabetically. Also remove an extraneous '.'
* src/chown.c (usage): Remove extraneous '.'
* src/cp.c (usage): Likewise.
* src/mktemp.c (usage): Likewise.
* src/pr.c (usage): Likewise.
* src/stat.c (usage): Likewise.
* src/uniq.c (usage): Likewise.
Prompted by the continuous integration build failure at:
http://hydra.nixos.org/build/277485
* tests/misc/timeout: Set all expected timeouts to 1s and all
unexpected timeouts to 10s. In this way, tests normally proceed
quickly but may delay up to 10s before reporting failures.
* tests/ls/infloop: Likewise.
* tests/tail-2/pid: Likewise.
* tests/tail-2/pipe-f: Likewise.
* tests/tail-2/wait: Likewise.
* tests/dd/skip-seek-past-dev: Likewise.
Previously passing an empty parameter to -t would
raise an error, but now it means to treat each line
as a single field for matching. This matches the
default operation of `sort` which is usually used
in conjunction with join.
* src/join.c (main): Set the field delimiter to '\n' if
an empty parameter is passed to -t.
(usage): Mention the operation of -t ''.
* tests/misc/join: Add 2 new tests, for the existing -t '\0'
and the new -t '' functionality.
* doc/coreutils.texi (join invocation): Mention that
join -t '' always operates on the whole line, while
join -t '\0' usually does.
* NEWS: Mention the change in behavior.
This essentially allows one to use --check-order with headings.
Note join without --check-order will already handle the common case
where headings do match in each file, however using --check-order will fail
often when the header sorts after the first line of data.
Note also that this will join header lines from each file even if
they don't match, with headings from the first file being used.
* NEWS: Mention the new option.
* doc/coreutils.texi (join invocation): Describe the new option.
* src/join.c (usage): Likewise.
(join): Join the header lines unconditionally.
* tests/misc/join: Add 5 new tests.
* src/join.c (join): Refactor the code that checks for misorder
at the tail of the files. The most significant change here is
that freeline() is called thus silencing a valgrind warning about
a definite but inconsequential memory leak.
(freeline): Make more general by doing nothing when passed NULL,
and setting freed pointers to NULL.
* tests/check.mk (TESTS_ENVIRONMENT): Use the generated CONFIG_INCLUDE
variable. Note $(abs_builddir)/$(CONFIG_HEADER) also currently works,
but $(CONFIG_HEADER) is deprecated and may not be generated in future.
$(CONFIG_INCLUDE) was made available by gnulib in commit, 22970f8a,
"syntax-check: detect incorrect boolean macro values in config.h"
* src/ls.c (main): With --color, avoid emitting the final color-
resetting escape sequence when it would be a no-op.
* tests/ls/color-clear-to-eol: Adjust expected output accordingly.
* tests/ls/color-dtype-dir: Likewise.
* tests/ls/multihardlink: Likewise.
* tests/ls/stat-free-symlinks: Likewise.
* tests/misc/ls-misc: Likewise.
* NEWS (Changes in behavior): Mention it.
C de-Avillez rebased and adapted four of the new sl-dangle*
tests in tests/misc/ls-misc.
Reported by Jim Avera in
http://bugs.launchpad.net/ubuntu/+source/coreutils/+bug/494663
Enabled when coreutils is configured with --with-tty-group.
Based on a patch written by Piotr Gackiewicz. Details at
http://bugzilla.redhat.com/454261
* src/who.c (is_tty_writable): A new function returning true if a TTY
device is writable by the group. Additionally it checks the group to be
the same as TTY_GROUP_NAME when compiled with --with-tty-group.
* m4/jm-macros.m4: Introduce a new configure option --with-tty-group.
* NEWS: Mention the change.
* cfg.mk (sc_x_sc_dist_check): This coreutils-specific syntax-check
rule would fail in a non-srcdir build, since in that case, each name
from $(VC_LIST) starts with "$(srcdir)/". Fix that.
* gnulib: Update to latest, to pull in a required maint.mk change.
* tests/cp-mv-enotsup-xattr: Create a file system from which to copy
the xattrs so that the test is not skipped if the host file system
does not have user_xattr support. Also don't erroneously fail when
built without xattr support.
* tests/cp/acl: Support USE_ACL not being defined.
* tests/mv/acl: Likewise. Also fix typo in skip message.
* tests/cp/preserve-slink-time: Support HAVE_UTIMENSAT being 0.
* tests/touch/no-dereference: Likewise.
* tests/ls/capability: Normalize so 1 is not required to be last char.
* src/pr.c (init_store_cols): Allocate N*sizeof(*VAR) bytes,
not N*sizeof(int*). The latter would mistakenly allocate double
the required space on a system with 8-byte pointers.
* README-release: Push the automated release and post-release
NEWS-updating commits.
Pádraig Brady reported that I'd pushed the tag without also
pushing the followup commit.
* tests/tail-2/inotify-hash-abuse: Use kill rather than wait
to determine if the tail process is still running.
* tests/tail-2/inotify-hash-abuse2: Ditto.
* tests/ls/infloop: OpenBSD4.5's /bin/sh would mistakenly include
"set -x"-output in an application's stderr stream when stderr is
redirected before stdout. This was causing one spurious test failure.
The work-around: redirect stdout first.
Reported by Nelson Beebe.
* tests/misc/sort-version: Don't use <<- and indented here-doc contents.
s/<<-/<</ and unindent the here-document contents. Otherwise,
bash would ignore the indented delimiter and use EOF, thus silently
skipping this test. OpenBSD5.4's shell reported the failure:
$ printf 'cat<<-x\n foo\n x\n'|sh
sh: <stdin>[4]: here document `x' unclosed
[Exit 1]
by contrast, bash warns but still exits successfully:
$ printf 'cat<<-x\n foo\n x\n'|bash && echo you lose
bash: line 3: warning: here-document at line 1 delimited by \
end-of-file (wanted `x')
foo
x
you lose
* tests/check.mk: Prepend /usr/xpg4/bin to the $PATH if present.
Using the more standard utilities allows tests such as misc/printenv,
which uses the -E option to grep, to complete.
Configure is supposed to detect insufficient XATTR support.
However, if a system has the required headers, but no library,
the configure script would mistakenly enable USE_XATTR.
* m4/xattr.m4 (gl_FUNC_XATTR): If the attr_copy_file function
is not found, don't set USE_XATTR.
Nelson Beebe reported a link failure on RHEL 5.3.
Also, do not let the combination of --disable-xattr and
a stray LIB_XATTR environment setting perturb the build.
* NEWS (Build-related): Mention it.
* src/ls.c: Include <sys/capability.h> later, to avoid build
failure with a header from libcap-2.16-1 or earlier.
See http://bugzilla.redhat.com/483548 for details.
Before this change, with too long a file name, the name would
abut the date field on the left and possibly also the "Page N"
field on the right, rather than leaving a one-space separator
in each case. Fixes a regression introduced on Mar 6 2009,
by commit a4053c5291
* src/pr.c (print_header): Ensure that there is at least one
space before and after the file name part of the header line.
* NEWS: Mention it.
* tests/pr/W20l24f-ll: s/xPage/ x Page/.
* THANKS: Update.
Reported by Denis McKeon, in https://savannah.gnu.org/bugs/?28492.
* configure.ac: Require autoconf-2.62 and automake-1.11.1 or newer.
* bootstrap.conf (buildreq): Require automake-1.11.1 or newer,
to ensure people use a version with the fix for CVE-2009-4029.
Note that the coreutils-8.2 tarball included a fixed Makefile.in.
Require autoconf-2.62, per automake.
* src/ls.c (print_color_indicator): When using 'LINK target' in
dircolors, treat broken symlink as C_ORPHAN.
* tests/misc/ls-misc (sl-dangle2, sl-dangle3, sl-dangle4)
(sl-dangle5): Test for it, and add more coverage.
* NEWS: Document it.
* THANKS: Update.
Reported by Chris Jones.
* src/tail.c (usage): Reword tail -F description, so that it no
longer mentions details specific to the non-inotify implementation.
Also, join diagnostic strings (while staying under the 509-byte limit)
to ease formatting of translations. The latter was prompted by
a report from Stéphane Raimbault.
* doc/coreutils.texi (tail invocation): Update description here, too.
This fixes a bug whereby tail -F would fail to track changes
to a file that was a target of a rename, and when the source of
the rename was another tailed file.
* src/tail.c (tail_forever_inotify): Ensure the wd is not already
present in the hash table before trying to add it. When a new watch
descriptor is added to the `wd_to_name' hash table, check that it is
not already present. If it is present then remove the previous element.
* src/tail.c (tail_forever_inotify): Avoid modifying fdspec->wd while
it is in the wd_to_name hash table. Once it is removed, it can be
added using the new `wd' as key for the hash table. This fixes the
abort-inducing bug reported by Rob Wortman in
http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/19372
* src/tail.c (tail_forever_inotify): Do not use f[i] in a context
where i may be larger than the largest valid index. In the final
"if" clause in which we'd remove an inotify watch, we might have
used f[n_files]. Use fspec instead, since it is guaranteed to
be defined.
* src/tail.c (fremote): Add a comment.
Move definition "up" to precede first use, so we can
remove its prototype and the #if..#endif around each use.
(any_remote_file): Rename from any_remote_files.
* src/tail.c (struct File_spec): Add a flag to record if file is remote.
(recheck): If we're using inotify then check if the file has gone remote
and if so, drop it with a warning.
(any_remote_files): A new function to check for any open remote files.
(tailable_stdin): A new function to refactor the check for whether
a tailable file was specified through stdin.
(fremote): A new function to check if a file descriptor
refers to a remote file.
(tail_forever_inotify): Add some comments.
(tail_file): Record if a file is remote when initially opened.
(main): Disable inotify if any remote files specified.
Also document the caveat about remounted files not
being noticed by inotify.
* NEWS: Mention the fix.
* src/wc.c (main): Set stdout to line buffered mode
to ensure parallel running instances don't intersperse
their output. This adds 6.5% to the run time in the worst case
of many zero length files, but has neglible impact for
standard sized files.
* tests/misc/wc-parallel: New test for atomic output.
* tests/Makefile.am: Reference it.
* NEWS: Mention the fix
This is similar to commit 710fe413, 20-10-2009,
"md5sum, sha*sum, sum: line-buffer the printed checksums"
* src/stat.c (human_fstype): Add the following FS types:
fuseblk, rpc_pipefs. Also fix a typo of minux3 to minix3,
and mention the fs-magic-compare make target to help update the list.
* NEWS: Mention the fix.
* dist-check.mk (built_programs): Use $(bin_PROGRAMS), not $(PROGRAMS).
Otherwise, my-instcheck would fail due to non-installation of e.g.,
the noinst_PROGRAMS, setuidgid and getlimits.
(taint-distcheck): Correct the grep command that checks for libtool
traces in configure.
Regression introduced in coreutils 8.1 due to a bug in the Linux
kernel implementation of utimensat with mtime of UTIME_OMIT.
* gnulib: Update to latest, to pick up utimensat fix.
* NEWS: Mention the change.
* THANKS: Update.
Reported by John Stanley.
* bootstrap.conf (gnulib_modules): Remove the strverscmp module
which is not used since commit e505736f, on 03-10-2008,
"ls and sort: use filevercmp instead of strverscmp"
* doc/coreutils.texi (sort invocation): Reference the additional
info about filevercmp rather than the unused strverscmp.
(Details about version sort): Add some examples that are not
handled well by fileversmp.
* src/ls.c: Change a comment referencing the now unused strverscmp.
* src/remove.c (rm_fts): Fix incorrect comparison of
device and inode numbers.
* tests/rm/one-file-system2: Add a separate test so
that it can be run as a normal user (It doesn't need to mount).
* tests/Makefile.am: Reference it.
* NEWS: Mention the fix.
Reported by Jan Larres.
* dist-check.mk (null_AM_MAKEFLAGS): Remove LIBTOOL. Adding it was
erroneous, since it is required when building from a distribution
tarball of a libtool-using project. Reported by Ralf Wildenhues.
(my-distcheck): Reorganize to use a subshell and set -e, so that
failures propagate "out". Without this change, setting LIBTOOL=false
would cause a failure that would then be ignored, probably due to a
problem in $(install-transform-check).
* dist-check.mk (built_programs): More generic, but still assumes src/.
Don't set GZIP in environment when untarring.
(my-distcheck): Use $(DIST_ARCHIVES), rather than assuming that
there is always a .tar.gz file.
* dist-check.mk (null_AM_MAKEFLAGS): Define here, not in maint.mk.
(built_programs): Likewise.
(my-distcheck): Move comments to...
(coreutils-path-check): ...the code they refer to.
Remove obsolete comments.
(null_AM_MAKEFLAGS): Add gperf, even though it's not used here.
* gnulib: Update to latest, for fixed maint.mk.
* gl/lib/tempname.c.diff: Adjust patch to apply to gnulib, now that
most TABs in indentation have been converted to spaces by running
this command: f=tempname.c.diff; patch-xform $f > k && mv k $f
Copyright (C) 2003, 2005-2007, 2009 Free Software Foundation, Inc.
Copyright (C) 2003, 2005-2007, 2009-2011 Free Software Foundation, Inc.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
