* src/local.mk [check-duplicate-no-install]: Depend on the
single-binary tr, or the system tr, as the edge case
where these are not available only result in the sanity
check being effectively ignored.
Switching to non-recursive makefiles broke the 'install-html' target:
The gettext plumbing requires an 'install-html' target in po/Makefile.
This was fixed in gettext v0.19.8.1-41-ge5a008a, but packages using
older gettext need to manually patch po/Makefile.in.in.
Reported (for 'sed') and suggested fix by Eric Blake in
https://bugs.gnu.org/25690 .
* bootstrap.conf (bootstrap_epilogue): Add 'install-{html,pdf,dvi,ps}'
targets to po/Makefile.in.in (if needed).
* bootstrap.conf: s/4.13/6.1/ as versions previous to that
generated invalid html with interspersed <span> tags that
were visible to the user. Version 6.1 is available for a
year now, and is available in most distros.
makeinfo issues the following:
doc/coreutils.texi:6568: warning: @sc argument all uppercase,\
thus no effect.
* doc/coreutils.texi (join invocation): Remove the @sc macro around
the all uppercase "GNU".
* src/timeout.c (install_sigchld): A new function to
install the SIGCHLD handler using sigaction() rather
than signal(), because with the latter on solaris
the signal handler is reset to default and thus
sigsuspend() only returns for the first finished child.
Reported by Assaf Gordon.
* src/libstdbuf.c: undef malloc so as libstdbuf is
not linked with gnulib, and anyway the replacement is
never needed since we never malloc(0).
Reported by Assaf Gordon.
* man/md5sum.x: Give a more direct warning againt the use
of this hash algorithm for security purposes.
* man/sha1sum.x: Likewise.
Suggested by Jim Meyering.
* src/expand-common (parse-tab-stops): Exit earlier upon overflow
so another warning isn't issued (on 32 bit) in add_tab_stop().
Flagged in https://hydra.nixos.org/build/49499970
* doc/coreutils.texi (sha1sum invocation): Given that a SHA-1
preimage attack has occurred as documented at http://shattered.io/,
document sha1sum as having the same limitations as md5sum.
(md5sum): Parameterize the warning for use in both cases.
* man/md5sum.x: Mention b2sum(1) as a more secure alternative.
* man/sha1sum.x: Give the same warning as done for md5sum(1).
* doc/coreutils.texi (expand invocation): Document the feature.
(unexpand invocation): Likewise.
* src/expand-common.c (extend_size): A new global to use
when the last tab stop is prefixed by '/'.
(set_extend_size): A new function to validate and set
the new extend_size global.
(parse_tab_stops): Call set_extend_size() for '/' prefixes.
(finalize_tab_stops): Ensure a single specified '/' is
treated like a standard tabsize, but also ensure that
when '/' is specified with a single other entry that
we process as a list rather than a tab size.
(get_next_tab_stop): Use the tab size if set,
for items after the user specified tab position list.
* tests/misc/expand.pl: Add test cases
* NEWS: Mention the new feature.
Fixes http://bugs.gnu.org/25540
This comes from the latest gnulib.
Also handling of OMP_NUM_THREADS has been adjusted
to support comma separated values indicating a nesting level,
in which case the first value is taken. Also OMP_NUM_THREADS=0
is now ignored instead of being treated as 1, to match
the behavior of libgomp.
* NEWS: Mention the OMP_THREAD_LIMIT improvement,
and OMP_NUM_THREADS now handling nested values.
* doc/coreutils.texi (nproc invocation): Describe OMP_THREAD_LIMIT
as a way to set the max value, with OMP_THREAD_LIMIT setting the min.
* tests/misc/nproc-override.sh: A new test to exercise the
updated gnulib code with all combinations of these OMP variables.
* tests/local.mk: Reference the new test.
* src/copy.c (set_process_security_ctx, set_file_security_ctx):
Export for use in cp.c.
* src/copy.h: Likewise.
* src/cp.c (make_dir_parents_private): Call the exported functions
to set the security context for new and updated directories.
* tests/cp/cp-a-selinux.sh: Add a test case.
Fixes http://bugs.gnu.org/25378
* bootstrap.conf, src/base64.c, src/cat.c, src/cksum.c:
* src/head.c, src/md5sum.c, src/od.c, src/split.c, src/sum.c:
* src/tac.c, src/tail.c, src/tee.c, src/tr.c, src/wc.c:
Adjust to renaming of the xsetmode module to xbinary-io,
and of the xsetmode function to xset_binary_mode.
This fixes a bug noted by Eric Blake. Code was using xfreopen to
change files to binary mode, but this fails for stdout when in
append mode. Such code should use xsetmode instead. This affects
only the port on platforms like MS-Windows which distiguish text
from binary I/O.
* bootstrap.conf (gnulib_modules):
Remove xfreopen and add xsetmode. Sort.
* src/base64.c (main):
* src/cat.c (main):
* src/cksum.c (cksum):
* src/head.c (head_file, main):
* src/md5sum.c (digest_file):
* src/od.c (open_next_file):
* src/split.c (main):
* src/sum.c (bsd_sum_file, sysv_sum_file):
* src/tac.c (tac_file, main):
* src/tail.c (tail_file):
* src/tee.c (tee_files):
* src/tr.c (main):
* src/wc.c (wc_file): Use xsetmode, not xfreopen.
* src/force-link.h: Don't include headers already included by system.h
* src/force-link.c: Likewise. Also include system.h and
explicitly mark extern functions as such.
If the file B already exists, commands like 'ln -f A B' and
'cp -fl A B' no longer remove B before creating the new link.
Instead, they arrange for the new link to replace B atomically.
This should fix a race condition reported by Mike Crowe (Bug#25680).
* NEWS, doc/coreutils.texi (cp invocation, ln invocation):
Document this.
* bootstrap.conf (gnulib_modules): Add symlinkat.
* src/copy.c, src/ln.c: Include force-link.h.
* src/copy.c (same_file_ok): It's also OK to remove a destination
symlink when creating symbolic links, or when the source and
destination are on the same file system and when creating hard links.
* src/copy.c (create_hard_link, copy_internal):
* src/ln.c (do_link):
Rewrite using force_linkat and force_symlinkat, to close a window
where the destination temporarily does not exist.
* src/cp.c (main): Do not set x.unlink_dest_before_opening
merely because we are in link-creation mode.
* src/force-link.c, src/force-link.h: New files.
* src/local.mk (copy_sources, src_ln_SOURCES): Add them.
* tests/cp/same-file.sh: Adjust test case to match fixed behavior.
The race is unlikely, as timeout(1) needs to receive a signal
in the few operations between waitpid() returning and exit().
Also the system needs to have reallocated the just released pid
in this time window.
Previously we never disabled the signal handler that sent
the termination signal to the "child" pid. However once waitpid()
has reaped the child, the system is free to allocate that pid,
so we must ensure we don't process any further signals.
* build-aux/gen-lists-of-programs.sh: Build timeout(1) optionally...
* configure.ac: ...predicated on sigsuspend() being available.
* src/timeout.c (block_cleanup): A new function to ensure the
cleanup() handler is disabled after waitpid has returned.
(main): Use sigsuspend() to wait with cleanup() enabled but
disabled once it returns, and thus disabled for the waitpid() call.
(monitored_pid): Change to the more accurate pid_t.
* NEWS: Mention the fix.
Fixes http://bugs.gnu.org/25624
* doc/coreutils.texi (realpath invocation): Mention that realpath
is the preferred command for canonicalization.
(readlink invocation): Likewise.
* man/readlink.x: Likewise.
* src/tail.c (check_fspec): Only enable printing of the file header
if we've actually read some data and this is a new file. Also
move printing of the file header to...
(dump_remainder): ...here, to allow printing only when data read.
* tests/tail-2/overlay-headers.sh: A new test for suspension
and resumption of tail.
* tests/local.mk: Reference the new test.
* NEWS: Mention the fix.
Fixes http://bugs.gnu.org/23539
* tests/tail-2/retry.sh: The replacement of the "missing" directory
is not atomic, and therefore tail(1) can take a different path,
especially if there is a delay between the rmdir(2) and creat(2).
This is noticeable for example with `make coverage` because in
that case the coverage files written by rmdir(1) on exit,
induce a significant delay thus triggering the issue.
* src/tail.c (recheck): Set f->ignore before we
use it to show the appropriate error.
* tests/tail-2/retry.sh: Ensure the "giving up" message
is not presented.
Remove old log files that have corresponding entries
in the source code repository.
This saves about 2.5MB uncompressed, 0.5M compressed.
* Makefile.am (gen-ChangeLog): Adjust to taking all
logs since a particular version (8.15 in this case).
Also mention in the truncated log where to get older entries.
(changelog_etc): Remove the no longer distributed files.
* build-aux/git-log-fix: Remove now unused entries.
* ChangeLog-200[5-8]: Delete.
* doc/ChangeLog-2007: Likewise.
* po/ChangeLog-2007: Likewise.
* old/*: Likewise.
* src/copy.c (punch_hole): Work around an empty definition
of HAVE_FALLOCATE which leads to a build error of:
"error: #if with no expression"
That was triggered by the inclusion of <linux/fs.h> in
commit v8.25-68-g89e1fef with kernel-headers-2.6.18.
Reported by Nelson H. F. Beebe
Problem reported by Paul Wise for Debian, in:
https://bugs.debian.org/851934
This is fallout from the fix for GNU Bug#23035.
* src/date.c (batch_convert): New args TZ and TZSTRING.
All uses changed.
(batch_convert, main): Adjust to parse_datetime2 API change.
(main): Allocate time zone object.
* tests/misc/date-debug.sh: Fix incorrect test case,
caught by the fix.
* tests/misc/date.pl: Test the fix.
* NEWS: Document this behavior, which comes with recent Gnulib.
* doc/coreutils.texi (Formatting file timestamps, du invocation)
(Time conversion specifiers, Setting the time, Options for date):
Mention when -00 is output for numeric time zones.
Be more careful about Internet RFC numbers, ISO 8601, etc.
A lot of this is converting http: to https:.
Also, gmane went away, so remove URLs that no longer work and
are not easy to figure out what they were.
Some of this stuff is so old that it no longer matters anyway.
The above new section looked a bit odd as the only general documentation
in between the utility chapters.
* doc/coreutils.texi (File timestamps): Move to a separate chapter.
Commit 4f650aad was incomplete; it changed NEWS but not the hash,
and introduced a grammar error.
* cfg.mk (old_NEWS_hash): Update via 'make update-NEWS-hash'.
* doc/coreutils.texi (File timestamps): Fix doubled word.
Prompted by a bug report from Scott Deerwester (Bug#25407).
* doc/coreutils.texi (File timestamps): New section.
Revamp other sections to use this new section, and
use more-consistent terminology.
* src/stty.c (apply_settings): A new function refactored
from main() that is used to both check and apply options.
(main): Call apply_settings before we open the device,
so all validation is done before interacting with a device.
* NEWS: Mention the improvement.
* tests/misc/stty.sh: Add a test case.
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
* tests/chgrp/basic.sh: On some NFS setups a user is
not allowed to set a group on a file even if a member
of that group. Therefore skip this test on remote file systems.
* tests/chgrp/default-no-deref.sh: Likewise.
* tests/chgrp/no-x.sh: Likewise.
* tests/chgrp/posix-H.sh: Likewise.
* tests/chgrp/recurse.sh: Likewise.
* tests/tail-2/inotify-rotate-resources.sh: Change to skipping
on remote file systems in the standard way.
* src/wc.c (wc): Avoid reading the end of the file
when the size is not a multiple of PAGE_SIZE,
as the special case handling for files in /proc and /sys
is only required when st_size is 0 or a multiple of PAGE_SIZE.
* tests/misc/wc-proc.sh: Add a test case.
While st_size would have been incorrect for subsequent
files since v7.1, it was only used since v8.24.
* tests/misc/wc-files0.sh: s/7.1/8.24/
* NEWS: Likewise.
Reported by Bernhard Voelker
* src/wc.c (main): Reset fstatus[0].failed between files when reusing
the fstatus[0] entry in --files-from0 mode. This ensures a stat() is
done for each file, avoiding incorrect counts and redundant reading.
* NEWS: Mention the bug fix.
* tests/misc/wc-files0.sh: Add a test case.
Fixes http://bugs.gnu.org/23073
When running "make check" on a Linux system running SELinux with a
non-MLS policy, tests/mkdir/restorecon.sh test fails with:
chcon: invalid context: root:object_r:tmp_t:s0: Invalid argument
Indeed in such a configuration, contexts cannot have ":s0" suffix.
* init.cfg (get_selinux_type): Refactor this function to here
from various tests. Update to work with a non-MLS policy.
(mls_enabled_): A new function to detect if MLS is enabled.
(skip_if_mcstransd_is_running_): Update to not skip when
MLS is not enabled.
* tests/mkdir/restorecon.sh: Use a valid non-MLS context when needed.
* tests/install/install-Z-selinux.sh: Likewise.
* tests/cp/cp-a-selinux.sh: Likewise.
* tests/misc/selinux.sh: Likewise.
* tests/misc/chcon.sh: Skip if non-MLS as --range used throughout.
Fixes http://bugs.gnu.org/22631
* tests/misc/ls-time.sh: Skip the test rather than ERROR
when `touch -m -d ...` fails (Hurd).
* tests/tail-2/follow-stdin.sh: Avoid false FAILs by ignoring
the variances in sterror output.
* tests/rm/rm-readdir-fail.sh: Likewise. Also avoid ERRORs
on systems that don't define _D_EXACT_NAMELEN.
* src/head.c (elide_tail_bytes_file): Ensure we don't use
st_size unless we've previously used seek() to determine
the CURRENT_POS in the seekable file.
This was seen to cause issue on FreeBSD 11 when the pipe
buffer was filled with `yes | head --lines=-0`, in which
case st_size was 64KiB while ST_BLKSIZE() was 4KiB.
Reported by Assaf Gordon.
... in the case where two or more directories nested in each other are
created and each of them defaults to a different SELinux context.
* src/install.c (make_ancestor): When calling defaultcon(), give it the
same path that is given to mkdir(). The other path is not always valid
wrt. current working directory.
* src/mkdir.c (make_ancestor): Likewise.
* NEWS: Mention the bug fix.
Reported at https://bugzilla.redhat.com/1398913
This was detected with ASAN, but can also be seen without ASAN with:
$ tac - - <&-
tac: standard input: read error: Bad file descriptor
*** Error in `tac': malloc(): memory corruption: 0x...
* src/tac.c (copy_to_temp): Don't close our output stream on
(possibly transient) output error, or on input error.
(temp_stream): clearerr() on the stream about to be reused,
to ensure future stream use is not impacted by transient errors.
* tests/misc/tac-2-nonseekable.sh: Add a test case.
* NEWS: Mention the bug fix.
Fixes http://bugs.gnu.org/25041
Reproduced under UBSAN with `tail -f <&-` giving:
tail.c:2220:18: runtime error: load of value 190,
which is not a valid value for type ‘_Bool'
* src/tail.c (tail_file): Ensure f->ignore is initialized
in all cases where we can't tail the specified file.
* tests/tail-2/follow-stdin.sh: Add a test case which
checks stderr has no UBSAN warnings.
Fixes http://bugs.gnu.org/25041
* NEWS: Mention in improvements about the workaround for
the glibc issue with closed stdin, and the new supported file systems.
* tests/misc/b2sum.sh: Spelling fix.
* src/longlong.h: Sync from gmp repo incorporating:
Protect umul_ppmm with do ... while (0)
Replace obsolete ARC asm 'J' constraints with 'Cal'
Provide umul_ppmm for riscv64
glibc has changed the public define
from _STRING_ARCH_unaligned to _STRING_INLINE_unaligned as per
https://sourceware.org/bugzilla/show_bug.cgi?id=19462
* gl/lib/rand-isaac.c: Cater for both defines.
* gl/lib/randread.c: Likewise.
* src/system.h: Update commented out code.
* tests/ls/quote-align.sh: Remove "total" line
which can vary per file system depending on allocation.
Reported by Assaf Gordon on OpenSolaris (5.11/5.10).
This was detected with:
echo a > a; pr "-S$(printf "\t\t\t")" a -m a > /dev/null
Resulting in ASAN triggering:
====================================================
ERROR: AddressSanitizer: global-buffer-overflow
READ of size 1 at 0x00000041b622 thread T0
#0 0x40506a in print_sep_string ../src/pr.c:2241
#1 0x407ec4 in read_line ../src/pr.c:2493
#2 0x40985c in print_page ../src/pr.c:1802
#3 0x40985c in print_files ../src/pr.c:1618
#4 0x4036e0 in main ../src/pr.c:1136
* src/pr.c (init_parameters): Ensure we only override the
specified separator when it's a single tab, thus matching
the calculated separator length.
* tests/pr/pr-tests.pl: Add a test case.
* NEWS: Mention the fix.
* src/ptx.c (fix_output_parameters): Ensure line_width doesn't
go negative, which can happen when the --width is less
than the --gap-size.
* tests/misc/ptx-overrun.sh: Add a test case that triggers
with ASAN. (Note the longer filename is needed to trigger).
Fixes http://bugs.gnu.org/25011
* tests/rm/rm-readdir-fail.sh: ASAN correctly indicated
that fts was writing to freed memory. This was because
we reused a single dirent in our readdir() test wrapper.
Since fts was deallocating those dirents, we now get
a new dirent for each call to our readdir wrapper.
ASAN reported this error for: split -n2/3 /dev/null
ERROR: AddressSanitizer: negative-size-param: (size=-1)
#0 0x7f0d4c36951d in __asan_memmove (/lib64/libasan.so.2+0x8d51d)
#1 0x404e06 in memmove /usr/include/bits/string3.h:59
#2 0x404e06 in bytes_chunk_extract src/split.c:988
#3 0x404e06 in main src/split.c:1626
Specifically there would be invalid memory access
and subsequent processing if the chunk to be extracted
was beyond the initial amount read from file (which is
currently capped at 128KiB). This issue is not in a
released version, only being introduced in commit v8.25-4-g62e7af0
* src/split.c (bytes_chunk_extract): The initial_read != SIZE_MAX
should have been combined with && rather than ||, but also this
condition is always true in this function so remove entirely.
* tests/split/b-chunk.sh: Add a test case.
Fixes http://bugs.gnu.org/25003
In recent commit v8.25-93-g7fc7206 we used the f->remote flag
which wasn't set in all cases. This was detected with
ASAN giving this error when reading f->remote;
runtime error: load of value 190,
which is not a valid value for type '_Bool'
* src/tail.c (fremote): Query the system even without inotify.
(recheck): Always set f->fremote for valid files.
Installing with -j2 or greater could result in a failure like
/bin/sh: line 29: /P/bin/install: Permission denied
when /P/bin/install specifies your PATH-selected install program.
This would arise because we're using "install" to install all
man/*.1 files, and that command would run concurrently with the one
that installs "/P/bin/install" itself. We would run this command:
"src/ginstall src/ginstall /P/bin/install", and it would result
in intervals during which the destination file does not exist,
is empty or incomplete and not executable. We addressed this
problem long ago for installation of actual binaries by telling the
installation rules to use our just-built bin/ginstall (only when not
cross-compiling) rather than the PATH-resolved "install" program.
This change is to do the same for those .1 files.
* src/local.mk (INSTALL): Override automake's default of something
like "INSTALL = /P/bin/install -c".
(INSTALL_PROGRAM): Now that we set INSTALL, there is no longer any
need to set this derived variable. Its default definition,
"INSTALL_PROGRAM = ${INSTALL}" does what we require.
Improved by Eric Blake.
* src/comm.c (total_option): Add bool variable for the new option.
(TOTAL_OPTION): Add enum value.
(long_options): Add array element for the new option.
(usage): Document the new option here.
(compare_files): Count the lines in total[3], and output the summary at
the end.
(main): Accept the new option.
* doc/coreutils.texi (comm invocation): Document it.
* tests/misc/comm.pl: Test it. While at it, improve the test data
to have 1 unique line in the first file, 2 unique lines in the second
file, and 3 common lines.
* NEWS (New Features): Mention the new option.
Fixes http://bugs.gnu.org/24929
* tests/misc/printf-quote.sh: FreeBSD 11 was seen to treat
\u0378 as a printable character. Therefore change to
using the \u0081 C1 control character. We use the UTF-8
representation because our printf implementation explicitly
disallows \u0081 as input.
Reported by Assaf Gordon
On BSD /bin/sh it was seen that unexported env vars passed to
returns_() would not be propagated to the wrapped command.
* cfg.mk (sc_prohibit_env_returns): Add a syntax check to disallow.
* tests/misc/csplit-io-err.sh: Rearrange to export vars in a subshell.
* tests/rm/rm-readdir-fail.sh: Likewise.
* tests/misc/nohup.sh: Export and unset vars around returns_.
* tests/misc/printenv.sh: Likewise.
Reported by Assaf Gordon
This provides better alignment when some names are quoted,
which also provides better indication that quotes are not
part of the name.
* src/ls.c (align_variable_outer_quotes): A new variable
set when ls is aligning columns (not using -m, non-zero -w),
and has a variable quoting style (shell, shell-escape, c-maybe).
(quote_name_buf): Writes to buffer rather than FILE,
taking care to avoid data copying if possible. Refactored from...
(quote_name): ...here. This now manages the buffer passed
to quote_name_buf() and outputs the padding, colors and name
in the appropriate order, while managing the --dired offsets.
(get_color_indicator): A new function to return the color sequence,
refactored from...
(print_color_indicator): ...here. This now simply outputs.
(print_dir): Refactor common parts to quote_name().
(clear_files): Reset the flag indicating at least one
file is quoted in the current directory.
(needs_quoting): A new function to indicate at the scan stage
whether a name needs quoting. Called from...
(gobble_file): ...here, until we find the first quoted file.
(print_name_with_quoting): Mostly refactored to quote_name().
* tests/ls/quote-align.sh: A new test for various output formats.
* tests/local.mk: Reference the new test.
* NEWS: Mention the improvement.
Note we don't support the --algorithm option of the
b2sum command in the external BLAKE2 project, as that
was deemed too confusing for users.
"BLAKE2b" was chosen as the default algorithm to use,
which is single threaded but performs well on 64 bit.
* src/blake2: CC0 source copied from external project.
* cfg.mk[VC_LIST_ALWAYS_EXCLUDE_REGEX]: Exclude blake2/
from syntax checks, make update-copyright, etc.
* src/local.mk: Reference the sources for b2sum,
and set the compilation flags.
* doc/coreutils.texi (b2sum invocation): Reference the
md5sum invocation node, and add descriptions of -l.
* tests/misc/b2sum.sh: Add new test.
* tests/local.mk: Reference new test.
* AUTHORS: Add new binary.
* README: Likewise.
* build-aux/gen-lists-of-programs.sh: Likewise.
* man/.gitignore: Likewise.
* scripts/git-hooks/commit-msg: Likewise.
* man/b2sum.x: New man page template.
* man/local.mk: Reference new template.
* src/.gitignore: Ignore new binaries.
* src/blake2/.gitignore: Ignore new build atrifacts.
* src/md5sum.c (usage): Describe the new -l option.
* NEWS: Mention the new program.
* Makefile.am (my-distcheck): Remove all .deps directories before the
recursive diff that searches for left-behind files. Otherwise, with
automake master (some time after v1.15), "make distcheck" would fail
due to those directories being left behind after "make distclean".
* src/copy.h (cp_options): Add a new flag for install(1).
* src/copy.c (copy_internal): For cp, also output remediation
advice which also indicates why directories aren't copied by default.
The message is unchanged for install(1).
* src/cp.c (cp_option_init): Init install_mode to false.
* src/mv.c (cp_option_init): Likewise.
* src/install.c (cp_option_init): Init install_mode to true.
* tests/install/basic-1.sh: Add a test case.
* tests/cp/link-deref.sh: Adjust test case.
Fixes http://bugs.gnu.org/24958
* tests/misc/sort-h-thousands-sep.sh: The sv_SE locale
may be available while sv_SE.utf8 is not. Therefore
test and use the same locale, now definfed in a variable.
Reported by Jim Meyering.
The previous "returns_"-using change failed to convert many
uses of "$?". Convert all but two of the remaining ones.
* tests/ls/stat-vs-dirent.sh: Likewise.
* tests/misc/head-write-error.sh: Likewise.
* tests/misc/nice.sh: Likewise.
* tests/misc/nohup.sh: Likewise.
* tests/misc/stdbuf.sh: Likewise.
* tests/misc/sync.sh: Likewise.
* tests/tail-2/pid.sh: Likewise.
* tests/tail-2/wait.sh: Likewise.
Thanks to Bernhard Volker for spotting this.
This reduces a standard coreutils install size by about 160K.
* src/cat.c: Change to proper_name() which removes about 18K text.
* src/cp.c: Likewise.
* src/df.c: Likewise.
* src/du.c: Likewise.
* src/getlimits.c: Likewise.
* src/realpath.c: Likewise.
* src/split.c: Likewise.
* src/stdbuf.c: Likewise.
* src/timeout.c: Likewise.
* src/truncate.c: Likewise.
* src/local.mk: Remove -llibiconv from the above programs.
* cfg.mk (sc_check-AUTHORS): Adjust to use factor(1).
* AUTHORS: Adjust to use ASCII to satisfy sc_check-AUTHORS.
The previous commit v8.25-96-g22063c8 lets 'make syntax-check' fail,
because the above check falsely detects a case where 'returns_' is
already in use.
* cfg.mk (sc_prohibit_and_fail_1): Add 'returns_' to exclude list.
* src/tail.c (tail_file): On failure to open a file,
set ignore=true when --retry is not specified.
* tests/tail-2/assert-2.sh: Adjust to the new behavior.
* tests/tail-2/retry.sh: Add a test case. Also change
from `tail ... && fail=1` to the more robust `returns_ 1 ...`
construct which detects segfaults etc.
* NEWS: Document the fix.
* src/tail.c (tail_forever): The BLOCKING optimization is only
enabled for non regular files (which can't be truncated), so ensure
we don't enable that unless we've a valid file descriptor.
* tests/tail-2/retry.sh: Add a test case.
* NEWS: Mention the bug fix.
* src/tail.c (tail_forever): Only read up to st_size on network
file systems to avoid the issue with a stale attribute cache
returning a smaller st_size than we have already read().
The was seen with glusterfs at least and caused the complete
file to be repeatedly output due to assuming the file was
truncated in this case.
* NEWS: Mention the fix.
* src/tail.c (ignore_pipe_or_fifo): Mark the descriptor as -1
for pipes so that any_live_files() detects correctly that
the entry is no longer live.
* tests/tail-2/pipe-f.sh: Add a test case.
* NEWS: Mention the fix.
Fixes http://bugs.gnu.org/24903 which was detected
using Symbolic Execution techniques developed in
the course of the SYMBIOSYS research project at
COMSYS, RWTH Aachen University.
* doc/coreutils.texi(cut invocation): Give a more accurate description
of cut field handling limitations.
* tests/misc/stat-fmt.sh: Fix the test header to be more general.
* tests/tail-2/retry.sh: Spellings.
* src/system.h (emit_backup_suffix_note): A new function to
output the backup suffix info. The strings are unchanged,
so translations are not impacted.
* src/cp.c (usage): Use the new function.
* src/ln.c (usage): Likewise.
* src/mv.c (usage): Likewise.
* src/install.c (usage): Likewise.
* src/cp.c (main): Avoid the getenv("SIMPLE_BACKUP_SUFFIX") call,
which is now done if needed in the gnulib backupfile module.
Also avoid the redundant strdup, as we don't modify this suffix.
* src/install.c (main): Likewise.
* src/ln.c (main): Likewise.
* src/mv.c (main): Likewise.
Fixes http://bugs.gnu.org/23153
Incorporating these coreutils specific changes:
backupfile: initialize default suffix within the implementation
strftime,strptime: support %q to represent the quarter
Also sync these copies with gnulib:
* bootstrap: Now supports AIX.
* gnulib: Update to the latest.
* tests/init.sh: dash(1) is now disabled.
* doc/coreutils.texi (stat invocation): Describe the
QUOTING_STYLE values now supported.
* src/stat.c (getenv_quoting_style): A new function called
from main, that sets the default quoting style for quotearg.
(main): Call getenv_quoting_style() when %N specified.
* tests/misc/stat-fmt.sh: Add a test case.
* NEWS: Mention the improvement.
Fixes http://bugs.gnu.org/23422
* src/dd.c (parse_integer): Suggest to use "00x" instead of "0x",
which is significant for the "count", "seek", and "skip" operands.
* tests/dd/misc.sh: Add a test case.
Fixes http://bugs.gnu.org/24874
* NEWS: Mention the fix.
* src/md5sum.c (digest_file): Add a new MISSING parameter to
return whether the file was missing, separately from the digest.
* tests/misc/md5sum.pl: Add a test case.
Fixes http://bugs.gnu.org/24795
* src/expr.c (integer_overflow): Remove an abort-after-die.
* src/paste.c (write_error): Likewise.
* src/sort.c (badfieldspec): Likewise.
* src/tr.c (string2_extend): Likewise. Also remove a few
break statements, each after an abort.
die() has the advantage of being apparent to the compiler
that it doesn't return, which will avoid warnings in some cases,
and possibly generate better code.
* cfg.mk (sc_die_EXIT_FAILURE): A new syntax check rule to
catch any new uses of error (CONSTANT, ...);
* src/die.h (die): New file/function from grep.
Note: we expect this file to migrate to gnulib.
* src/csplit.c: Include die.h.
(check_format_conv_type): Use die in place of error-nonzero;break;
* src/install.c (strip): Likewise.
* src/nl.c (proc_text): Likewise. This also suppresses a new warning
from GCC 7's -Werror=strict-overflow.
* src/tail.c (parse_options): Likewise.
* src/basename.c (main): Adjust "fall through" comment
so that GCC 7's -Wimplicit-fallthrough honors it.
* src/cp.c (main): Add a "fall through" comment.
* src/ls.c (gobble_file): Likewise.
(get_funky_string): Adjust a "fall through" comment so it is
recognized.
* cfg.mk (exclude_file_name_regexp--sc_system_h_headers): Add die.h
to this list of exempt src/*.h files.
* src/pr.c (main): Avoid this warning from GCC 7:
src/pr.c:1119:6: error: assuming signed overflow does not occur when \
simplifying conditional to constant [-Werror=strict-overflow]
if (n_files == 0)
* src/rm.c (main): Ensure the full --no-preserve-root
option is specified, rather than allowing --n etc.
* tests/rm/r-root.sh: Add a test case.
* NEWS: Mention the change in behavior.
Improved by Jim Meyering.
Fixes http://bugs.gnu.org/24604
which was not the case when inotify was not available.
* src/tail.c (any_live_files): Simplify, since the IGNORE
flag is now only set when a file should be ignored indefinitely.
(recheck): Only output the "giving up on name" message
when that's actually the case. Only set the IGNORE flag
when ignoring a file indefinitely.
(tail_file): Likewise.
* tests/tail-2/retry.sh: Add a test case. Also run
all existing test cases with and without inotify.
NEWS: Mention the fix.
THANKS.in: Add the reporter.
Fixes http://bugs.gnu.org/24495 which was detected
using Symbolic Execution techniques developed in
the course of the SYMBIOSYS research project at
COMSYS, RWTH Aachen University.
* src/who.c (idle_string): This function would fail to compile
with -Werror and today's built-from-git gcc due to this warning:
src/who.c: In function 'print_user':
src/who.c:201:36: error: may write format character ':' at offset 4 \
past the end of the destination [-Werror=format-length=]
sprintf (idle_hhmm, "%02d:%02d",
^~~~~
The fix is to use an assertion to inform gcc of the existing
invariant that guarantees the number of hours is less than 24.
The old code mishandled --time-spec='+%%b', as it misinterpreted
the '%b' as being the month abbreviation. Also, it mishandled
the extremely unlikely case of a month abbreviation containing '%'.
The performance part of this patch sped up 'ls' by about 1% on my
little benchmark of 'ls -lR' on the source directory in the
en_US.UTF-8 locale (Fedora 24 x86-64).
* NEWS: Document the bug fix.
* src/ls.c (first_percent_b, abformat_init): New static functions.
(ABFORMAT_SIZE): New constant.
(use_abformat): New static var.
(abmon, required_mon_width): Remove these static vars.
(abmon_init): Now accepts a pointer to abmon, and returns a boolean.
All callers changed. Reject month abbrs containing '%', as these
would mess up strftime. Simplify mbsalign result checking,
since (size_t) -1 exceeds ABFORMAT_SIZE.
(abformat_init, align_nstrftime): Precompute all 24 formats at
startup, rather than computing a format for each time stamp.
(decode_switches): Call abformat_init instead of abmon_init.
(align_nstrftime): Accept recentness bool instead of format.
All callers changed.
* tests/misc/time-style.sh: Test for format with '%%b'.
Given that `ttyname` is already conditionally declared in src/system.h,
other declarations are redundant and problematic for example in ChromeOS
which has a new FORTIFY implementation that, as an artifact of how it's
implemented, causes the compiler to complain if certain standard library
functions are redeclared without special compiler-specific attributes.
* src/pinky.c: Remove declaration (which was unused anyway).
* src/who.c: Remove declaration.
* src/system.h (getlogin, getuid, geteuid, getgrgid, getpwuid, ttyname):
Add the parameter type to the declarations to avoid warnings when these
backup declarations are used.
This doesn't affect the generated code on my system; it's merely
a cleaner way to use the recently-introduced Linux-specific API.
* m4/jm-macros.m4 (gl_CHECK_ALL_HEADERS): Check for linux/fs.h.
* src/copy.c: Include <linux.fs.h> if available.
(FICLONE) [__linux__]: Define if not already defined.
(clone_file): Use FICLONE instead of the older BTRFS_IOC_CLONE,
as this ioctl is no longer btrfs-specific.
* bootstrap.conf (gnulib_modules): Add flexmember.
* src/sort.c: Include flexmember.h.
(struct tempnode): Make the last member flexible.
(create_temp_file): Port to strict C11/C99 rules for
allocation alignment with flexible array members.
* src/factor.c (lbuf_putc): Only buffer more than a line
when not using the tool interactively.
* NEWS: Mention the bug fix.
Fixes http://pad.lv/1620139
Postpone installation of signal handlers until they're needed.
That is right before the first escape sequence is printed.
* src/ls.c (signal_setup): A new function refactored from main()
to set and restore signal handlers.
(main): Move signal handler setup to put_indicator()
so that the default signal handling is untouched as long as possible.
Adjusted condition for restoring signal handlers to reflect the change.
(put_indicator): Install signal handlers if called for the very first
time. It uses the same code that was in main() prior to this commit.
* NEWS: Mention the improvement.
See https://bugzilla.redhat.com/1365933
Fixes http://bugs.gnu.org/24232
* src/tr.c (unquote): Use indices of type "unsigned int", rather
than a mix or "size_t" and "int", presuming that no command-line-
specified string will have length longer than UINT_MAX.
* src/ptx.c (fix_output_parameters): Switch to an unsigned type that
matches the OCCURS.file_index type. This avoids the following error
from gcc-7.0.0 20160829 (experimental):
src/ptx.c:1220:14: error: assuming signed overflow does not occur \
when simplifying conditional to constant [-Werror=strict-overflow]
if (file_index > 0)
* src/cp.c (make_dir_parents_private): Use default permissions for
created directories when --no-preserve=mode is specified.
* tests/cp/cp-parents.sh: Add a test case.
* NEWS: Mention the fix.
Fixes http://bugs.gnu.org/24251
* src/expand.c, src/unexpand.c: Move global variables from here...
* src/expand-common.h, src/expand-common.c: ... to here.
* src/expand.c, src/unexpand.c: (parse_tab_stops, validate_tab_stops,
next_file): Move identical functions to new module.
(add_tab_stop): Move to new module, including additional code from
'unexpand' (keeping max_column_width) which will have no effect in when
used in 'expand'. Refactor common next-column calculation code
into a new function 'get_next_tab_column'.
* src/local.mk: (src_expand_SOURCES, src_unexpand_SOURCES): Add
'expand-common.c'; (noinst_HEADERS): Add 'expand-common.h'.
* po/POTFILES.in: Add 'expand-common.c'.
* tests/misc/expand.pl: Add more tests.
* tests/misc/unexpand.pl: Likewise.
* TODO: Move conclusions to above test after investigation.
Use hash table for seaching in filter_mount_list() and get_dev()
This improves performance for 20K mount entries from:
real 0m1.731s
user 0m0.532s
sys 0m1.188s
to:
real 0m1.066s
user 0m0.028s
sys 0m1.032s
* src/df.c (devlist_table): Define hash table.
(devlist_hash): Add hash function.
(devlist_compare): Add hash comparison function.
(devlist_for_dev): Add lookup function.
(devlist_free): Add cleanup function.
(filter_mount_list): Use the above hash table.
While at it, rename the variable 'devlist' to 'seen_dev' for
better readability.
(me_for_dev): Use the above lookup function.
NEWS: Mention the improvement.
THANKS.in: Remove the committer; add original submitter Josef Cejka.
* src/date.c (usage): This was thought to introduce other ambiguities,
and was inconsistent with the format presented in the touch(1) docs.
* doc/coreutils.texi (date invocation): Likewise.
See http://bugs.gnu.org/24077
* cfg.mk (sc_THANKS_in_sorted): This check would fail on systems
for which "." is not ignored. Add a quick sort-based check for
that error, and skip the check on any broken system.
* src/date.c (usage): Make the characters used to summarize
the input format, match the output +FORMAT characters.
* doc/coreutils.texi (date invocation): Likewise.
Fixes http://bugs.gnu.org/24077
The following is a reproducer for the wrong dependency:
$ ./configure --enable-install-program=arch
$ make
$ rm -f src/arch man/arch.1
$ make man/arch.1
GEN man/arch.1
help2man: can't get `--help' info from man/arch.td/arch
Try `--no-discard-stderr' if option outputs to stderr
Makefile:14378: recipe for target 'man/arch.1' failed
make: *** [man/arch.1] Error 127
* man/local.mk (man/arch.1): Change to depend on src/arch rather than
src/uname: while the arch binary depends on uname.c and uname-arch.c,
its man page depends on the arch binary.
Reported downstream by Rodrigues Goldwyn <rgoldwyn@suse.com> in
https://build.opensuse.org/request/show/415172
* src/sort.c (traverse_raw_number): Accept thousands separator only
if it is immediately followed by a digit.
* tests/misc/sort-h-thousands-sep.sh: Cover the fix for this bug.
Suggested by Pádraig Brady in http://bugs.gnu.org/24015
* src/sort.c (traverse_raw_number): Allow to skip only one occurrence
of thousands_sep to avoid finding the unit in the next column in case
thousands_sep matches as blank and is used as column delimiter.
* tests/misc/sort-h-thousands-sep.sh: Add regression test for this bug.
* tests/local.mk: Reference the test.
* NEWS: Mention the bug fix.
Reported at https://bugzilla.redhat.com/1355780
Fixes http://bugs.gnu.org/24015
* src/sort.c (traverse_raw_number): New function for traversing numbers.
(find_unit_order): Use traverse_raw_number() instead of open-coding it.
(debug_key): Likewise.
* doc/coreutils.texi (install invocation): Update -Z documentation.
* src/install.c (make_ancestor): Set default security context before
calling mkdir() if the -Z option is given.
(process_dir): Call restorecon() on the destination directory if the
-Z option is given.
(usage): Update -Z documentation.
* tests/install/install-Z-selinux.sh: A new test for 'install -Z -D'
and 'install -Z -d' based on tests/mkdir/restorecon.sh.
* tests/local.mk: Reference the test.
* NEWS: Mention the improvement.
Reported at https://bugzilla.redhat.com/1339135
Fixes http://bugs.gnu.org/23868
* tests/rm/rm-readdir-fail.sh: Since we use the returned dirent
from the readdir wrapper it must be the correct type and not
just cast. Therefore setup so that we only have to define a
wrapper for readdir(), which works appropriately on 32 and 64 bit.
Issue reported by Bernhard Voelker, where rm was seen to invoke
rmdir() on invalid file names.
Follow-up to commit c92585b1: improve epipe test script, with
suggestions from Bernhard Voelker and Pádraig Brady.
see: http://lists.gnu.org/archive/html/coreutils/2016-04/msg00067.html .
* tests/misc/seq-epipe.sh: Avoid too many subshells,
stricter validation of errors and output from seq,
skip if trapping SIGPIPE is not supported.
* src/stty.c: Disable setting of "swtch" to ^z as that
conflicts with and disables using ^z as "susp".
* NEWS: Mention the bug fix.
Reported and tested by Rich Burridge at:
http://bugs.gnu.org/23866
* tests/rm/rm-readdir-fail.sh: A new test to simulate readdir()
failing immediately or after returning a few entries, and verifying
that rm does the appropriate thing.
This was initially reported at:
http://bugzilla.opensuse.org/show_bug.cgi?id=984910
where it was mentioned that readdir() may fail
when an NFS server has a poor readdir cookie implementation.
* gnulib: Update to latest.
* NEWS: Specifically mention the fts readdir() fix
and reindent to standard indentation.
* tests/init.sh: Update from gnulib.
* src/yes.c: Include full-write.h.
(main): Use full_write, not write, to simplify handling of partial
writes. Don't bother using stdio to output data; just use
full_write with a buffer as large as needed. Reuse operand
strings if possible, and if the buffer would otherwise be large.
* src/yes.c (main): Loop over the write buffer to
handle the case where write may write less than requested.
* NEWS: Mention the bug fix.
Reported by Paul Eggert.
GCC 7 warned about undefined behavior in this unlikely case.
Problem reported by Jim Meyering in: http://bugs.gnu.org/23825
* src/md5sum.c (main):
* src/paste.c (main):
* src/yes.c (main):
Avoid undefined behavior when argc == INT_MAX.
* doc/coreutils.texi (sort invocation): Mention in the summary
dicussion that --key is used to specify fields. Give a summary
in the --key description, of the most common use case of specifying
a field, and that by default those fields include the blank separators
at the start of each field in the comparisons.
Print warning regardless of locale, avoid warning if key is zero width.
Problem reported by Karl Berry in http://bugs.gnu.org/23665 .
* src/sort.c: (key_warnings): change conditions for 'leading spaces'
warning.
* tests/misc/sort-debug-warn.sh: adjust tests accordingly.
Easier troubleshooting of individual 'sort --debug' messages.
Discussed in http://bugs.gnu.org/23665 .
* tests/misc/sort-debug-warn.sh: add progress number before each sort
invocation.
* src/dircolors.hin: Reduce the list by replacing all specific
mentions of "color" with the pattern "*color*". This will also
cater for other entries like "konsole-256color".
Fixes http://bugs.gnu.org/23542
* stc/stat.c (human_fstype): Add file system ID definition,
and use "smb2" as the name.
* NEWS (Improvements): Mention the change.
Fixes http://bugs.gnu.org/23516
* src/id.c (main): When configured with --enable-gcc-warnings and using
the very latest gcc built from git, building would fail with this:
src/id.c:200:8: error: assuming signed overflow does not occur when \
simplifying conditional to constant [-Werror=strict-overflow]
bool default_format = (just_user + just_group + just_group_list
^~~~~~~~~~~~~~
Rewrite to use bool-appropriate operators.
Ensure I/O errors are detected (and terminate seq), preventing seq
from infloop (or running for long time with a large
range) upon write errors or ignored SIGPIPE. Examples:
seq 1 inf > /dev/full (seq_fast)
seq 1.1 0.1 inf >/dev/full (print_numbers)
* src/seq.c (io_error): A new function to diagnose appropriate
stdio errors and exit the program with failure status.
(seq_fast, print_numbers): Explicitly check for write errors
and terminate the program with diagnostic.
* tests/misc/seq-io-errors.sh: Test error detection with /dev/full.
* tests/misc/seq-epipe.sh: Test error detection with broken pipes.
* tests/local.mk: Add new tests.
* NEWS: Mention the fix.
* src/stat.c (human_fstype): Add file system ID definition,
and use "wslfs" as the name.
* NEWS (Improvements): Mention the change.
Fixes http://bugs.gnu.org/23273
* src/sleep.c (main): Allow ERANGE since we allow "inf" values.
* src/timeout.c (parse_duration): Likewise.
* tests/misc/sleep.sh: New file. Tests for sleep(1).
* tests/misc/timeout-parameters.sh: Add case for newly allowed
$LDBL_MAX. Also use returns_ throughout the file. Also avoid
small timeout values which might give false failures under load.
* cfg.mk (sc_prohibit_expr_unsigned): expr uses intmax_t internally
when GMP is not available, so flag any uses of unsigned limits.
* tests/misc/timeout-parameters.sh: Remove the overflow related
to UINT_MAX as it's handled by the following case. Change the
following case from 99... to $TIME_T_OFLOW to be more expressive
in what it's actually testing. Directly check that commands succeed,
rather than inspecting $? afterwards.
* tests/dd/skip-seek-past-dev.sh: Guard against large device sizes.
* tests/id/setgid.sh: Protect: Guard against large group IDs.
* tests/misc/cut-huge-range.sh: Tweak comment to avoid syntax check.
It's tempting to simplify to just skip the test if expr fails,
but SIZE_MAX > INTMAX_MAX is the common case.
* tests/cp/parent-perm-race.sh: This new race introduced in
commit v8.25-5-g632eda5 is quite hard to hit, but is due to
`ls > fifo` doing write()/close()/exit() once `cp` has
open() the source fifo. Then the subsequent comparison of the
destination file may fail due to the file being missing or empty.
Previously `ls` generated output that was independent of `cp`.
Now we must wait for `cp` to finish before inspecting the
destination file that it wrote.
* src/seq.c (isnan): Define macro.
(scan_arg): Add check if the given argument is NaN, and exit with
a proper error diagnostic in such a case.
(usage): Document it.
* tests/misc/seq.pl: Add tests.
* doc/coreutils.texi (seq invocation): Document the change.
* NEWS (Changes in behavior): Mention the change.
* src/seq.c (main): Exit with an error diagnostic when the given
step value is Zero.
(usage): Document it.
* doc/coreutils.texi (seq invocation): Likewise.
* tests/misc/seq.pl: Add tests.
* NEWS (Changes in behavior): Mention the change.
Reported by Маренков Евгений in:
http://bugs.gnu.org/23110
* src/stat.c (human_fstype): Add file system ID definition.
* NEWS (Improvements): Mention the change, moving the previously added
"prl_fs" change note from "Changes in behavior" to here.
Fixes http://bugs.gnu.org/23283
* src/stat.c (human_fstype): Identify the parallels file system.
Also tag as remote so that tail(1) doesn't use inotify, which
fails to detect changes made outside a VM.
Fixes http://bugs.gnu.org/23143
The problematic code computed a struct tm in one time zone, and
then printed it or converted it to a string in another. To be
portable the same time zone needs to be used for both operations.
On GNU platforms this is not an issue, but incorrect output can be
generated on System V style platforms like AIX where time zone
abbreviations are available only in the 'tzname' global variable.
Problem reported by Assaf Gordon in: http://bugs.gnu.org/23035
* NEWS: Document the bug.
* src/date.c (show_date):
* src/ls.c (long_time_expected_width, print_long_format):
* src/pr.c (init_header):
* src/stat.c (human_time): Use localtime_rz instead of localtime,
so that the time zone information is consistent for both localtime
and time-formatting functions like fprintftime and nstrftime. For
'stat' this change is mostly just a code cleanup but it also
causes stat to also print nanoseconds when printing time stamps
that are out of localtime range, as this is more consistent with
what other programs do. For programs other than 'stat' this fixes
bugs with time zone formats that use %Z.
* src/du.c, src/pr.c (localtz): New static var.
(main): Initialize it.
* src/du.c (show_date): New time zone argument, so that localtime
and fprintftime use the same time zone information. All callers
changed.
* tests/misc/time-style.sh: New file.
* tests/local.mk (all_tests): Add it.
* tests/misc/date.pl: Test alphabetic time zone abbreviations.
* gl/lib/regcomp.c.diff: Remove file, now that gnulib's
regcomp.c compiles regex.c with -Wno-unused-parameter.
* gl/lib/regex_internal.h.diff: Likewise.
* gl/lib/regex_internal.c.diff: This file induced a change to ensure
that the "Idx" type was unsigned and to remove a few "VAR < 0"
comparisons. These days, it is probably fine to stay in sync with
gnulib/glibc's copies
of these files, so remove these patches, too.
* gl/lib/regexec.c.diff: Likewise.
Prompted by a report by Assaf Gordon and a suggestion from Paul Eggert.
* tests/split/filter.sh: Use OFF_T_MAX-1 rather than OFF_T_MAX
as the size of a test file, to avoid false failure on an XFS file
system (or any file system permitting a file of size OFF_T_MAX).
Reported as http://bugs.gnu.org/22931
POSIX recommends avoiding -a and -o, for good reason.
src/test.c (usage): Mention that inherent ambiguities exist with
binary -a and -o.
Problem reported by Martin Gebert in: http://bugs.gnu.org/22909
sort, tail, and uniq now support traditional usage like 'sort +2'
and 'tail +10' on systems conforming to POSIX 1003.1-2008 and later.
* NEWS: Document this.
* doc/coreutils.texi (Standards conformance, tail invocation)
(sort invocation, uniq invocation, touch invocation):
Document new behavior, or behavior's dependence on POSIX 1003.1-2001.
* src/sort.c (struct keyfield.traditional_used):
Rename from obsolete_used, since implementations are now allowed
to support it. All uses changed.
(main): Allow traditional usage if _POSIX2_VERSION is 200809.
* src/tail.c (parse_obsolete_option): Distinguish between
traditional usage (which POSIX 2008 and later allows) and obsolete
(which it still does not).
* src/uniq.c (strict_posix2): New function.
(main): Allow traditional usage if _POSIX2_VERSION is 200809.
* tests/misc/tail.pl: Test for new behavior.
Problem reported by Yanyan Jiang 蒋炎岩 in: http://bugs.gnu.org/22769
* doc/coreutils.texi (sort invocation):
Mention possibility of 'sort -o F F' data loss during crashes.
* src/split.c (lines_rr): Reinstate the conditional
setting of the WROTE boolean, as otherwise split -n r/1 would
consume all input when all --filter commands are stopped.
There was a test in place to check for this, but
it was incorrect as detailed below.
(input_file_size): Immediately disallow --number with
non seekable inputs, as such an invocation is not currently
generally supported and will fail as the data overflows
the internal buffer.
* tests/split/l-chunk.sh: Adjust to again disallow -n /dev/zero.
Also change all '&& fail=1' checks to use the 'returns_ 1' form.
* tests/split/filter.sh: Change the no longer supported /dev/zero
case to a regular $OFF_T_MAX file (supported on XFS for example).
Also fix the timeout(1) commands so they're not subject to
pipefail issues.
* src/stty.c (usage): Remove an erroneous call to translate an
empty string, added in commit v8.23-112-g564f84a, which results
in the gettext header being printed for translated languages.
* THANKS.in: Remove the now committer.
* NEWS: Mention the bug fix.
* tests/cp/parent-perm-race.sh: Add timeouts so that the test does
not wait forever on GNU/Hurd. This does not fix the underlying
bug but at least lets the tests make progress.
Problem reported by Nelson H.F. Beebe in: http://bugs.gnu.org/22624
Also, check that 'cp' actually copies the data; this is a better
test anyway, and simplifies the test code.
Also, avoid unlikely race if 'pid' is set in the environment.
Problem reported by Nelson H.F. Beebe in: http://bugs.gnu.org/22624
Other problems also fixed: basically, the code got confused because
GNU/Linux reports that /dev/zero has size zero.
* src/split.c (input_file_size): Now takes struct stat *, not just
size. Always store the first buffer. All callers changed. Treat
/dev/zero as an infinitely-large file, both on GNU/Linux where
fstat and lseek say its size is zero, and on GNU/Hurd where they
say the size is OFF_T_MAX.
(cwrite): Return true on success.
(bytes_split): Don't try to read past EOF, and stop if a write fails.
(lines_rr): Omit stray check for ignorable errno.
(main): Get file size only when n_units > 1, since that's the only
time it is needed. Defer most of the work to input_file_size.
* tests/split/l-chunk.sh: Adjust tests to match new behavior
on oddball inputs.
* src/date.c (usage): The colon is used in the timezone offset
since commit v8.24-64-g17bbf6c.
* THANKS.in: Remove committer.
Fixes http://bugs.gnu.org/22491
Solaris Studio 12 on sparc (not x86) will not remove
unused functions, thus leaving a reference to an
undefined program_name symbol from emit_try_help().
* src/system.h (emit_try_help): Change from an inline function
to a macro, so that the inline function is not actually
defined in libstdbuf.c.
Fixes http://bugs.gnu.org/22430
* tests/tail-2/wait.sh: Restrict inotify specific test portion
to local file systems and also not with ---disable-inotify specified.
Failure noticed on NFS.
* tests/rm/rm1.sh: Also remove the group write bit which
was required on one NFS setup at least. Note u-w was
enough to deny file creation, g-w was also required to
deny file removal.
* tests/rm/cycle.sh: Likewise.
* tests/mv/perm-1.sh: Likewise.
* tests/cp/preserve-slink-time.sh: Add a delay between the
ln and the cp so that there is enough difference between
the timestamps so GPFS won't discard the update.
Reported by Assaf Gordon.
* tests/misc/shred-passes.sh: Specify an exact amount to shred,
to avoid running out of simulated random data on file systems
with a large st_blksize like GPFS for example.
Reported by Assaf Gordon.
tests/tail-2/F-headers.sh and test/tail-2/retry.sh fail on
on remote file systems due to tail going into inotify mode
due to not being able to determine the remoteness of the
non existent files.
* src/tail.c (any_non_remote_file): A new function used
to disable inotify when there are no open files, as
we can't determine remoteness in that case.
* NEWS: Mention the bug fix.
* gnulib: Update to latest where the only change is to
not unconditionally enable leaf optimization for fts on NFS,
as it was seen to abort() with some NFS servers as per:
https://bugzilla.redhat.com/1299169
This affects utilities that traverse directories like
cp, rm, chmod etc.
* NEWS: Adjust the improvement message to leave only XFS.
* tests/rm/dangling-symlink.sh: The effective delay was only
1.5s. So delay before the operation, and increase the iteration
count by 1 to increase the delay to 6.3s. This failure was
noticed once on a FreeBSD 10.2 x86_64 virtual machine.
* tests/dd/stats.sh: Remove quotes on sleep argument
to ensure arguments are accumulated appropriately.
* tests/du/move-dir-while-traversing.sh: sleep before operation,
to increase the effective delay from 1.6s to 3.2s.
* tests/tail-2/flush-initial.sh: Likewise.
* tests/tail-2/pipe-f2.sh: Sleep first so that the effective delay
matches the commented value.
* tests/rm/r-root.sh: This test was seen to ERROR due to
sed input not having a terminating '\n'. Therefore just skip
with diagnostics upon failure to adjust the error output with sed.
Reported by Assaf Gordon.
* init.cfg (dump_mount_list_): A new function to output the
system mount list.
* tests/df/df-symlink.sh: Call dump_mount_list_ upon failure.
* tests/df/over-mount-device.sh: Likewise.
* tests/df/problematic-chars.sh: Likewise.
* tests/df/skip-rootfs.sh: Likewise.
* tests/df/total-verify.sh: Likewise.
* tests/df/unreadable.sh: Likewise.
* tests/df/df-symlink.sh: Only check the symmetry of
the source <-> target lookup, when the source is only mounted once,
which isn't the case if '.' is a BTRFS subvolume for example.
Reported by Assaf Gordon on a default OpenSUSE 42.1 install.
* src/copy.c (copy internal): Remember directories irrespective
of their link count, because on some file systems like BTRFS,
directories always have a link count of 1.
* tests/misc/date.pl: This test was dependent on perl
unconditionally calling the system localtime() as date(1) does.
However perl has its own configure checks in place to avoid buggy
localtime implementations. Therefore comment out this test
as a FIXME for now. This test was seen to fail on FreeBSD 10
where localtime() returns nonsense values instead of failing
for arguments >= 2^56.
* tests/misc/head-c.sh: Further increase the limit, to account
for the fact that head(1) needs at least 1.576MB over the base test
command on FreeBSD 10. 4MB should be enough headroom while
still providing over allocation protection.
Reported by Assaf Gordon.
* src/operand2sig.c (operand2sig): Add a detailed comment explaining
why we validate even very large shell exit status values.
* tests/misc/kill.sh: Add a test case for the ksh scheme.
Simplify the INVALID signal number determination which also avoids
a false failure on systems like FreeBSD 10 with incomplete
signal list (caused by inaccurate NSIG).
* tests/cp/fiemap-extents.sh: Support RHEL6 fallocate
which doesn't support IEC suffixes like "MiB" on numbers.
Also add some extra framework_failure_ protections.
Reported by Assaf Gordon.
Locale categories are not equivalent on OpenBSD,
and LC_COLLATE only supports "C" for example.
Now LC_ALL is supported to set multiple other categories
on OpenBSD, so setlocale(LC_ALL, "") returns a string
indicating which categories were updated and which ignored.
Therefore...
* src/sort.c (main): ...Call setlocale(LC_COLLATE, "")
to explicitly check whether a specified LC_ALL or
LC_COLLATE environment variable value is supported
for the LC_COLLATE category. Also use !! to explicitly
convert to bool to support c89 systems where bool is an int,
and thus would get values > 1.
Reported by Assaf Gordon.
doc/coreutils.texi (dd invocation): Add oflag=sync to the streaming
example. Also reference the "direct" flag. Mention this is only
a request to the system.
* src/dd.c (usage): Mention the "sync" flag along with "nocache".
Also mention that it's only a request to drop the cache.
* THANKS.in: Add reporter Francois Rigault.
* src/stat.c (usage): Mention the '#' and '0' flags are useful with %a.
* doc/coreutils.texi (stat invocation): Likewise. Also give an example
printing unambiguous octal output.
Reported at http://bugs.debian.org/810539
commit v8.23-31-g90aa291 failed to consider this case,
where the previous rename has failed, thus causing the
following to remove the specified directory:
mv dir dir dir
* src/copy.c (copy_internal): Assume this rename attempt has
succeeded, as a previous failure will already have been handled,
and we don't want to remove the source directory in this case.
* tests/cp/duplicate-sources.sh: Consolidate this test file to...
* tests/mv/dup-source.sh: ...here. Add test cases for same
source and dest.
* tests/local.mk: Remove the consolidated test.
* NEWS: Mention the bug fix.
Reported at https://bugzilla.redhat.com/1297464
* src/wc.c (write_counts): Shell escape the file name
if it contains '\n' so only a single line per file is output.
* tests/misc/wc-files0.sh: Add a test case.
* NEWS: Mention the improvement.
* doc/coreutils.texi (numfmt invocation): Reference the description.
* src/numfmt.c: Parameterize '\n' references.
* tests/misc/numfmt.pl: Add tests for character and field processing.
* NEWS: Mention the new feature.
* doc/coreutils.texi (paste invocation): Reference -z description.
* src/paste.c (main): Parameterize the use of '\n'.
* tests/misc/paste.pl: Add test cases.
* NEWS: Mention the new feature.
* src/comm.c (main): Track the output delimiter length,
so that it can be adjusted to 1 for the NUL delimiter.
Also rename the global variable from "delimiter" to
"col_sep" so its use is more obvious, and to distinguish
from the recently added "delim" global variable.
* tests/misc/comm.pl: Adjust accordingly.
* doc/coreutils.texi (comm invocation): Reference option description.
* src/comm.c (main): Use readlinebuffer_delim() to support
a parameterized delimiter.
* tests/misc/comm.pl: Add test cases.
* NEWS: Mention the new feature.
* doc/coreutils.texi (tac invocation): Mention the
NUL delineation with an empty --separator.
* src/tac.c (main): Allow an empty separator when -r not specified.
* tests/misc/tac.pl: Add test cases.
* NEWS: Mention the new feature.
Fixes http://bugs.gnu.org/8103
* doc/coreutils.texi (cut invocation): Reference the description.
* src/cut.c: Parameterize '\n' references.
* tests/misc/cut.pl: Add tests for character and field processing.
* NEWS: Mention the new feature.
* NEWS: Mention the change in behavior.
* doc/coreutils.texi (newlineFieldSeparator): A new description,
referenced from ({join,sort,uniq} invocation).
* src/system.h (field_sep): A new inline function to determine
if a character is a field separator.
* src/join.c (usage): s/whitespace/blank/ to be more accurate
wrt which characters are field separators.
(xfields): s/isblank/field_sep/.
* src/sort.c (inittables): Likewise.
* src/uniq.c (find_field): Likewise.
* tests/misc/join.pl: Adjust -z test, and add a test/example
for processing the whole record with field processing.
* tests/misc/sort.pl: Add -z test cases, including case with '\n'.
* tests/misc/uniq.pl: Add -z -f test case with \n.
At least the false positive in tail-2/follow-stdin.sh could be seen
on a 4-core i5 system with -j8. Fix similar cases. too.
* tests/tail-2/follow-stdin.sh: Empty the 'out' file in each iteration.
Otherwise, under heavy system load, 'check_tail_output' would see the
expected output from the previous round before tail would have the
chance to come up in the background.
While at it, move the creation of the 'exp' file out of the loop.
* tests/dd/stats.sh: Empty the output file of the background process
here, too.
* tests/misc/cat-buf.sh: Likewise.
* tests/misc/stdbuf.sh: Likewise.
* tests/tail-2/wait.sh: Likewise.
This is used to discard terminal output on FreeBSD based systems.
* src/stty.c (mode_info): Add FLUSHO where supported.
(usage): Document for "local" and "combination" settings.
* doc/coreutils.texi (local stty settings): Document,
mentioning that this option is supported but ignored on GNU/Linux.
Instead of commit v8.24-132-g5171bef which only provides
control to disable this behavior (with -I), provide
the symmetrical "[-]drain" special setting.
* src/stty.c (main): Parse the [-]drain setting instead of -I,
and treat like a global option.
(usage): Adjust accordingly.
* tests/misc/stty.sh: Test "drain" with and without options.
* NEWS: Mention the new feature.
In some cases an initial drain may block indefinitely as discussed at:
http://lists.gnu.org/archive/html/coreutils/2016-01/msg00007.html
* src/stty.c (main): Use TCSANOW rather than TCSADRAIN if -I specified.
(usage): Document the new option.
* doc/coreutils.texi (stty invocation): Likewise.
* tests/misc/stty.sh: Ensure -I is supported.
* NEWS: Mention the new feature.
* doc/coreutils.texi (coreutils): Remove this direntry
so that `info coreutils` will display the table of contents,
rather than the unlikely intended coreutils Multi-call invocation.
This also supports lookups like `info coreutils readlink`,
but note that only works with the standard info viewer,
and not with pinfo for example.
* src/install.c (usage): Mention this commonly required functionality
in the -D option description.
* doc/coreutils.texi (install invocation): Likewise for the
--target-directory description.
* src/install.c (main): As an optimization, when calling
install_file_in_dir() for each file, only attempt to create
the target directory once, as this is invariant over the loop.
* src/install.c (mkancesdirs_safe_wd): Unconditionally
restore the current working directory when possibly called
multiple times (from install_file_in_dir()).
* tests/install/create-leading.sh: Add a test case.
* NEWS: Mention the fix.
Fixes http://bugs.gnu.org/21497
Since commit v8.23-60-g414a8cf, the above check failed to find sources
where the length of the longest line is a 3 or more digit number.
* cfg.mk (sc_long_lines): Remove the '\' escape character before '{'
and '}', because this would mean literal '{' and '}' characters in
an extended regular expression in sed(1).
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
Problem noted by Pádraig Brady in: http://bugs.gnu.org/22277#8
Also, make the output a bit more precise while we're at it.
* NEWS: Document this.
* src/dd.c (previous_time): Remove, replacing with ...
(next_time): New var. All uses changed.
This avoids some rounding errors, and should be a bit faster.
(newline_pending): Remove, replacing with ...
(progress_len): New var. All uses changed.
This lets us keep track of how many trailing spaces to append.
(print_xfer_stats): Get the time first thing, so that it's
closer to being correct. Count the bytes output, and append
trailing spaces if needed. Add remarks to translators about
translation lengths.
Problem reported by Linda Walsh in: http://bugs.gnu.org/17505
* NEWS: Document this.
* doc/coreutils.texi (dd invocation): Use a simpler script.
Adjust output example to match new behavior.
* src/dd.c (human_size): Remove.
All uses changed to use human_readable and ...
(human_opts): ... this new constant.
(abbreviation_lacks_prefix): New function.
(print_xfer_stats): Use it. Output both --si and --human-readable
summaries, but only if they have prefixes.
* tests/dd/reblock.sh, tests/dd/stats.sh: Test new behavior.
Mainly for these changes:
- freadptr: fix to work with ungetc on all uClibc configs
- fts: enable leaf optimization for XFS
- fts: ensure leaf optimization used for NFS
- strftime-tests: avoid false failure on OS X
- intprops-tests: avoid warnings (causing CI failures)
* NEWS: Update with items from above that are
significant from the previous coreutils release.
* doc/coreutils.texi (Connectives for test): Add notes
on precedence and associativity. Also mention the
portability caveats with these operators.
* cfg.mk: Avoid sc_prohibit_test_minus_ao for coreutils.texi.
Fixes http://bugs.gnu.org/22216
Commit v8.24-116-g896006b changed the file permissions for the tarball
using ug+rw. Better to let the umask decide whether the file should
be group-writeable or not.
* Makefile.am (dist-hook): Change chmod call from 'ug+rw' to '+rw'.
* init.cfg (get_min_ulimit_v_): Increase the determined
virtual memory limit for the command by a page size
to avoid alignment issues causing false failures for
subsequent runs.
* tests/misc/cut-huge-range.sh: Be defensive and match
the cut invocations under test, more closely with the
form used to determine the VM memory limit.
This was noticed on i686 linux at:
http://hydra.nixos.org/build/28990456
* Makefile.am (dist-hook): Ensure all files are writable
so that extracted tarballs are easy to remove, without
getting prompts about removing read-only files.
Reported by Benno Schulenberg.
We were erroneously skipping blanks in the marked comparison
_after_ the key start offset was applied.
* src/sort.c (debug_keys): Don't skip starting blanks
if already handled by begfield().
* tests/misc/sort-debug-keys.sh: Add a test case.
* NEWS: Mention the bug fix.
Fixes http://bugs.gnu.org/22155
* src/stat.c (human_fstype): Add "overlayfs", and tag it as "remote"
to ensure that tail continues to use the more conservative polling mode.
* README-release: Remove a stale comment about updating fremote()
in tail.c. Also give a link to *_SUPER_MAGIC definitions
not in the standard linux/magic.h location.
* NEWS: Mention that this file system is recognized.
procps, bash >= 4.4, and util-linux >= 2.23
all have the -L option to output a table.
* src/kill.c (main): Have -L synonymous with -t.
Luckily there is no signal that begins with 'L'.
* tests/misc/kill.sh: Update the test.
The character 'K' representing numeric user input can be confusing,
especially in the context where it can be suffixed by the multipliers
K, M, G, etc.
Use NUM instead.
* doc/coreutils.texi (head invocation): Replace @var{k} by @var{num}.
Also change @var{count} in the paragraph about the obsolete option
syntax.
(tail invocation): Likewise. Furthermore, adjust the option order to
alphabetical sorting.
* src/head.c (usage): s/K/NUM/ in as placeholder in the -c and -n
options. Furthermore, print the actual DEFAULT_NUMBER instead of the
hard-coded 10.
* src/tail.c (usage): s/K/NUM/ in as placeholder in the -c and -n
options. Add "[+]" to clarify that NUM may optionally be preceded by
this sign. Remove the redundant explanation of the optional '+' prefix
for NUM, as that is now clear from the option description above.
Suggested by Dan Jacobson in http://bugs.gnu.org/22042
* man/head.x: Add "SEE ALSO" section referencing tail(1).
* man/tail.x: Add "SEE ALSO" section referencing head(1).
Suggested by Dan Jacobson in http://bugs.gnu.org/22041
Use a consistent date in the generated man pages
even if SOURCE_DATE_EPOCH is not set.
* Makefile.am: Generate .timestamp in the tarball.
* man/local.mk: Pass .timestamp if available to help2man.
* man/dummy-man: Don't bother with the year in the stub.
Reported in http://bugs.debian.org/806321
Sync with version 1.47.3 which provides these significant changes:
- support for reproducible builds by using $SOURCE_DATE_EPOCH.
- Adjust spacing of italic text at roman/italic boundaries.
* doc/local.mk (constants.texi): Avoid calling makeinfo unless
the constants derived from tail and shred have changed.
This avoids a dependence on makeinfo from patched sources,
and avoids an expensive makeinfo call for developers.
* doc/coreutils.texi (uniq invocation): Fix the description of
problematic input to say "blank lines" rather than "two or more
blank lines".
(tr invocation): Clarify that -s only works with SET1 when
not translating (when SET2 not specified). Also explicitly
state in examples where blank lines are deleted.
Also add "deleting" to the menu item.
* src/tr.c (usage): Improve the -s summary to say it always
operates on the last specified SET.
* doc/coreutils.texi (tail invocation): Provide an example using awk
to convert tail ==> file <== headers to file: prefixes on each line.
Suggested by Stephen Shirley.
fallocate can allocate extents beyond EOF via FALLOC_FL_KEEP_SIZE.
Where there is a gap (hole) between the extents, and EOF is within
that gap, the final hole wasn't reproduced, resulting in silent
data corruption in the copied file (size too small).
* src/copy.c (extent_copy): Ensure we don't process extents
beyond the apparent file size, since processing and allocating
those is not currently supported.
* tests/cp/fiemap-extents.sh: Renamed from tests/cp/fiemap-empty.sh
and re-enable parts checking the extents at and beyond EOF.
* tests/local.mk: Reference the renamed test.
* NEWS: Mention the bug fix.
Fixes http://bugs.gnu.org/21790
* src/md5sum.c (digest_check): Update a matched_checksums bool upon
matched checksum, and fail (loudly unless --status is specified)
if there were no matches. Also change properly_formatted_lines
to a bool while at it since we don't need to track the plurality.
* tests/misc/md5sum.pl: Add a test case.
Suggested by Jim Meyering.
* doc/coreutils.texi (md5sum invocation): Document the new option.
* src/md5sum.c (digest_file): Return an empty digest to indicate
a missing file.
(digest_check): Don't fail or output status given an empty checksum.
(usage): Document the new option.
(main): Process and validate the new option.
* tests/misc/md5sum.pl: Add new test cases.
* NEWS: Mention the new feature.
Fixes http://bugs.gnu.org/15604
This can be useful if you want to further process data
from process substitutions. For example:
datagen | tee >(md5sum --tag) > >(sha256sum --tag) | sort
* doc/coreutils.texi (tee invocation): Mention that -p is
useful with pipes that may not consume all data.
Add an example, similar to the one above.
* THANKS.in: Add Jirka Hladky.
* tests/ls/stat-failed.sh: Skip the test if 'd' is returned as the type,
and document where this was seen. Also flag failure to write small
temp files during the test as an error rather than a failure.
Fixes http://bugs.gnu.org/21130
* src/local.mk: Run `mkdir -p src` in all our explicit rules,
as in a VPATH build the src/ dir is only created at configure time
as a side effect of dependency tracking generation.
discussed in:
http://lists.gnu.org/archive/html/coreutils/2015-10/msg00091.html
* src/csplit.c: (save_line_to_file): check fwrite failures, report
and exit immediately instead of deferring to 'close_output'.
* tests/misc/csplit-io-err.sh: test fwrite failure using LD_PRELOAD.
* tests/local.mk: add new test.
programs may not be built due to missing system dependencies,
or any program can be excluded at configure time with
--enable-no-install-program. So ensure we're not testing the
system versions in these cases.
* init.cfg (print_ver_): Call require_built_ first.
* tests/misc/tty-eof.pl: Skip programs not built.
* tests/Coreutils.pm (run_tests): Likewise.
* tests/misc/ls-misc.pl: Use 'env test' rather than abs path.
* tests/misc/test-diag.pl: Likewise.
* tests/local.mk: Adjust include order for dependencies.
* tests/misc/arch.sh: Remove redundant calls to require_built_.
* tests/misc/chroot-fail.sh: Likewise.
* tests/misc/groups-dash.sh: Likewise.
* tests/misc/groups-version.sh: Likewise.
* tests/misc/stdbuf.sh: Likewise.
* tests/cp/acl.sh: Remove problematic call to print_ver_ [gs]etfacl.
* tests/mv/acl.sh: Likewise.
* cfg.mk (sc_env_test_dependencies): A new syntax check to enforce
specifying dependencies with print_ver_ for programs
specified through the env command.
* du/bigtime.sh: Add new print_ver_ dependencies.
* du/max-depth.sh: Likewise.
* dd/ascii.sh: Likewise.
* tests/ls/capability.sh: Likewise.
* tests/ls/root-rel-symlink-color.sh: Likewise.
* tests/misc/chroot-fail.sh: Likewise.
* tests/misc/readlink-fp-loop.sh: Likewise.
* tests/misc/sort-debug-keys.sh: Likewise.
* tests/readlink/can-e.sh: Likewise.
* tests/readlink/can-f.sh: Likewise.
* tests/readlink/can-m.sh: Likewise.
* tests/tail-2/inotify-race.sh: Likewise.
* tests/tail-2/inotify-race2.sh: Likewise.
* tests/touch/no-create-missing.sh: Likewise.
* tests/touch/no-dereference.sh: Likewise.
* tests/misc/printenv.sh: Tweak to avoid syntax check trigger.
* tests/misc/help-version.sh: Likewise.
* tests/misc/yes.sh: Likewise.
* tests/misc/printf-quote.sh: Use previously unused $prog.
* configure.ac (EXTRA_MANS): Add $gl_no_install_prog to the list
so that check-x-vs-1 syntax check is satisfied.
A side effect of this cleanup is we no longer
depend on our own kill command being built.
* init.cfg (require_trap_signame_): A new function to verify
that the shell supports specifying traps by signal name.
(require_kill_group_): A new function to ensure the shell
supports sending a signal to a group.
* tests/du/move-dir-while-traversing.sh: Ensure trap supports
signal names.
* tests/misc/stty-invalid.sh: Likewise.
* tests/misc/stty-pairs.sh: Likewise.
* tests/misc/stty-row-col.sh: Likewise.
* tests/misc/stty.sh: Likewise.
* tests/misc/sort-compress.sh: Likewise. Also simplify trap call.
* tests/install/trap.sh: Likewise.
* tests/misc/timeout.sh: Likewise.
* tests/dd/stats.sh: Likewise. Also use default kill command.
* tests/misc/timeout-group.sh: Likewise.
* init.cfg (require_dirent_d_type_): Don't use df -x
to exclude XFS, since this depends on a correct mtab
which is brittle and often not correct within chroots.
* tests/d_type-check: Check also the d_type of files,
which excludes XFS appropriately. Specify all argument
and return types to avoid truncated pointers being passed,
which skipped the test due to crashes on x86_64 at least.
Simplify the C library lookup by reusing the interpreter's.
chroot issue reported at https://bugzilla.redhat.com/1263341
* src/paste.c (main): Use our styled wrapper for quotearg_colon().
* cfg.mk (sc_prohibit-quotearg): A new syntax check to avoid
future uses of unstyled quotearg to one of the internal slots,
and thus destined for diagnostic output.
* src/paste.c (main): Setting the quoting style to "escape"
went against the intent of the comment about presenting
doubled backslashes to the user. Instead use "c-maybe"
which is the only mode which avoids doubled backslashes,
and provides protection against arbitrary control characters.
* tests/misc/paste.pl: Adjust accordingly.
* src/ls.c (decode_switches): Set "shell-escape" if isatty().
* doc/coreutils.texi (ls invocation): Update the defaults description.
* NEWS: Mention the change in behavior. It should not have
backwards compat issues, but mentioning here just in case.
* src/printf.c (usage): Mention the new format.
(print_formatted): Handle the quoting by calling
out to the quotearg module with "shell-escape" mode.
* doc/coreutils.texi (printf invocation): Document %q.
* tests/misc/printf-quote.sh: New test.
* tests/local.mk: Reference new test.
* NEWS: Mention the new feature.
This is mainly noticeable when the multi-byte code
within ls.c is triggered by multi-byte quotes.
$ seq 200000 | xargs touch
$ time ls-old -U --quoting=locale --hide-control-chars >/dev/null
real 0m0.483s
$ time ls-new -U --quoting=locale --hide-control-chars >/dev/null
real 0m0.430s
* src/ls.c (quote_name): Avoid rescanning the output looking for
unprintable chars when we know the quoting mode already escapes them.
* tests/misc/ls-misc.pl: Add tests for all quoting modes, with and
without -q, to verify this assumption.
* doc/coreutils.texi (ls invocation): Describe the new
'shell-escape' and 'shell-escape-always' quoting options.
* src/ls.c (usage): Mention the new quoting options.
* tests/misc/ls-misc.pl: Add a test for 'shell-escape'
Quote file names using the "shell-escape" or "shell-escape-always"
methods, which quote as appropriate for most shells,
and better support copy and paste of presented names.
The "always" variant is used when the file name is
embedded in an error message with surrounding spaces.
* cfg.mk (sc_error_shell_quotes): A new syntax check rule
to suggest quotef() where appropriate.
(sc_error_shell_always_quotes): Likewise for quoteaf().
* src/system.h (quotef): A new define to apply shell quoting
when needed. I.E. when shell character or ':' is present.
(quoteaf): Likewise, but always quote.
* src/*.c: Use quotef() and quoteaf() rather than quote()
where appropriate.
* tests/: Adjust accordingly.
* src/md5sum.c: Use the same file name escaping method used
when generating and checking checksums. I.E. ensure a single line
per file by starting the line with '\' for any file name containing '\n'
and replacing those with "\\n".
* NEWS: Move the item from changes in behavior to improvements,
since this is no longer a backwards incompat change when
processing stdout status messages.
* tests/misc/md5sum.pl: Remove quotes from expected status output.
* tests/misc/sha1sum.pl: Likewise.
Related to commit v8.24-61-g6796698 this provides
more consistent quoting, as quotearg_colon() defaults
to "literal" quoting by default, while quote()
provides appropriate quoting for diagnostics by default.
* gl/modules/randread: Depend on quote module rather than quotearg.
* gl/lib/randread.c: Used quote() not quotearg_colon().
* src/: Likewise.
* src/shred.c: Likewise. Also avoid unnecessary quoting
introducing overhead when wiping names.
* cfg.mk: Relax the matching expression to allow
"qname" variables as used in shred.c to satisfy the check.
* tests/: Adjust accordingly.
* doc/coreutils.texi: (tail invocation): Add missing -s,
along with the existing --sleep-interval description.
(csplit invocation): s/--suffix/--suffix-format/.
(head invocation): Use same variable (COUNT) for -n and --head-count.
(seq invocation): Add opindex items for all options.
(ptx invocation): Likewise.
Fix typo s/--flac-truncation/--flag-truncation/.
(touch invocation): State explicitly that -d takes a parameter,
which also indicates that an '=' is not to be used
for the short option syntax.
(ls invocation): Likewise for the -w option.
Fixes http://bugs.gnu.org/21809
* doc/coreutils.texi (du invocation): Remove 'like' from
mentions of ISO-8601 as the components are individually conformant.
(ls invocation): Likewise, except for --time-style=iso for recent
files, where the MM-DD component is not a valid ISO-8601 timestamp.
* src/date.c (main): Use %:z rather than %z with --iso-8601
as the standard states to consistently use extended format.
Note either format can be parsed by date.
* tests/misc/date.pl: Adjust accordingly.
* doc/coreutils.texi (du invocation): Clarify that "iso"
time styles are only similar to ISO-8601.
(ls invocation): Likewise.
(date invocation): Adjust the comment stating
that only --rfc-3339 output can be parsed by date(1).
* NEWS: Mention the change in behavior.
Reported at http://bugs.debian.org/799479
* src/factor.c: Move LIKELY() definition to...
* src/system.h: ...here.
(is_nul): Reimplement with a version that doesn't
require a sentinel after the buffer, and which calls
down to (the system optimized) memcmp.
Performance analyzed at http://rusty.ozlabs.org/?p=560
* src/dd.c (alloc_obuf): Simplify the is_nul() call by
not needing to write the sentinel.
* src/copy.c (sparse_copy): Likewise.
(copy_reg): Simplify the buffer allocation by avoiding
consideration of the sentinel in the buffer size calculation.
These strings are often file names or other user specified
parameters, which can give confusing errors in
the presence of unexpected characters for example.
* cfg.mk (sc_error_quotes): A new syntax check rule.
* src/*.c: Wrap error() string arguments with quote().
* tests/: Adjust accordingly.
* NEWS: Mention the improvement.
This is especially significant when using --check
with files generated on a windows system, where the \r
characters produce corrupted and confusing error messages.
This also ensures status messages are output on a single line.
* src/md5sum.c: Use quote() for printed file names.
* tests/misc/md5sum.pl: Adjust accordingly.
* NEWS: Mention the change in behavior.
Fixes http://bugs.gnu.org/21757
* src/ls.c (main): Account for the first column not including
a separator when calculating max_idx.
* tests/ls/w-option.sh: Add a test case.
* NEWS: Mention the bug fix.
* src/ls.c (print_with_separator): Renamed from print_with_commas,
and parameterized to accept the separator to print.
Also fix an edge case where '\n' not printed when
the POS variable overflows SIZE_MAX.
(print_current_files): Degenerate -x and -C to using the
cheaper print_with_separator() in the -w0 case.
* doc/coreutils.texi (ls invocation): Document the new feature.
* tests/ls/w-option.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the improvement.
Fixes http://bugs.gnu.org/21325
* src/ls.c (know_term_type): Corresponding to commit v8.24-48-gc249a5a,
use fnmatch to inspect the dircolors database. Noticed due to
failing tests/ls/color-{dtype-dir,term}.sh tests.
The gnulib provided sc_tight_scope target was ineffective,
as it was checking against an invalid blank regular expression,
and thus ignoring any extern function issues. This is now
fixed up in gnulib, and so we need to fix our scoping issues
before the next gnulib update.
* cfg.mk: Setup and document the tight_scope config variables
appropriately.
* src/selinux.h: Since declared in *_SOURCES, use the two line
form for the extern function declarations.
* src/set-fields.h: Add the extern declarations, and since declared
in noinst_HEADERS use the single line form.
It was a little confusing as to whether the SQUFOF algorithm was
enabled, and in fact there were no options available to enable it.
Therefore clarify the 3 configurable behaviors for the code to
3 defines at the top of the program, and only include the SQUFOF
code if enabled at compile time.
$ size src/factor-before
text data bss
93997 1412 2504
$ size src/factor-after
text data bss
87885 1404 2504
* src/factor.c: Only include the SQUFOF factor code
when enabled via the USE_SQUFOF define.
* doc/coreutils.texi (factor invocation): Update note about
factor limits, as we can factor 128 bit numbers without GMP.
* tests/local.mk (all_tests): Remove the tests 'tests/id/setgid.sh' and
'tests/mkdir/smack-root.sh' because they are mentioned in the
'all_root_tests' list; these tests are skipped anyway during a non-root
run because flagged with 'require_root_'.
sha512sum can be faster than sha256sum.
E.g., ‘dd if=/dev/zero bs=1024k count=1024 | time sha256sum’
reports 8.16 user CPU seconds on my host, whereas sha512sum
consumes 5.45 seconds (Fedora x86-64 on an AMD Phenom II X4 910e).
Although sha512sum is still considerably slower on x86, a good
chunk of uses are on 64-bit hosts and anyway there’s little point
to scaring people away from sha512sum nowadays.
* doc/coreutils.texi (sha2 utilities): Remove obsolete comment.
* src/uniq.c (main): Make the type of "nfiles" unsigned,
to avoid a brand new warning from a gcc I built from today's
sources (gcc version 6.0.0 20151015 (experimental) (GCC)):
src/uniq.c:523:14: error: assuming signed overflow does not occur \
when simplifying conditional to constant [-Werror=strict-overflow]
if (nfiles == 2)
^
* src/dircolors.c (dc_parse_stream): Support globbing of
TERM entries, to allow entries like "TERM *256color*" for example.
* src/dircolors.hin: Reduce the internal list with globbing.
* tests/misc/dircolors.pl: New test cases.
* NEWS: Mention the improvement.
* tests/rm/r-root.sh: Skip the test if there are gdb warnings
that will impact further stderr checks. For example some
buggy gdb versions may report "Got object file from memory
but can't read symbols: File truncated". Also fix an incorrect
stderr check from the previous change.
Reported by Bernhard Voelker.
* tests/rm/r-root.sh: Use gdb rather than timeout(1) as the
last resort protection against unlinkat() calls. The timeout
of 2s was susceptible to false positives under load, and
gdb is stronger protection in any case. We remove the
"expensive" tag on this test also since it should be robust.
Reported by Jim Meyering.
* tests/tail-2/follow-stdin.sh: Use the standard tail
testing framework to avoid the race seen under very high load,
and also test the non inotify case.
Reported by Jim Meyering
To reproduce:
setfacl -dm group::rx .
setfacl -dm other::rx .
make check
* init.cfg (require_no_default_acl_): A new function to skip
when default ACLs are detected, or if the getfacl utility is
not available then skip if any non LSM ACLs detected.
* tests/cp/existing-perm-race.sh: Call require_no_default_acl_.
* tests/mkdir/parents.sh: Likewise.
* tests/mkdir/perm.sh: Likewise.
* src/tail.c (fremote): No longer prompt the user to email
with the unrecognized file system constant, since we have
process in place to sync periodically with the latest Linux
constants, and the fall back polling mode is still fully functional.
* tests/cp/cp-a-selinux.sh: Ensure we skip the portion of the test
depending on restorecon to be effective. I.E. also skip when restorecon
warns, as it doesn't exit with error status when matchpathcon fails to
find a match for a file. This is the case in /tmp on Fedora for
example, in which case the new destination that cp creates will have the
default security context of the process, rather than the explicit
context we set on the source file.
Details at: http://bugzilla.redhat.com/1247641
* tests/misc/sort-compress-hang.sh: Use --foreground with the
timeout(1) command (noting the caveats), to run the sort command
in the foreground program group, and thus be responsive to Ctrl-C.
This very_expensive_ test takes over a minute on a i3-2310M,
with RAM backed /tmp.
* tests/dd/no-allocate.sh: Account for timeout(1) when
determining the required mem, as timeout has additional shared libs.
This avoids the need for the hardcoded 4M addition to the limit.
* tests/misc/head-c.sh: Increase the base limit, to account for
the fact that head(1) will allocate some additional mem in this case.
* tests/misc/cut-huge-range.sh: Remove mention of specific limits.
* tests/misc/printf-surprise.sh: Likewise.
Reported by Dmitry V. Levin
* src/tee.c (tee_files): Last arg is now char ** instead of char
const **, as that is a bit simpler. All callers changed. Modify
files[-1], not files[nfiles], as that is a bit faster and simpler.
Latter problem pointed out by Rainer Deyke in:
http://bugs.gnu.org/21611
This pacifies GCC 5 in a better way, without disabling diagnostics.
* src/df.c (main): Tell compiler that optind is positive.
* src/shred.c (known): New function.
(dopass): Go back to off_t for file sizes.
Avoid integer overflow if we run off the end of the file.
Tell compiler that a write cannot write more bytes than requested.
Avoid the intermittent loss of "... has become inaccessible" messages.
That would cause tests/tail-2/assert.sh to fail sometimes,
mainly on uniprocessor systems.
* src/tail.c (tail_forever_inotify): Also monitor IN_DELETE
events on the directory, to avoid a dentry unlink()..open() race,
where the open() on the deleted file was seen to succeed after an,
unlink() and a subsequent IN_ATTRIB, was sent to tail. Note an
IN_ATTRIB is sent on the monitored file to indicate the change in
number of links, and we can't just use a decrease in the number of
links to determine the file being unlinked, due to the possibility
of the file having multiple links.
Reported by Assaf Gordon and Ludovic Courtès.
Fixes http://bugs.gnu.org/21460
* src/shred.c (dopass): With -O2, GCC 5.1 gives "assuming signed
overflow does not occur when simplifying conditional to constant"
warnings, in regard to the signed (off_t) variables. Therefore
use unsigned (uintmax_t) instead, and a separate boolean to cater
for the special meaning of the negative part of the integer range.
Noticed at http://hydra.nixos.org/build/24983447
Includes a change to xalloc.h to avoid -Wstrict-overflow warnings
with GCC 5.1 on 32 bit with optimization enabled. A subsequent
commit will fix similar issues in shred.
* src/dircolors.hin: Add "xterm-termite" as this VTE based terminal
emulator is quite different from xterm, despite the name.
For example "Termite supports italic text and it won't work if TERM
is set to xterm. Even the backspace key won't work properly anymore
for applications relying on terminfo".
Reported also by Lukas Sabota and Sven-Hendrik Haase.
In the presence of bind mounts of a device, the 4th "mount root" field
from /proc/self/mountinfo is now considered, so as to prefer mount
points closer to the root of the device. Note on older systems with
an /etc/mtab file, the source device was listed as the originating
directory, and so this was not an issue.
Details at http://pad.lv/1432871
* src/df.c (filter_mount_list): When deduplicating mount entries,
only prefer sources nearer or at the root of the device, when the
target is nearer the root of the device.
* NEWS: Mention the change in behavior.
This includes a change to propagate the 4th "mount root"
field from /proc/self/mountinfo from the mountlist module,
which is needed in a subsequent commit in df.
* gl/lib/regcomp.c.diff: Regenerate against latest gnulib.
* gl/lib/regex_internal.c.diff: Likewise.
* gl/lib/regex_internal.h.diff: Likewise.
* cfg.mk: Exclude diffs from trailing whitespace check,
which is generally correct, and now needed.
When configured with either 'symlinks' or 'shebangs' as value for
the --enable-single-binary option, tests based on `ulimit -v` are
skipped. The reason is that the multicall 'coreutils' binary requires
much more memory due to shared libraries being loaded, and the size of
the 'date' binary (~290KiB) compared to the multicall binary (~5MiB),
of course. Finally, in the case of 'shebangs', the starting shell
requires more memory, too
Instead of using hard-coded values for the memory limit, use an
adaptive approach: first determine the amount of memory for a similar,
yet more trivial invocation of the command, and then do the real test
run using that limit (plus some buffer in some cases).
* init.cfg (require_ulimit_v_): Remove function.
(get_min_ulimit_v_): Add function to determine the minimum memory limit
required for a given command in an adaptive way.
* cfg.mk (sc_prohibit_test_ulimit_without_require_): Change the name
of the above function in the syntax-check rule.
* tests/cp/link-heap.sh: Use the above function to determine the
minimum memory required to run a command simpler than in the real test
run. Use that limit plus a buffer there. While at it, change to list
of commands in the subshell to fail also if the beginning `ulimit -v`
fails.
* tests/dd/no-allocate.sh: Likewise.
* tests/misc/csplit-heap.sh: Likewise.
* tests/misc/cut-huge-range.sh: Likewise.
* tests/misc/head-c.sh: Likewise.
* tests/misc/printf-surprise.sh: Likewise.
* tests/split/line-bytes.sh: Likewise.
* tests/rm/many-dir-entries-vs-OOM.sh: Likewise - doing it separately
for each program under test.
* src/runcon.c (main): As per the compile time warning from
libselinux-2.4-3, lookup the class with string_to_security_class(),
rather than using defines from flask.h.
* src/sort.c (main): With --debug, warn upon setlocale() failure,
which can happen due to incorrectly specified environment variables,
or due to memory exhaustion (simulated with ulimit -v), etc.
* tests/misc/sort-debug-warn.sh: Add a test case.
See also http://savannah.gnu.org/bugs/11004
du calls stat for each mount point at startup. This would block or
even make du fail if stat for an unrelated mount point hangs.
The result is not needed in the normal case anyway and therefore
should be avoided. Issue introduced in commit v8.19-2-gcf7e1b5.
* src/du.c (fill_mount_table): Move function up as it's not used ...
(mount_point_in_fts_cycle): ... here, i.e., the DI_MNT set is
initialized and filled only iff FTS has detected a directory cycle.
(main): Remove the initialization and filling of the DI_MNT set here,
and free the DI_MNT set only if it was used.
The -NUMBER option was removed from ‘shred’ in 1999, but the
manual wasn’t updated to match. Problem reported by Nick Rose in:
http://bugs.gnu.org/21502
* doc/coreutils.texi (shred invocation):
Remove documentation for -NUMBER option.
numfmt --field=LIST can accept the same options as cut.
* bootstrap.conf: remove xlist, linked-list
* src/local.mk: link numfmt with set-fields
* src/numfmt.c: use set-fields.c instead of custom field parsing code.
(include_field): adapt to new code.
* tests/misc/numfmt.pl: add new tests, adapt current tests to new
error message wording from set-fields.c
Extract the functionality of parsing --field=LIST into a separate
module, to be used by other programs.
* src/cut.c: move field parsing code from here ...
* src/set-fields.{c,h}: ... to here.
(set_fields): generalize by supporting multiple parsing/reporting
options.
(struct range_pair): rename to field_range_pair.
* src/local.mk: link cut with set-field.
* po/POTFILES.in: add set-field.c
* tests/misc/cut.pl: update wording of error messages
* src/sort.c (main): Ensure we don't free() and invalid
pointer when reading implicit stdin. Also avoid
"definitely lost" valgrind warnings in the --files0-from case.
Since commit v8.23-19-g8defcee, main() will return,
rather than call exit(), this inducing "definitely lost"
warnings in valgrind's leak checker. That precludes using
the following to flag memory leaks:
valgrind --leak-check=full --error-exitcode=1 \
--errors-for-leak-kinds=definite
* src/pr.c (main): In dev builds, explicitly free memory allocated.
* src/sort.c (main): Likewise.
* src/tail.c (main): Likewise.
* src/tsort.c (tsort): Likewise.
* src/ls.c (getenv_quoting_style, decode_switches, parse_ls_color):
Use quote() rather than quotearg(), as the latter defaults to
outputting the input unquoted.
* src/ptx.c (main): Likewise.
* src/base64.c (main): Support decimal numbers with leading zeros,
by disabling the auto detection of octal and hex. It's not
envisaged that base conversion is needed for --wrap parameters,
and in the edge case it is, $((0x0)) shell constructs can be used.
* tests/misc/base64.pl: Adjust accordingly.
* NEWS: Mention the change in behavior.
* src/base64.c (main): Use the higher level xnumtoumax()
rather than xstrtoumax(), which is simpler and improves
validation of input. Also pass the _empty_ rather than NULL
string as the suffixes parameter so that invalid trailing
characters are not allowed. For example -w08 is now
flagged as an error, rather than being interpreted as 0.
A subsequent commit will further improve verification
of numbers with leading zeros by dropping backwards compatibility
wrt auto parsing oct and hex numbers.
* tests/misc/base64.pl: Add tests for invalid wrap values.
Suggested in https://bugzilla.redhat.com/1250113
* AUTHORS: Add base32.
* THANKS.in: Add suggester.
* README: Reference the new program.
* NEWS: Mention the new program.
* src/.gitignore: Ignore the new binary.
* bootstrap.conf: Reference the gnulib base32 module.
* build-aux/gen-lists-of-programs.sh: Add base32.
* man/base32.x: A new template.
* man/.gitignore: Ignore the new man page.
* man/local.mk: Reference the new man page.
* doc/coreutils.texi (base32 invocation): Document the new command.
* src/local.mk: Adjust to build base32 based on base64.c.
* src/base64.c: Parameterize to use the correct headers,
functions and buffer sizes, depending on which binary
is being built.
* tests/misc/base64.pl: Adjust to test both base32 and base64.
* tests/misc/tty-eof.pl: Add base32 as a program that
accepts input on stdin without any options specified.
* scripts/git-hooks/commit-msg: Add base32 to the template.
* src/shred.c (usage): For -u, separate the decscription
of the short and long option, to clarify that the short option
takes no parameter.
* src/split.c (usage): Likewise for -d.
* src/tee.c (usage): Likewise for -p.
* src/uniq.c (usage): Likewise for -D.
Suggested by Stephane Chazelas
Since glibc-2.22, specifically commit [0], the opendir() implementation
implicitly makes an additional stat call thus leading to a FP.
Seen on openSUSE:Tumbleweed since snapshot 20150821.
[0]
https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=46f894d8c60a
* tests/ls/stat-free-color.sh: Change the test to verify that ls(1)
needs the same number of stat-like calls for a single, empty directory
argument as for one with a few directory entries (sub-directory,
regular file, symlink, etc.).
* doc/coreutils.texi (split invocation): Clarify that -d takes no param.
(uniq invocation): Likewise for -D.
(shred invocation): Likewise for -u.
(tee invocation): Likewise for -p.
* cfg.mk (sc_man_check_x_copyright): Add rule to ensure that non-trivial
.x files in the 'man/' subdirectory, i.e., files exceeding a line count
of 20 or a byte count of 1000, contain a proper Copyright notice.
* bootstrap.conf (gnulib_modules): Add time_rz,
since the main source code now uses timezone_t.
* src/date.c (batch_convert, main, show_date):
* src/ls.c (align_nstrftime, long_time_expected_width)
(print_long_format):
* src/stat.c (human_time):
Use timezone_t rather than boolean to specify which time zone
is wanted.
* src/ls.c (localtz): New static var.
(main): Initialize it.
* src/uname.c (usage): State that the non POSIX -i and -p options
are non-portable.
* doc/coreutils.texi (uname invocation): Mention the discrepancies
even across GNU/Linux distros, and that the results should
be used as informational only, rather than impacting any
logic decisions.
Fixes http://bugs.gnu.org/13001
* src/realpath.c (usage): Mention 'directory' in the --help
output, so that ENOTDIR errors may be more easily investigated,
by inspecting the man page.
Reported at http://pad.lv/1474519
This was detected in about 25% of runs with gcc -fsanitize=address
ERROR: AddressSanitizer: global-buffer-overflow on address ...
READ of size 4 at 0x000000416628 thread T0
#0 0x40479f in genpattern src/shred.c:782
#1 0x4050d9 in do_wipefd src/shred.c:921
#2 0x406203 in wipefile src/shred.c:1175
#3 0x406b84 in main src/shred.c:1316
#4 0x7f3454a1ef9f in __libc_start_main (/lib64/libc.so.6+0x1ff9f)
#5 0x4025d8 (/tmp/coreutils-8.23/src/shred+0x4025d8)
0x000000416628 is located 56 bytes to the left of
global variable '*.LC49' from 'src/shred.c' (0x416660) of size 17
0x000000416628 is located 12 bytes to the right of
global variable 'patterns' from 'src/shred.c' (0x416540) of size 220
SUMMARY: AddressSanitizer: global-buffer-overflow src/shred.c:782
* src/shred.c (gen_patterns): Restrict pattern selection
to the K available, which regressed due to v5.92-1462-g65533e1.
* tests/misc/shred-passes.sh: Add a deterministic test case.
* NEWS: Mention the bug fix.
Fixes http://bugs.gnu.org/20998
* Makefile.am: Remove our dependence on src/sort which
induces awkward dependencies for `make dist` since
THANKS will be rebuilt once src/sort is newer.
Instead we remove the problematic -f option to sort
which actually doesn't change the order given
our current input.
Since non interactive shells don't generally set $SHELL,
its value is propagated through the tests and may cause issues;
for example if $SHELL implicitly adjusts $PATH when run.
Instead we set $SHELL to that determined by the posix-shell module,
and use that consistently for all test sub scripts,
including those created thorugh the `split --filter` command.
* tests/local.mk: Explicitly set $SHELL to $(PREFERABLY_POSIX_SHELL)
which defaults to $CONFIG_SHELL and thus usually /bin/sh.
* tests/envvar-check: Remove bash environment variables with
side effects, in case /bin/bash was selected for $SHELL.
* tests/misc/help-version.sh: Remove redundant initialization of $SHELL.
* tests/install/strip-program.sh: Use $SHELL for sub script.
* tests/misc/sort-compress-hang.sh: Likewise.
* tests/misc/sort-compress-proc.sh: Likewise.
* tests/misc/sort-compress.sh: Likewise.
* tests/misc/timeout-group.sh: Likewise.
* tests/rm/fail-eperm.xpl: Remove redundant elision of bash env vars.
* tests/misc/pwd-long.sh: Likewise.
tests/misc/wc-proc.sh fails when the page size is 64K
* src/wc.c (wc): The lseek adjustment should be based on st_blksize,
rather than on the internal buffer size. This is significant on
aarch64 where st_blksize in /proc is the 64K (the page size) and
thus larger than the internal buffer.
* src/split.c (main): Even though the similar processing is done
on the internal buffer size, that's based on st_blksize and
so fine in this regard. Add an assert to enforce this.
Avoid this path for the undocumented ---io-blksize option.
* m4/jm-macros.m4 (HAVE_FPSETPREC): Define if needed.
* src/numfmt.c (main): Call fpsetprec() if needed.
Fixes large-15 and large-16 test failures on 32 bit FreeBSD.
The new tests/misc/factor-parallel.sh test was
seen to fail on FreeBSD (derived) systems, which was
due to split(1) --filter reading partial lines
through pipes, as factor(1) was writing a little
over PIPE_BUF each time.
* src/factor.c (lbuf): A new structure to internally buffer lines.
(lbuf_alloc): A new function to allocate enough at program start.
(lbuf_putint): A new function to buffer a uintmax_t.
(lbuf_flush): A new function to write directly to standard output.
(lbuf_putc): A new function to buffer a character and if enough
lines are buffered, then output complete lines <= PIPE_BUF,
and continue to buffer the rest.
(main): Call the internal buffer allocator, and register
the final flush from the internal buffer at program exit.
* tests/dd/stats.sh: Wait 20s for dd to write 250MB through a fifo,
rather than 10s for 500MB. The failure was seen often on
a lightly loaded SPARC-Enterprise-T5220 running Solaris 10.
* bootstrap.conf (gnulib_modules): Add setenv, to make this
module dependency explicit; setenv is also used by split.
* src/stdbuf.c (set_LD_PRELOAD) [__APPLE__]: Use the OS X setenv
function, rather than putenv, per that documentation:
https://developer.apple.com/\
library/mac/documentation/Darwin/Reference/ManPages/man3/putenv.3.html
* src/numfmt.c (parse_field_arg): Rename parameter s/optarg/arg/,
to avoid shadowing getopt's global variable.
Otherwise, building on OS X, with --enable-gcc-warnings, I saw this:
In file included from src/numfmt.c:19:0:
src/numfmt.c: In function 'parse_field_arg':
./lib/config.h:3109:25: error: declaration of 'rpl_optarg' shadows\
a global declaration [-Werror=shadow]
* src/numfmt.c (double_to_human): Fix the argument order
passed to snprintf, which happened to work on amd64 with
its separate va_arg storage area for floats¹,
but would fail tests for example on i686.
¹ https://blog.nelhage.com/2010/10/amd64-and-va_arg/
* src/df.c (filter_mount_list): Clarify why we still stat even
though devices IDs may already be available. Note using
/proc/self/mountinfo is still an advantage to get filtered items
with accurate device patchs in chroots and with bind mounts.
I.E. on older setups with static /etc/mtab, df will now
bypass that to get the more accuracte and dynamic info.
* src/factor.c (print_uintmaxes): Comment that the
value of n_out doesn't matter on error, and add an
explicit cast to avoid any future warnings.
Suggested by Jim Meyering RE commit v8.23-229-g4d2d6c5
The LD_PRELOAD checks by -fsanitize=address are overly strict:
https://groups.google.com/forum/#!topic/address-sanitizer/jEvOJgkDqQk
A workaround is to first export LD_PRELOAD=libasan.so.2
The tests below are adjusted so that workaround is not discarded.
* tests/cp/no-ctx.sh: Append to $LD_PRELOAD.
* tests/df/no-mtab-status.sh: Likewise.
* tests/df/skip-duplicates.sh: Likewise.
* tests/ls/getxattr-speedup.sh: Likewise.
* tests/rm/r-root.sh: Likewise.
* tests/cp/nfs-removal-race.sh: Likewise. Also check that
LD_PRELOAD is effective to aid future maintainability
and avoid false failure if libasan.so.2 is not preloaded.
GCC 5.1.1 -fsanitize=undefined with glibc 2.21 is returning:
"runtime error: null pointer passed as argument 1,
which is declared to never be null"
* src/ptx.c (sort_found_occurs): Avoid the call with no entries.
* src/factor.c (n_out): A new global variable to track
how much data has been written to stdout.
(print_factors_single): Use n_out to determine whether
to flush the current (and previous) lines.
* tests/misc/factor-parallel.sh: Add a new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
src/seq.c (scan_arg): Set precision and width _after_ exponentiation.
For example, this will make '1.1e1 12' and '11 1.2e1' equivalent.
One can still set the precision by specifying extra precision on
the start value, or more naturally with a precision on a step value.
* tests/misc/seq-precision.sh: Add new cases.
* src/seq.c (scan_arg): Set precision to 0 for hex constants
(while avoiding hex floats). This will use then use the
fast path for these arguments. Note we also set the precision
of inf to 0 here, which ensures we use consistent precision
on output where possible.
* tests/misc/seq-precision.sh: Add corresponding test cases.
* src/seq.c (main): Call seq_fast for infinite last value.
This implicitly avoids format conversion on the
999999 -> 1000000 transition.
* src/seq.c (seq_fast): Generalize the buffer handling,
and adjust to handle the "inf" last value specifics.
* tests/misc/seq-precision.sh: A new test.
* tests/local.mk: Reference the new test.
* src/numfmt.c (MAX_UNSCALED_DIGITS): Set this to LDBL_DIG
rather than hardcoding at 18 for better portability.
* tests/misc/numfmt.pl: Restrict limit tests to supported platforms.
* src/numfmt.c (simple_strtod_int): Don't count leading zeros
as significant digits. Also have leading zeros as optional
for floating point numbers.
* tests/misc/numfmt.pl: Add test cases.
* NEWS: Mention the fix.
Due to existing limits this is usually triggered
with an increased precision. We also add further
restrictions to the output of increased precision numbers.
* src/numfmt.c (simple_round): Avoid intmax_t overflow.
(simple_strtod_int): Count digits consistently
for precision loss and overflow detection.
(prepare_padded_number): Include the precision
when excluding numbers to output, since the precision
determines the ultimate values used in the rounding scheme
in double_to_human().
* tests/misc/numfmt.pl: Add previously failing test cases.
* NEWS: Mention the fix.
* src/numfmt.c (usage): Update the --format description
to indicate precision is allowed.
(parse_format_string): Parse a precision specification
like the standard printf does.
(double_to_human): Honor the precision in --to mode.
* tests/misc/numfmt.pl: New tests.
* doc/coreutils.texi (numfmt invocation): Mention the new feature.
* NEWS: Likewise.
* src/numfmt.c: Replace field handling code with logic that understands
field range specifiers. Instead of processing a single field and
printing line prefix/suffix around it, process each field in the line
checking whether it has been included for conversion. If so convert and
print, otherwise just print the unaltered field.
(extract_fields): Removed.
(skip_fields): Removed.
(process_line): Gutted and heavily reworked.
(process_suffixed_number): FIELD is now passed as an arg instead of
using a global.
(parse_field_arg): New function that parses field range specifiers.
(next_field): New function that returns pointers to the next field in
a line.
(process_field): New function that wraps the field conversion logic
(include_field): New function that checks whether a field should be
converted
(compare_field): New function used for field value comparisons in a
gl_list.
(free_field): New function used for freeing field values in a gl_list.
Global variable FIELD removed.
New global variable all_fields indicates whether all fields should be
processed.
New global variable all_fields_after stores the first field of a N-
style range.
New global variable all_fields_before stores the last field of a -M
style range.
New global variable field_list stores explicitly specified fields to
process (N N,M or N-M style specifiers).
(usage): Document newly supported field range specifiers.
* bootstrap.conf: Include xlist and linked-list modules. numfmt now
uses the gl_linked_list implementation to store the field ranges.
* tests/misc/numfmt.pl: Add tests for 'cut style' field ranges.
Adjust existing tests as partial output can occur before an error
Remove test for the 'invalid' field -5.. this is now a valid range.
* gnulib: update to avoid compiler warnings in linked-list.
* NEWS: Mention the new feature.
* src/numfmt.c (unit_to_umax): Support SI (power of 10) suffixes
with the --from-unit and --to-unit options. Treat suffixes like
is done with --from=auto, which for example will change the meaning
of --to-unit=G to that of --to-unit=Gi. The suffix support was
previously undocumented and it's better to avoid the traditional
coreutils suffix handling in numfmt by default.
* doc/coreutils.texi: Document the new behavior. Also fix a typo
mentioning {from,to}=units=.
* tests/misc/numfmt.pl: Adjust accordingly.
* NEWS: Mention the change in behavior.
* gnulib: Update to get the new gnu-web-doc-update with --mirror option.
* README-release: Use the --mirror option in the instructions.
Also clarify and update various release steps.
* src/tail.c (recheck): Display diagnostices for replaced files
even with reused inodes which is a common case.
* tests/tail-2/F-vs-missing.sh: Use correct diagnostic in comment.
* tests/tail-2/F-vs-rename.sh: Likewise.
* src/tail.c (tail_forever_inotify): Use the fspec pointer to
distinguish previously output files, rather than a descriptor
from the inotify event. That event descriptor was that of
the parent directory when files were created or renamed etc.
(check_fspec): Adjust for the new comparison. Also show the
header when the file is truncated, since we show data
in this case also.
* tests/tail-2/F-headers.sh: A new test case.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
* .gitignore: Add entries for potentially generated headers.
Also remove a couple of items already present in lib/.gitignore.
* cfg.mk (sc_gitignore_missing): A new syntax check rule to
identify missing .gitignore entries.
(sc_gitignore_redundant): A new syntax check rule to
identify redundant .gitignore entries.
Reported by Tomas Nordin.
* configure.ac: Comment on why we link rather than run the test,
and remove the moot __ELF__ check since we never ran it anyway,
and the new CFLAGS and LDFLAGS are a more direct test of support.
* tests/misc/wc-parallel.sh: Fix a syntax error in the previous change.
* tests/misc/md5sum-parallel.sh: Use better error checking, consistent
with that used in wc-parallel.sh.
Problems reported by Michael Felt, and and part of this fix taken
from code suggested by Pádraig Brady in:
http://bugs.gnu.org/20733#112
* configure.ac (stdbuf_supported): Check for warnings, and
for -fPIC and -shared, for AIX.
* src/stat.c (STRUCT_STATVFS): Define to struct statvfs64 if
STATFS is statvfs64.
* src/sync.c (sync_arg) [_AIX]: Open in write mode,
since AIX fsync doesn't work on read-only file descriptors.
* tests/misc/wc-parallel.sh: Skip test if xargs -P does not work.
Problem reported privately by Michael Felt.
* Makefile.am (install-exec-hook):
* src/local.mk (src/coreutils_symlinks, src/coreutils_shebangs)
(clean-local):
Port to POSIX shell, which doesn't allow 'for i in ; do ...'.
* bootstrap.conf: Add "tempname" which is needed by mktemp(1).
The explicit dependency supports running gnulib-tool with
the --conditional-dependencies option, used to minimize built
modules. Note on a Fedora 22 system, that results in avoiding
redundant builds of: areadlinkat.o asnprintf.o fd-hook.o
fseterr.o printf-args.o printf-parse.o sockets.o vasnprintf.o.
However --conditional-dependencies is not enabled, since it
currently precludes the inclusion of gnulib tests.
* tests/tail-2/wait.sh: Without inotify, skip a portion of the test
that checks that -F never outputs from a tailed descriptor
after the followed name is recreated, because tail_forever()
doesn't guarantee that.
Noticed at http://hydra.nixos.org/build/22766288
* tests/misc/uniq-perf.sh: Use our standard 10s timeout,
which is sufficient to trigger the failure and also
avoids a false failure on slow/loaded systems.
Noticed at http://hydra.nixos.org/build/22766288
* src/sync.c (sync_arg): Initialise variable to avoid
unitialized access if assert is disabled.
* src/head.c (elide_tail_bytes_file): Support this function
with ---presume-input-pipe and larger files,
which regressed with commit v8.23-47-g2662702.
(elide_tail_lines_file): Likewise.
* src/dd.c (dd_copy): Explicitly don't try to ftruncate()
upon failure to lseek() (the existing check against
st_size was already protecting that).
* src/factor.c (factor_using_squfof): Assert (only when
linting due to performance) to avoid the implication of
divide by zero.
* src/od.c (read_block): Remove dead code.
* src/tac.c (tac_seekable): Likewise.
* src/ls.c (gobble_file): Likewise.
Now that we depend on gettext >= 0.19.2 remove the workaround
for issues in autopoint 0.18.3. Note the scheme currently used in
newer gettext (autopoint) to avoid these issues requires
autoconf >= 2.69, therefore we update this requirement also.
Note the gettext version dependence from gnulib comes from
gnulib using gettext macros, and coreutils indirectly depends on
the gettext module due to:
http://git.sv.gnu.org/gitweb/?p=gnulib.git;a=commitdiff;h=41dca647
bootstrap will then update m4/po.m4 and thus require a
supportng gettext version.
* bootstrap: Remove moot warning (resyncing with gnulib).
* configure.ac (AC_PREREQ): Change to 2.69 (now 3 years old).
* src/copy.c (CAN_HARDLINK_SYMLINKS): Don't enable use of linkat()
on Darwin 14, as the gnulib fallback emulation there doesn't
preserve ownership and timestamps etc. This fixes a test failure
in tests/cp/link-symlink.sh
* tests/cp/link-deref.sh: Adjust accordingly.
When the parent directory exists and has a different
default context to the final directory, the context
was incorrectly left as that of the parent directory.
* src/mkdir.c (process_dir): Because defaultcon() is called for
existing ancestors (as it must be to avoid races), then we must
unconditionally call restorecon() on the last component due to
the already documented caveat with make_dir_parents().
Alternatively you could temp disable o->set_security_context
around make_dir_parents(), but that would be subject to races.
* tests (tests/mkdir/restorecon.sh): Add a TODO for improvement.
Reference mknod and mkfifo with print_ver_.
* NEWS: Mention the bug fix.
Fixes http://bugs.gnu.org/20616
* tests/cp/no-ctx.sh: Scope of `var=val func` is inconsistent
across shells, so avoid that construct with functions.
* tests/df/no-mtab-status.sh: Likewise.
* tests/tail-2/inotify-race.sh: `read` needs an argument.
* tests/tail-2/inotify-race2.sh: Likewise.
* cfg.mk: Various syntax-check adjustments so that it's
not assumed the $builddir is the base distribution directory.
* Makefile.am: Likewise for the 'dist' target.
* cfg.mk (sc_case_insensitive_file_names): A new syntax-check rule.
* tests/tail-2/descriptor-vs-rename.sh: Rename from
tests/tail-2/f-vs-rename.sh
* tests/local.mk: Reference the renamed test.
Reported by Jim Meyering.
* tests/dd/sparse.sh: Sync files before checking allocations,
which may be done asynchronously on NFS and BTRFS at least.
Also mark this test as very expensive on remote file systems.
* tests/du/2g.sh: Likewise, also use fallocate if available
to efficiently allocate the large file, otherwise skip
on remote file systems.
* tests/tail-2/inotify-rotate-resources.sh: Use the more
standard is_local_dir_() to check remoteness.
* tests/cp/fiemap-empty.sh: Comment on the sync issue
for this currerntly unused test.
Fixes http://bugs.gnu.org/20570
* src/timeout.c (cleanup): Don't send SIGCONT to the monitored program
when --foreground is specified, as it's generally not needed for
foreground programs, and can cause intermittent signal delivery
issues with monitors like GDB for example.
* doc/coreutils.texi (timeout invocation): Mention that SIGCONT
is not sent with --foreground.
* NEWS: Mention the behavior change.
Supporting `split --numeric-suffixes=1 -n100` for example.
* doc/coreutils.texi (split invocation): Mention the two
use cases for the FROM parameter, and the consequences on
the suffix length determination.
* src/split.c (set_suffix_length): Use the --numeric-suffixes
FROM parameter in the suffix width calculation, when it's
less than the number of files specified in --number.
* tests/split/suffix-auto-length.sh: Add test cases.
Fixes http://bugs.gnu.org/20511
* src/wc.c (usage): State that it calculates display width.
* doc/coreutils.texi (wc invocation): Detail the distinct
items used to determine the display width.
Generally if logs are truncated, they're truncated to 0 length,
so output all existing data when our heuristic determines truncation.
Note with inotify, truncate() and write() are often determined
independently and so all data would be written if that was the case.
* src/tail.c (check_fspec): Reset file offset to 0 upon truncation.
(tail_forever): Likewise.
(recheck): Add a FIXME for the related issue where tail may lose
data due to tail discounting older log files too early.
* tests/tail-2/truncate.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the fix.
The previous fixes to races in the various tail tests,
identified actual races in the tail inotify implementation.
With --follow=descriptor, if the tailed file was replaced before
the inotify watch was added, then any subsequent changes were ignored.
Similarly in --follow=name mode, all changes to a new name were
effectively ignored if that name was created after the original open()
but before the inotify_add_watch().
* src/tail.c (tail_forever_inotify): Fix 3 cases.
1. With -f, don't stop tailing when file removed before watch.
2. With -f, watch right file when file replaced before watch.
3. With -F, inspect correct file when replaced before watch.
Existing tests identify these when tail compiled with TAIL_TEST_SLEEP.
* tests/tail-2/inotify-rotate-resources.sh:
This test also identifies the issue with --follow=name
when TAIL_TEST_SLEEP is used. Adjust so the test is immune
to such races, and also fail quicker on remote file systems.
* tests/tail-2/inotify-race2.sh: A new test using GDB,
based on inotify-race.sh, which tests the -F race
without needed recompilation with sleeps.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug.
* tests/tail-2/inotify-race.sh: Add a `wait` to ensure that
we reap all background gdb and tail processes. That resulted
in the test hanging intermittently and upon investigation was
due to gdb intermittently failing to terminate the child process
due to receiving a SIGCONT signal. Therefore we avoid using
timeout(1) which sends that signal, and instead rely on tail's
inbuilt --pid monitoring on a background sleep process.
Given this new implementation, the VERY_EXPENSIVE guard was removed.
Related issues with this test hanging were previously discussed at:
https://lists.gnu.org/archive/html/bug-coreutils/2009-12/msg00025.html
* tests/tail-2/F-vs-missing.sh: Use standard "fastpoll" options
(-s.1 --max-unchanged-stats=1) to speedup the non-inotify case.
Add the non-inotify case to the test. `wait` on the background
tail process to terminate which should avoid the need for the
non standard `retry_delay_ cleanup ...` on NFS.
* tests/tail-2/F-vs-rename.sh: Remove 'out' at the start of the loop,
to avoid a race in checking its contents. Also ensure 'a' & 'b'
files are present before the tail process starts. Use the standard
"fastpoll" options as above.
* tests/tail-2/f-vs-rename.sh: Likewise.
* tests/tail-2/append-only.sh: Use more standard variable names.
* tests/tail-2/flush-initial.sh: Use "fastpoll" options for
non-inotify platforms. Also `wait` on the background tail to avoid
stray processes and file cleanup issues on NFS.
* tests/tail-2/inotify-hash-abuse.sh: Always run non-inotify case.
Use "fastpoll" options. Use a more standard retry_delay_ instead
of a hardcoded sleep loop. Add a `wait` on the background tail.
* tests/tail-2/inotify-hash-abuse2.sh: Likewise.
* tests/tail-2/inotify-rotate-resources.sh: Wait just on the
specific tail $pid needed.
* tests/tail-2/inotify-rotate.sh: Use "fastpoll" options.
* tests/tail-2/pid.sh: Use standard variable names.
Add a `wait` on the background tails.
* tests/tail-2/pipe-f2.sh: Likewise.
* tests/tail-2/tail-n0f.sh: Likewise.
* tests/tail-2/retry.sh: Use "fastpoll" options.
* tests/tail-2/symlink.sh: Likewise.
* tests/tail-2/wait.sh: Likewise. Speedup by using sub second
parameters to timeout(1). Improve the part ensuring that
-F never follows a renamed file.
* tests/tail-2/infloop-1.sh: Remove invalid test. tail(1) was not
being passed the --pid=$yes_pid option, retry_delay_ wasn't used
to avoid races, and yes could write huge files before being killed.
* tests/local.mk: Remove the invalid test reference.
* tests/tail-2/assert-2.sh: Rewrite using retry_delay_(). Since
no longer hardcoding large delays, remove the VERY_EXPENSIVE tag.
* tests/tail-2/assert.sh: Likewise.
Without this change, very recent gcc (e.g., version 6.0.0 20150509)
would print the following when configured with --enable-gcc-warnings:
src/copy.c:165:30: error: logical 'or' of equal expressions \
[-Werror=logical-op]
&& (errno == EOPNOTSUPP || errno == ENOTSUP || errno == ENOSYS))
^
* src/system.h (is_ENOTSUP): New function.
* src/copy.c (punch_hole): Use it.
* src/ls.c (errno_unsupported, gobble_file): Use it.
* src/system.h (emit_stdin_note): A new function, refactoring
the usage note about the '-' FILE implying stdin.
* src/base64.c (usage): Use the new function to emit the
note in a standard location and with standard separation.
* src/cat.c (usage): Likewise.
* src/csplit.c (usage): Likewise.
* src/cut.c (usage): Likewise.
* src/expand.c (usage): Likewise.
* src/fmt.c (usage): Likewise.
* src/head.c (usage): Likewise.
* src/md5sum.c (usage): Likewise.
* src/nl.c (usage): Likewise.
* src/od.c (usage): Likewise.
* src/paste.c (usage): Likewise.
* src/pr.c (usage): Likewise.
* src/ptx.c (usage): Likewise.
* src/shred.c (usage): Likewise.
* src/shuf.c (usage): Likewise.
* src/sort.c (usage): Likewise.
* src/sum.c (usage): Likewise.
* src/tac.c (usage): Likewise.
* src/tail.c (usage): Likewise.
* src/tsort.c (usage): Likewise.
* src/unexpand.c (usage): Likewise.
* src/wc.c (usage): Likewise.
* src/join.c (usage): Adjust the separation used for
the message referring to FILE1 or FILE2 as stdin.
* src/comm.c (usage): Add a message using the same
wording (translation) as used in join.
* src/split.c (usage): Reword to using FILE rather than
INPUT, allowing use of emit_stdin_note(). Also remove
the mention of "fixed-size" pieces as this isn't now
always the case.
Fixes http://pad.lv/1450179
* tests/df/no-mtab-status.sh: getmntent is no longer called
when /proc/self/mountinfo is present, thus causing the test
to be skipped. Therefore wrap fopen() to ignore mountinfo,
and use the test genmntent table instead.
* tests/df/skip-duplicates.sh: Likewise.
* src/coreutils.c (usage): include coreutils.h outside
the printf call, because if it's a macro you will get the error:
embedding a #include directive within macro arguments is not supported
* src/yes.c (main): Simplify the logic so that the
compiler can see this function always returns a value.
This was seen with GCC 5.0 in SINGLE_BINARY mode.
* tests/ls/no-cap.sh: Ensure the test isn't skipped even if
capability coloring is disabled in the current $LS_COLORS.
Also just enable/disable capability coloring to avoid the
dircolors(1) overhead.
gnulib now only checks that the printf routines never crash,
which is all coreutils currrently requires, and so we revert
commit v8.23-81-gf57bfbb to let gnulib decide whether to replace
the system printf routines.
With GCC 5 and the newly added warnings from gnulib, ensure the
correct signed integer is passed for the printf format,
to avoid -Werror=format= failures.
* bootstrap.conf: 0.19.2 is available on openSUSE-13.2,
Debian-8.0, and Ubuntu-14.10. Given there were issues
with earlier 0.19 gettext releases, set this as the new minimum.
* configure.ac (AM_GNU_GETTEXT_VERSION): Likewise.
Reported by Bernhard Voelker
* doc/coreutils.texi (truncate invocation): The word 'their' is
incorrect; 'each file' is the antecedent, and is singular,
so 'its' is the correct pronoun.
* src/longlong.h: Sync with the latest longlong.h from libgmp to:
- Use __builtin_c[lt]zl on arm64.
- Fix sparc64 vis3 build failure due to missing __clz_tab.
- Avoid a clang build issue on mips.
- Support thumb2 arm 32 bit system.
* src/cfg.mk (sc_ensure_comma_after_id_est): Exclude longlong.h
to ease merges.
All warnings were of the form: "assuming signed overflow does not occur
when simplifying conditional to constant [-Werror=strict-overflow]"
* src/dd.c (cache_round): Use an appropriately sized unsigned type,
to avoid possibility of undefined signed overflow.
* src/mknod.c (main): Likewise.
* src/pr.c (pad_down): Likewise.
* src/wc.c (main): Likewise.
* src/tail.c (main): Assert that argc >= 0 thus allowing the
compiler to assume without implication that argc - optind
is positive.
* src/dircolors.hin: Add the MISSING entry, to indicate
this as a possibility in new templates output from dircolors,
and also to ease comparison with existing databases that
generally do define a MISSING entry.
* doc/coreutils.texi (cp invocation): Mention that when copying files
without preserving permissions, the umask or a default ACL affect
the mode of new files.
* THANKS.in: Remove committer.
Related to http://bugs.gnu.org/8527
* doc/coreutils.texi: `mkfifo' and `mknod' use the optContext macro
which adds a description for the SELinux security context in addition to
the single option already described in each case. The result in both
cases is two options being introduced as `option' (singular). Fix this
by introducing them as `options' (plural).
Conditional jump or move depends on uninitialised value(s)
at 0x40380C: get_field_values (df.c:840)
by 0x403E16: get_dev (df.c:994)
by 0x404D65: get_all_entries (df.c:1364)
by 0x405926: main (df.c:1714)
* src/df.c (get_dev): Initialize the fsu.fsu_bavail_top_bit_set
member, when adding placeholder entries.
(main): Avoid a "definitely lost" memory leak warning from valgrind,
reported by Bernhard Voelker.
* src/ls.c (usage): Add punctuation to avoid ambiguity in the
description of the --time option. Mention that both the -u
and --sort=time default order is newest first.
* src/tail.c (tail_forever_inotify): Only monitor write()s and
truncate()s to files in --follow=descriptor mode, thus avoiding
the bug where we removed the watch on renamed files.
Also adjust the inotify event processing code that is
now significant only in --follow=name mode.
* tests/tail-2/F-vs-rename.sh: Improve this existing test by running
in both polling and inotify modes.
* tests/tail-2/f-vs-rename.sh: A new test based on the existing one.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug.
Fixes http://bugs.gnu.org/19760
* tests/fs/skip-duplicates.sh: On this platform .mnt_opts is significant
so define to empty to avoid a NULL deref in read_file_system_list().
Fixes http://bugs.gnu.org/20210
* src/date.c (usage): Use FMT rather than TIMESPEC as the parameter,
since it's simpler to understand and can be better aligned.
Give an example for the --iso-8601 output format.
Adjust the example used for the 3 standard formats to be unambiguous
with respect to day/mon ordering and use of leading zeros in the time.
Reorder the options descriptions slightly, so that the
3 standards options are together.
Indent the multi-line descriptions so that grouping is obvious.
Remove a redundant description of the --rfc-3339 format,
which is obvious in the existing example.
Separate these 3 standards options to their own translatable string
to simplify translation.
Change 'date and time' to 'date/time' in the --iso-8601 description
to be consistent with --rfc-3339 and to help avoid the implication
that the time is always output or even output by default.
Fixes http://bugs.gnu.org/20203
* README-release: Reference http://www.gnu.org/s/coreutils/manual.css
to apply basic styling to the online coreutils manual, consistent
with the Emacs documentation.
* src/wc.c (wc): Allow any block to select the count implementation,
rather than just using the first 10 lines. This also simplifies
the code from 3 loops to 2.
* cfg.mk (sc_tests_executable): The previous commit avoided
the globbing, but also passed on the quoted wildcards to find(1).
We could use eval to handle the quoting, though that's a bit
awkward and dangerous, so instead explicitly disable globbing
for the whole make target subshell. Note noglob is not available
on solaris, where we fall back to set -f. Note also that zsh
uses set -F for this, but that's moot here. Also correct the
find(1) expression to include the -o between each wildcard.
* cfg.mk (sc_tests_executable): If there are files with
$TEST_EXTENSIONS in the current directory, then the
lack of quoting of the $test_extensions_rx contents
could result in globbing and an inconsequential run.
find(1) produces warnings only with more than one expansion.
Using a test file generated with:
yes | head -n100M > 2x100M.txt
before> time wc -l 2x100M.txt
real 0.842s
user 0.810s
sys 0.033s
after> time wc -l 2x100M.txt
real 0.142s
user 0.111s
sys 0.031s
* src/wc.c (wc): Split the loop that deals with -l into 3.
The first is used at the start of the input to determine if
the average line length is < 15, and if so the second loop is
used to look for '\n' internally to wc. For longer lines,
memchr is used as before to take advantage of system specific
optimizations which any outweigh function call overhead.
Note the first 2 loops could be combined, though in testing,
GCC 4.9.2 at least, wasn't sophisticated enough to separate
the loops based on the "check_len" invariant.
Note also __builtin_memchr() isn't significant here as
GCC currently only applies constant folding with that.
* NEWS: Mention the improvement.
* src/yes.c (main): Even when the internal buffer isn't large enough,
output what we've buffered already, and interate over the rest.
This improves the performance in the edge case where there are
many small arguments that overflow the buffer.
* tests/misc/yes.sh: Add a test case for the many small arguments case.
yes(1) may be used to generate repeating patterns of text
for test inputs etc., so adjust to be more efficient.
Profiling the case where yes(1) is outputting small items
through stdio (which was the default case), shows the overhead
of continuously processing small items in main() and in stdio:
$ yes >/dev/null & perf top -p $!
31.02% yes [.] main
27.36% libc-2.20.so [.] _IO_file_xsputn@@GLIBC_2.2.5
14.51% libc-2.20.so [.] fputs_unlocked
13.50% libc-2.20.so [.] strlen
10.66% libc-2.20.so [.] __GI___mempcpy
1.98% yes [.] fputs_unlocked@plta
Sending more data per stdio call improves the situation,
but still, there is significant stdio overhead due to memory copies,
and the repeated string length checking:
$ yes "`echo {1..1000}`" >/dev/null & perf top -p $!
42.26% libc-2.20.so [.] __GI___mempcpy
17.38% libc-2.20.so [.] strlen
5.21% [kernel] [k] __srcu_read_lock
4.58% [kernel] [k] __srcu_read_unlock
4.27% libc-2.20.so [.] _IO_file_xsputn@@GLIBC_2.2.5
2.50% libc-2.20.so [.] __GI___libc_write
2.45% [kernel] [k] system_call
2.40% [kernel] [k] system_call_after_swapgs
2.27% [kernel] [k] vfs_write
2.09% libc-2.20.so [.] _IO_do_write@@GLIBC_2.2.5
2.01% [kernel] [k] fsnotify
1.95% libc-2.20.so [.] _IO_file_write@@GLIBC_2.2.5
1.44% yes [.] main
We can avoid all stdio overhead by building up the buffer
_once_ and outputting that, and the profile below shows
the bottleneck moved to the kernel:
$ src/yes >/dev/null & perf top -p $!
15.42% [kernel] [k] __srcu_read_lock
12.98% [kernel] [k] __srcu_read_unlock
9.41% libc-2.20.so [.] __GI___libc_write
9.11% [kernel] [k] vfs_write
8.35% [kernel] [k] fsnotify
8.02% [kernel] [k] system_call
5.84% [kernel] [k] system_call_after_swapgs
4.54% [kernel] [k] __fget_light
3.98% [kernel] [k] sys_write
3.65% [kernel] [k] selinux_file_permission
3.44% [kernel] [k] rw_verify_area
2.94% [kernel] [k] __fsnotify_parent
2.76% [kernel] [k] security_file_permission
2.39% yes [.] main
2.17% [kernel] [k] __fdget_pos
2.13% [kernel] [k] sysret_check
0.81% [kernel] [k] write_null
0.36% yes [.] write@plt
Note this change also ensures that yes(1) will only write
complete lines for lines shorter than BUFSIZ.
* src/yes.c (main): Build up a BUFSIZ buffer of lines,
and output that, rather than having stdio process each item.
* tests/misc/yes.sh: Add a new test for various buffer sizes.
* tests/local.mk: Reference the new test.
Fixes http://bugs.gnu.org/20029
In certain parallel build situations this would give the error:
help2man: can't get `--help' info from man/test.td/[
Makefile:14189: recipe for target 'man/test.1' failed
* man/local.mk (test.1): Depend on `[` rather than `test`,
as `test --help` outputs nothing. Also move dir.1 and vdir.1
back to the main list, as they're no more exceptions than
sha1sum etc.
With "umask 0027" or even "umask 0077", the git clone of coreutils
does not have the executable bit set for 'other' (or 'group).
Therefore, "make syntax-check" would fail.
* cfg.mk (sc_tests_executable): Change the -perm argument of find(1)
to only print the names of the files which are not executable by the
user, rather than insisting on ugo+x (octal 111).
Adjust commit v8.23-140-gfdd6ebf to add the --output-error option
instead of --write-error, and treat open() errors like write() errors.
* doc/coreutils.texi (tee invocation): s/write-error/output-error/.
* src/tee.c (main): Exit on open() error if appropriate.
* tests/misc/tee.sh: Add a case to test open() errors.
* NEWS: Adjust for the more general output error behavior.
Suggested by Bernhard Voelker.
Note that IBRIX used to have a different magic number 0x013111A7
instead of the current 0x013111A8. However, the former is no longer
used and the version of IBRIX it was used in is really ancient, so
it's extremely unlikely anyone is still using it. Therefore, just
add the newer magic number.
Mark IBRIX as a 'remote' file system type as inotify support had
never been officially tested with it.
* src/stat.c (human_fstype): Add file system ID definition.
* NEWS: Mention the improvement.
Fixes http://bugs.gnu.org/19951
tee is very often used with pipes and this gives better control
when writing to them. There are 3 classes of file descriptors
that tee can write to: files(1), pipes(2), and early close pipes(3).
Handling write errors to 1 & 2 is supported at present with the caveat
that failure writing to any pipe will terminate tee immediately.
Handling write errors to type 3 is not currently supported.
To improve the supported combinations we add these options:
--write-error=warn
Warn if error writing any output including pipes.
Allows continued writing to still open files/pipes.
Exit status is failure if any output had error.
--write-error=warn-nopipe, -p
Warn if error writing any output except pipes.
Allows continued writing to still open files/pipes.
Exit status is failure if any non pipe output had error.
--write-error=exit
Exit if error writing any output including pipes.
--write-error=exit-nopipe
Exit if error writing any output except pipes.
Use the "nopipe" variants when files are of types 1 and 3, otherwise
use the standard variants with types 1 and 2. A caveat with the above
scheme is that a combination of pipe types (2 & 3) is not supported
robustly. I.e. if you use the "nopipe" variants when using both type
2 and 3 pipes, then any "real" errors on type 2 pipes will not be
diagnosed.
Note also a general issue with type 3 pipes that are not on tee's
stdout, is that shell constructs don't allow to distinguish early
close from real failures. For example `tee >(head -n1) | grep -m1 ..`
can't distinguish between an error or an early close in "head" pipe,
while the fail on the grep part of the pipe is distinguished
independently from the resulting pipe errors. This is a general
issue with the >() construct, rather than with tee itself.
* NEWS: Mention the new feature.
* doc/coreutils.texi (tee invocation): Describe the new option.
* src/tee.c (usage): Likewise.
(main): With --write-error ignore SIGPIPE, and handle
the various exit, diagnostics combinations.
* tests/misc/tee.sh: Tess all the new options.
Fixes http://bugs.gnu.org/11540
This is a cleanup to the previous commit v8.23-138-g7ceaf1d.
* src/tee.c (tee_files): Do not exempt the "-" file from being closed,
as this is no longer stdout but a normal file.
Since v5.2.1-1247-g8dafbe5, tee(1) treated '-' as stdout while POSIX
explicitly requires to treat this as a file name. Revert this change,
as the interleaved output - due to sending another copy of input to
stdout - is not considered to be useful. Discussed in
http://lists.gnu.org/archive/html/coreutils/2015-02/msg00085.html
* src/tee.c (tee_files): Remove the special handling for "-" operands.
(usage): Remove the corresponding sentence.
* doc/coreutils.texi (common options): Remove the "tee -" example.
(tee invocation): Document that tee(1) now treats "-" as a file name.
* tests/misc/tee.sh: Add a test case for "tee -".
While at it, re-indent the above multi-argument processing case and
extend that to 13 operands, as POSIX mandates that, too.
* tests/misc/tee-dash.sh: Remove now-obsolete test.
* tests/local.mk (all_tests): Remove the above test.
* NEWS (Changes in behavior): Mention the change.
* src/tee.c (main): Don't continue reading if we can't
output anywhere.
* tests/misc/tee.sh: Ensure we exit when no more outputs.
* NEWS: Mention the change in behavior.
* tests/init.sh (returns_): Disable tracing for this wrapper
function, so that stderr of the wrapped command is unchanged,
allowing for verification of the contents.
* cfg.mk (sc_prohibit_strncmp): Improve the search pattern: use
_sc_search_regexp to find all invocations of strncmp except when
used on a macro definition line; just match the function name with
an opening parenthesis. Before, the expression missed places where
the comparison against 0 was in a subsequent line.
* src/system.h (STRNCMP_LIT): Shorten 'literal' to 'lit' to move
the whole definition of the macro into one line - thus making
sc_prohibit_strncmp pass.
(STRPREFIX): Add space before parenthesis.
* src/du.c (main): Prefer STREQ_LEN over strncmp.
* src/pinky.c (scan_entries): Likewise.
* src/tac.c (tac_seekable): Likewise.
* src/who.c (scan_entries): Likewise.
This setting is unusual on BSD as it's read normally in the local
flags returned by tcgetattr(), but can only be set with an ioctl.
Setting with tcsetattr() is ignored.
* src/stty.c (NO_SETATTR): A new flag to indicate the setting
is read and displayed like a normal termios flag, but is set
in some other manner.
(main): Skip tcsetattr() for this setting when this flag is set.
Also fixup the exiting 'extproc' processing to handle the
'-extproc' case correctly.
(sane_mode): Skip setting '-extproc' for 'sane' to avoid the error.
This isn't ideal but matches the operation of the BSD native stty.
* .mailmap (jeff.liu@oracle.com): There are 3 different names in the
'git log' output for this email address; choose "Jeff Liu" as canonical
form.
(Алексей Шилин): Convert name to latin1 ("Aleksej Shilin")
to improve the sort order of the generated 'THANKS' file.
At least 'sort' on openSUSE/Fedora have a bug in the case-folding code
of their I18N downstream patch which leads to wrong sort results,
e.g. "Dániel" coming after "Dylan".
* Makefile.am (THANKS): Sort the final contributor list using our
own sort implementation - as others may result in a different order;
add a FIXME comment to remove this again once common platforms have
a functional 'sort -f'. Add '-k1,1' for a better sort order.
While at it, save a grep and perl call to prepare the list from
'THANKS.in' by doing all in the first perl call.
This includes a change to require --with-libmount
to be used with configure, due to the many libmount dependencies.
* bootstrap: Sync with gnulib to exit early on gnulib-tool error.
* gl/lib/tempname.c.diff: Adjust for gnulib changes.
* gl/lib/tempname.h.diff: Likewise.
* gl/modules/tempname: Likewise.
* doc/.gitignore: Add new gendocs_template_min gnulib script.
* cfg.mk: Add .diff files to the exclusion expression
for sc_long_lines, since the gnulib code might be >= 80 chars.
Note 80 char lines trigger due to the added +/- diff marks.
Also normalize the $$ used in the other sc_long_lines exclusion
expressions.
* src/ls.c (align_nstrftime): Be defensive and validate the tm_mon
index before using to access the abmon array. This was _not_ seen
to be an issue any system. See https://bugzilla.redhat.com/1190454
* doc/coreutils.texi (Random sources): Give an example using openssl,
generating a reproducible arbitrary amount of randomly distributed
data, given a seed value.
The construct "diff ... || diff=1 || diff=" does not set the variable
in all cases. This could be triggered with:
$ env make diff=1 make sc_dd_O_FLAGS
dd_O_FLAGS
maint.mk: ./src/dd.c has inconsistent O_ flag lists
cfg.mk:59: recipe for target 'sc_dd_O_FLAGS' failed
make: *** [sc_dd_O_FLAGS] Error 1
* cfg.mk (sc_dd_O_FLAGS): Remember $? of the diff command directly
and check its value later rather than using the above mentioned
mapping.
The previous commit v8.23-124-g7b1ca5f made the above syntax-check rule
fail, because that took the whole content of THANKS.in for comparison.
* cfg.mk (sc_THANKS_in_duplicates): Strip off the header (all before the
first empty line) and the footer (all past the next empty line) from
'THANKS.in' for the check.
* THANKS.in: Document the preferred sort order as a comment
at the top of the file. Change "Марк Коренберг" to latin1
("Mark Korenberg"). Sort all entries.
* cfg.mk (sc_THANKS_in_sorted): Add rule to ensure that
'THANKS.in' remains sorted.
Each user has a maximum number of inotify watches,
so handle the cases where we exhaust these resources.
* src/tail.c (tail_forever_inotify): Ensure we inotify_rm_watch()
the watch for an inode, when replacing with a new watch for a name.
Return all used inotify resources when reverting to polling.
Revert to polling upon first indication of inotify resource exhaustion.
Revert to polling on any inotify resource exhaustion.
Diagnose resource exhaustion correctly in all cases.
Avoid redundant reinsertion in the hash for unchanged watches
(where only attributes of the file are changed).
* tests/tail-2/retry.sh: Avoid false failure when reverting to polling.
* tests/tail-2/inotify-rotate.sh: Likewise.
* tests/tail-2/symlink.sh: Likewise.
* tests/tail-2/inotify-rotate-resources.sh: New test to check
that we're calling inotify_rm_watch() for replaced files.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
* THANKS.in: Thanks for reporting and problem identification.
* tests/tail-2/inotify-rotate.sh (cleanup_fail_): Set fail=1
so that failures are identified. Regression in v8.23-63-g111a2b9
Also use print_ver_ rather than open coding --verbose support.
Also check for more than a single 'b' which seems brittle.
The -fsanitize=address run associated with v8.22-75-gf940fec
failed to check make-prime-list, as src/primes.h is not
regenerated with `make clean`. Running with -fsanitize=address
indicates a read 1 byte beyond the allocated buffer.
$ rm src/make-prime-list.o
$ make AM_CFLAGS=-fsanitize=address src/make-prime-list
$ src/make-prime-list 5000
=================================================================
==13913==ERROR: AddressSanitizer: heap-buffer-overflow on address
0x61e00000fa43 at pc 0x4016f5 bp 0x7fff9d9840e0 sp 0x7fff9d9840d0
READ of size 1 at 0x61e00000fa43 thread T0
#0 0x4016f4 in main src/make-prime-list.c:214
#1 0x7f98892c5fdf in __libc_start_main (/lib64/libc.so.6+0x1ffdf)
#2 0x401774 (src/make-prime-list+0x401774)
0x61e00000fa43 is located 0 bytes to the right of 2499-byte
region [0x61e00000f080,0x61e00000fa43) allocated by thread T0 here:
#0 0x7f98896ba7b7 in malloc (/lib64/libasan.so.1+0x577b7)
#1 0x400f3f in xalloc src/make-prime-list.c:163
#2 0x400f3f in main src/make-prime-list.c:198
SUMMARY: AddressSanitizer: heap-buffer-overflow
src/make-prime-list.c:214 main
Shadow bytes around the buggy address:
0x0c3c7fff9ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c3c7fff9f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c3c7fff9f10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c3c7fff9f20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c3c7fff9f30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c3c7fff9f40: 00 00 00 00 00 00 00 00[03]fa fa fa fa fa fa fa
0x0c3c7fff9f50: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c3c7fff9f60: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c3c7fff9f70: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c3c7fff9f80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c3c7fff9f90: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
...
==13913==ABORTING
* src/make-prime-list.c (main): Bounds check the incremented index,
before using to access the buffer.
Fixes http://bugs.gnu.org/19784
* gl/lib/tempname.c.diff: Fix recent breakage so it applies again.
Invalid patch was noticed at http://hydra.nixos.org/eval/1172233
* cfg.mk: Exempt diff files from these "id_est" syntax checks.
(sc_ensure_gl_diffs_apply): A new syntax check, to ensure all
patches under gl/ apply cleanly. Note we use --fuzz=0 to check
patches apply cleanly for safety, due to the patch(1) issue detailed
in commit v8.21-117-g46f7e05
* gl/lib/regcomp.c.diff: Rediffed.
* gl/lib/regex_internal.c.diff: Likewise.
* gl/lib/regex_internal.h.diff: Likewise.
* gl/lib/regexec.c.diff: Likewise.
* gl/lib/tempname.h.diff: Likewise.
The following test fails on aarch64 on openSUSE's OpenBuildService
due to glibc's execvp reversing the pointers of 'environ', i.e.,
the output of "env|tac" equals "env env" on that platform.
* tests/misc/printenv.sh: Use 'env env' to work around the behavior
on that platform.
While at it, fix the grep pattern which suppressed all environment
variables starting with an underscore "_" instead of "$_" (and
"$LD_PRELOAD") only.
* src/md5sum.c (usage): Detail the reasons for the default
double space between checksum and file name.
* doc/coreutils.texi (md5sum invocation): Likewise.
Explicitly mention the 3 formats that --check supports.
Fixes http://bugs.gnu.org/19725
* m4/jm-macros.m4 (coreutils_MACROS): Check for syncfs().
* man/sync.x: Add references to syncfs, fsync and fdatasync.
* doc/coreutils.texi (sync invocation): Document the new feature.
* src/sync.c: Include "quote.h".
(AUTHORS): Include myself.
(MODE_FILE, MODE_DATA, MODE_FILE_SYSTEM, MODE_SYNC): New enum values.
(long_options): Define.
(sync_arg): New function.
(usage): Describe that arguments are now accepted.
(main): Add arguments parsing and add support for fsync(2),
fdatasync(2) and syncfs(2).
* tests/misc/sync.sh: New (and only) test for sync.
* tests/local.mk: Reference the new test.
* AUTHORS: Add myself to sync's authors.
* NEWS: Mention the new feature.
* scripts/autotools-install: Increase automake's version number
to 1.15 and add Stefano Lattarini's new GPG key ID.
Increase gettext's version to 0.19.4 and add Daiki Ueno's GPG key ID.
Also move VERSION definition "up" so that it is once again
automatically updated via the emacs hook snippet at the end
of the file.
* src/stty.c (usage): Don't reference unsupported options,
in the combined options descriptions.
* doc/coreutils.texi (stty invocation): Adjust for the
new order of the 'sane' and 'raw' combined options.
Also add -iutf8 to the 'sane' list.
* src/stty.c (usage): On systems that support this setting (BSD),
display 'status' in the list of adjustable special characters.
* doc/coreutils.texi (stty invocation): Mention the option, and that
it's not currently supported on Linux.
The equivalent of this is 'flush', but that was never documented
as an option (though was output with stty -a). Therefore use
the more descriptive name, also generally used on BSD systems.
Note even though this setting seems ineffective on Linux, supporting
the setting is useful to allow terminal programs to receive
the default ^O character code.
* doc/coreutils.texi (stty invocation): Document the 'discard' option.
* src/stty.c (struct control_info): Add 'discard'; same as 'flush'.
(display_all): Show 'discard' rather than 'flush' char.
(display_changed): Likewise.
(usage): Document the 'discard' option.
Add support for the "extproc" option which is well described at:
http://lists.gnu.org/archive/html/bug-readline/2011-01/msg00004.html
* src/stty.c (usage): Describe the extproc option if either the
Linux EXTPROC local option is defined, or the equivalent
BSD TIOCEXT ioctl is defined.
(main): Make the separate ioctl call for extproc on BSD.
* doc/coreutils.texi (stty invocation): Describe the option,
and reference the related RFC 1116.
* NEWS: Mention the new feature.
* tests/split/record-sep.sh: Use the recently added returns_ function
to simplify the shell syntax in this test. Also remove the redirection
of stdout/stderr to /dev/null as this eases analyzing errors.
* src/split.c (eolchar): A new variable to hold
the separator character (unibyte for now).
This is reference throughout rather than hardcoding '\n'.
(usage): Describe the new --separator option, and
mention records along with lines so there is no ambiguity
that all options treat lines and records equivalently.
(main): Have -t update eolchar, or default to '\n'.
* tests/split/record-sep.sh: New test case.
* tests/local.mk: Reference the new test.
* doc/coreutils.texi (split invocation): Document the new option.
Adjust --lines, --line-bytes, --number=[lr]/... to mention
they pertain to records if --separator is specified.
* NEWS: Mention the new feature.
Following on from http://bugs.gnu.org/17546
make it more obvious that du may elide specified operands
to avoid double counting in the set.
* src/du.c (usage): Specify that du operates on the set of
operands, rather than each independently.
* doc/coreutils.texi (du invocation): Likewise. Also state
that the number of entries printed may change due to the
order specified. Currently, deeper items specified earlier
will result in them being displayed, but don't mention that
implementation detail in the documentation.
* THANKS.in: Add reporter.
Reported by Stephen Shirley
When some program produces unexpected output, that use of
compare-vs-/dev/null will ensure that the surprising output is
printed in the test's output. With "test -s err" only, one
would have to instrument and rerun in order to see the offending
output.
* cfg.mk (sc_prohibit_and_fail_1): Exempt 'compare' from this check.
* tests/dd/misc.sh: Change "tests -s ... || fail=1" to
"compare /dev/null ... && fail=1".
* tests/misc/nice.sh: Likewise.
* tests/rm/read-only.sh: Likewise.
* tests/tail-2/inotify-race.sh: Likewise.
* tests/touch/no-dereference.sh: Likewise.
Suggested by Jim Meyering in
http://lists.gnu.org/archive/html/coreutils/2015-01/msg00042.html
Many tests use `program ... && fail=1` to ensure expected
error situations are indicated. However that would mask
an unexpected exit (like a crash). Therefore explicitly
check the expected exit code.
Note where error messages are also verified, the extra
protection is not added.
* tests/init.sh (returns_): A new helper function to
check the return code of a command, and used
throughout the tests.
* cfg.mk (sc_prohibit_and_fail_1): Add a syntax check
to avoid new instances of this issue.
* tests/df/no-mtab-status.sh: Provide libmount placeholders,
to avoid skipping the test when libmount is in use.
* tests/df/skip-duplicates.sh: Likewise.
* tests/df/skip-rootfs.sh: Comment that the test is moot
when libmount (/proc/self/mountinfo) is being used.
Problem reported by Daiki Ueno in: http://bugs.gnu.org/19520
* src/shuf.c (main): Avoid core dump if !input_range.
* tests/misc/shuf.sh: Test for this bug.
* bootstrap: Update copyright year manually (missing in previous
gnulib update).
* tests/init.sh: Likewise.
The entries in the exemption list are processed by
"grep -vEf ./.x-update-copyright", and therefore evaluated as an
extended regular expression (ERE). Thus, the "bootstrap" entry
also matched for bootstrap.conf which we want to be updated.
* .x-update-copyright: Change all entries to EREs, i.e. including
the caret ^ and dollar sign $ meta-characters matching the beginning
and the end of a line.
* bootstrap.conf: Update copyright year by "make update-copyright".
Finally, the only one showing up with the following command should
be the COPYING file:
$ git grep 'Copyright .* Free Software' | grep -v '2015 Free Software'
* src/split.c (usage): Indent the info on CHUNKS so that
help2man can match it and align appropriately in its own section.
Fixes http://bugs.gnu.org/19228
Pick up an errno adjustment in xstrtol() that fixes
a spurious test failure on Darwin 14.0.0.
Also update copyright year to 2015 avoiding a syntax-check failure.
Run "make update-copyright" and then...
* tests/sample-test: Adjust to use the single most recent year.
* tests/du/bind-mount-dir-cycle-v2.sh: Fix case in copyright message,
so that year is updated automatically in future.
Commit v8.23-63-g111a2b9 removed the expensive tag on this test,
as it runs quickly on systems with inotify. However without that
it would take about 8 minutes for the test to complete all iterations.
* tests/tail-2/inotify-rotate.sh: Tag as expensive without inotify.
Also adjust the polling parameters used on systems without inotify
so that the test completes within about 15 seconds.
Included in this are gnulib changes 3ea43e02 2768ceb7
which make the device IDs from /proc/self/mountinfo
available to df. This can be leveraged by a subsequent
change to df to present a more accurate list of file systems.
* bootstrap: Merge from gnulib.
* src/ls.c (dev_ino_pop): s/obstack_blank/obstack_blank_fast/
as this API/ABI has changed, giving memory exhausted errors
if negative (large positive) numbers are passed to obstack_blank().
* tests/df/skip-duplicates.sh: Adjust as the new gnulib code
requires a non NULL mnt_opts even when mnt_type is not "none".
http://hydra.nixos.org/build/18129583 identified (on OS X)
an incorrect test assumption in the previous commit.
* gl/lib/xdectoint.c (__xnumtoint): Suppress the EINVAL
error message as it's redundant in this context.
* tests/misc/tail.pl: Suppress _optionally_ appended
strerror messages.
* tests/fmt/base.pl: Likewise.
* tests/pr/pr-tests.pl: Likewise.
* tests/split/l-chunk.sh: Likewise.
Following on from commit v8.23-82-gaddae94, consistently diagnose
numbers that are too large, so as to distinguish from other errors,
and make the limits obvious.
* gl/modules/xdectoint: A new module implementing xdecto[iu]max(),
which handles the common case of parsing a bounded integer and
exiting with a diagnostic on error.
* gl/lib/xdectoimax.c: The signed variant.
* gl/lib/xdectoint.c: The parameterized implementation.
* gl/lib/xdectoint.h: The interface.
* gl/lib/xdectoumax.c: The unsigned variant.
* bootstrap.conf: Reference the new module.
* cfg.mk (exclude_file_name_regexp--sc_require_config_h_first):
Exclude the parameterized templates.
* src/csplit.c: Output EOVERFLOW or ERANGE errors if appropriate.
* src/fmt.c: Likewise.
* src/fold.c: Likewise.
* src/head.c: Likewise.
* src/ls.c: Likewise.
* src/nl.c: Likewise.
* src/nproc.c: Likewise.
* src/shred.c: Likewise.
* src/shuf.c: Likewise.
* src/stdbuf.c: Likewise.
* src/stty.c: Likewise.
* src/tail.c: Likewise.
* src/truncate.c: Likewise.
* src/split.c: Likewise.
* src/pr.c: Likewise.
* tests/pr/pr-tests.pl: Adjust to avoid matching errno diagnostic.
* tests/fmt/base.pl: Likewise.
* tests/split/l-chunk.sh: Likewise.
* tests/misc/shred-negative.sh: Likewise.
* tests/misc/tail.pl: Likewise. Also remove the redundant
existing ERR_SUBST from test err-6.
* tests/ls/hex-option.sh: Check HEX/OCT options.
* tests/misc/shred-size.sh: Likewise.
* tests/misc/stty-row-col.sh: Likewise.
OS/2 traditional shells(cmd) do not expand a response file(@file)
or a wildcard. Expand them in each utility itself.
* src/system.h (initialize_main): Define on OS/2. Expand a response
file and a wildcard.
* THANKS.in: Change the comment at the top to send change requests
regarding this file to the main mailing list rather than referring
to cp's --help output for the mailing list's address - which does
not include that information anymore.
glibc <= 2.5 would crash when passed invalid long double values,
therefore internal gnulib routines were used, essentially only by od,
to output such invalid values. Later glibc versions don't crash,
as per https://sourceware.org/bugzilla/show_bug.cgi?id=4586
and subsequently od was adjusted to use the system printf routines
through the use of the ftoastr module with commit v8.7-22-ga71c22f.
Consequently our testing of this feature was moot, and use of
the gnulib printf replacement for printf(1), od(1) and error(3) etc.
was redundant.
* configure.ac (gl_printf_safe): Unset so that we don't check that
"nan" is output for these long double values.
* tests/misc/od-float.sh: Adjust all existing checks to fail if od
exits with failure status (like crashing for example). Add a new case
for one of the problematic invalid long double values for x86_64.
We only check that od exits successfully at present, which may change
if https://sourceware.org/bugzilla/show_bug.cgi?id=17661 is resolved.
* gl/lib/rand-isaac.c (isaac_refill): readisaac() purposefully passes
unaligned pointers to avoid memory copies. This is only done on
platforms where this is defined, so avoid the associated
runtime warning generated with -fsanitize=undefined, which is:
lib/rand-isaac.c:125:182: runtime error: store to misaligned address
0x63100003d7fd for type 'isaac_word', which requires 8 byte alignment
0x63100003d7fd: note: pointer points here
47 ce ed a4 be be be 00 00 00 00 00 00 00 00 ...
^
Prompted by the implicit -O3 added by american-fuzzy-lop,
seen with GCC 4.9.2 on x86_64.
src/pr.c: In function 'print_files.part.5':
src/pr.c:1781:6: error: assuming signed overflow does not occur
when simplifying conditional to constant [-Werror=strict-overflow]
if (cols_ready_to_print () == 0)
This happens because cols_ready_to_print() is inlined
thus reducing the comparison to the N variable in print_page().
Now this can't overflow due to the protection when parsing the
specified column, but use an unsigned type to avoid the
apparent signed overflow.
* src/pr.c (cols_ready_to_print): Increment an unsigned type to
avoid the subsequent signed overflow warning.
This patch fixes the handling of sub-bind-mount cycles which are
incorrectly detected as the file system errors. If you bind mount the
directory 'a' to its subdirectory 'a/b/c' and then run 'du a/b' you
will get the circular dependency warning even though nothing is wrong
with the file system. This happens because the first directory that is
traversed twice in this case is not a bind mount but a child of bind
mount. The solution is to traverse all the directories in the cycle
that fts detected and check whether they are not a (bind) mount.
* src/du.c (mount_point_in_fts_cycle): New function that checks whether
any of the directories in the cycle that fts detected is a mount point.
* src/du.c (process_file): Update the function to use the new function
that looks up all the directories in the fts cycle instead of only the
last one.
* tests/du/bind-mount-dir-cycle-v2.sh: New test case that exhibits the
described behavior.
* tests/local.mk: Reference the new root test.
* NEWS: Mention the bug fix.
These checks weren't correctly avoided in commit v8.23-66-g222d7ac
* tests/cp/same-file.sh: Avoid all hardlink to symlink tests
on platforms where that's not supported.
Identified by http://hydra.nixos.org/build/17636446
"zu" was output on solaris 8 for example rather than the number,
since coreutils-8.22.
* cfg.mk: Disallow %z, since we don't currently use the gnulib
fprintf module, so any usage with it is non portable. Also
our usage with error() currently works only through an ancillary
dependency on the vfprintf gnulib module.
* src/rm.c (main): Use %PRIuMAX rather than %zu for portability.
* src/dd.c (alloc_[io]buf): Likewise for consistency.
* src/od.c (main): Likewise.
* src/split.c (set_suffix_length): Likewise.
* NEWS: Mention the rm bug fix.
Reported in http://bugs.gnu.org/19184
Solaris 8 was seen to issue this error:
"printf: `&': illegal format character"
* test/dd/ascii.sh: Use the coreutils printf in this test
rather than the system one, to avoid portability issues.
If '\n' was present at the size_t boundary of a file,
then that and subsequent data would be discarded.
* src/paste.c (paste_parallel): Avoid the overflow issue
by changing the flag to a boolean rather than a count.
* NEWS: Mention the bug fix.
* src/df.c (filter_mount_list): Separate remote locations are
generally explicitly mounted, so list each even if they share
the same remote device and thus storage. However with --total
keep the suppression to give a more accurate value for the
total storage available.
(usage): Expand on the new implications of --total and move
it in the options list according to alphabetic order.
doc/coreutils.texi (df invocation): Mention that --total impacts
on deduplication of remote file systems and also move location
according to alphabetic order.
* tests/df/skip-duplicates.sh: Add remote test cases.
* NEWS: Mention the change in behavior.
Reported in http://bugs.debian.org/737399
Reported in http://bugzilla.redhat.com/920806
Reported in http://bugzilla.opensuse.org/866010
Reported in http://bugzilla.opensuse.org/901905
commit v8.22-125-g9d736f8 printed placeholder "-" values
for device names that didn't match the preferred device name
for a particular mount point. However that was seen to erroneously
suppress values for aliased host names or exports, common with
remote file systems.
* src/df.c (me_for_dev): Rename from devname_for_dev() so that
we can determine the remoteness as well as the name for the
preferred mount entry.
(get_dev): Don't output place holder values when both
current and preferred mount entries are remote.
Reported in http://bugs.debian.org/737399
* NEWS: Update the recent entry to also mention the avoidance
of incorrectly unlinking a multi-hardlinked "source" file when
presented with source and dest that only differ in case.
* src/copy.c (same_file_ok): Mention the case issue with same_name().
* tests/mv/hardlink-case.sh: Test the issue on HFS+.
* tests/local.mk: Reference the new test case.
* tests/mv/vfat: Remove an old related but unused test case.
file_t is now mapped to unlabeled_t as per:
http://danwalsh.livejournal.com/68189.html
Therefore use the latter to ensure we match correctly.
This is needed on >= Fedora 21 for example,
while it also works on earlier releases.
We may run into a race condition if we treat hard links to the same file
as distinct files. If we do 'mv a b' and 'mv b a' in parallel, both a
and b can disappear from the file system. The reason is that in this
case the unlink on src is called and the system calls can end up being
run in the order where unlink(a) and unlink(b) are the last two system
calls. Therefore exit with an error code so that we avoid the potential
data loss.
* src/copy.c (same_file_ok): Don't set unlink_src that was used by mv,
and return false for two hardlinks to a file in move_mode.
*src/copy.c (copy_internal): No longer honor the unlink_src option,
used only by mv.
NEWS: Mention the change in behavior.
* tests/cp/same-file.sh: Augment to cover the `cp -a hlsl1 sl1` case.
* tests/mv/hard-verbose.sh: Remove no longer needed test.
* tests/local.mk: Remove the reference to hard-verbose.sh.
* tests/mv/hard-4.sh: Adjust so we fail in this case.
* tests/mv/i-4.sh: Likewise.
* tests/mv/symlink-onto-hardlink-to-self.sh: Likewise.
* tests/tail-2/inotify-rotate.sh: Use retry_delay_
to employ an exponential backoff with a total delay of
up to 25.5s. The 15s delay was seen to trigger a false
failure in http://hydra.nixos.org/build/16546517
Also remove the .1s sleep in each of the 50 iterations
to reduce the running time of the test and thus the
expensive_ tag on this test was removed.
Also ensure that we use the standard exit procedure
upon failure to avoid any erroneous diagnostics due
to persistent files on NFS.
* src/df.c (usage): Mention that duplicate file systems are shown
with this option, not just dummy file systems.
* doc/coreutils.texi (df invocation): For the --all option, expand
on the class of normally suppressed mount entries that it includes.
Reported in http://bugs.debian.org/737399
sc_long_lines was the slowest syntax check
before$ time make sc_long_lines
long_lines
real 0m2.740s
after $ time make sc_long_lines
long_lines
real 0m0.677s
* src/cfg.mk (sc_dd_max_sym_length): s/--max-line-length/-L/
for compat with BSDs.
(sc_long_lines): Prefilter with wc -L to only identify lines
in files that have lines longer than 80 characters.
* tests/d_type-check: The hardcoded name doesn't hold true for all
Linux/glibc platforms, let alone Linux/non-glibc.
Use ctypes.util.find_library() instead to search for the library.
At least the MHz number in /proc/cpuinfo may change, thus leading to
a false positive failure when comparing the expected against the
actual output file. Use an invariant file instead: /proc/version.
* tests/misc/head-c.sh: s/cpuinfo/version/
* src/chroot.c (is_root): Adjust to compare canonicalized paths
rather than inodes, to handle (return false in) the case where
we have a tree that is constructed by first bind mounting "/"
(thus having the same inode).
(main): Unconditionally call chroot() because it's safer
and of minimal performance benefit to avoid in this case.
This will cause inconsistency with some platforms
not allowing `chroot / true` for non root users.
* tests/misc/chroot-fail.sh: Adjust appropriately.
* NEWS: Mention the bug fixes.
Fixes http://bugs.gnu.org/18736
With --sparse=always use fallocate(...PUNCH_HOLE...) to
avoid any permanent allocation due to speculative
preallocation employed by file systems such as XFS.
* m4/jm-macros.m4: Check for <linux/falloc.h> and fallocate().
* src/copy.c (punch_hole): A new function to try and punch
a hole at the specified offset if supported.
(create_hole): Call punch_hole() after requesting a hole.
(extent_copy): Likewise.
* NEWS: Mention the improvement.
Previously cp would not detect runs of NULs that were
smaller than the buffer size used for I/O (currently 128KiB).
* src/copy.c (copy_reg): Use an independent hole_size, set to
st_blksize, to increase the chances of detecting a representable hole,
in a run of NULs read from the input.
(create_hole): A new function refactored from sparse_copy() and
extent_copy() so we have a single place to handle holes.
(sparse_copy): Adjust to loop over the larger input buffer
in chunks of the passed hole size. Also adjust to only call
lseek once per hole, rather than at least once per input buffer.
* tests/cp/sparse.sh: Add test cases for various sparse chunk sizes.
* NEWS: Mention the improvement.
Prompted by http://hydra.nixos.org/build/15682577
with GCC 4.8.3 on i686
src/tac.c:557:6: error: assuming signed overflow does not occur
when simplifying conditional to constant [-Werror=strict-overflow]
if (bytes_copied < 0)
This happens because copy_to_temp() is inlined in tac_nonseekable(),
thus reducing the comparison to the bytes_copied variable in
copy_to_temp. Now this can't overflow on either 32 or 64 bit
due to the protection of the preceding fwrite(). We could use a
guard like "if (bytes_copied <= OFF_T_MAX - bytes_read)" to avoid
the warning, but rather than a runtime branch, just use an unsigned
type to avoid apparent signed overflow on systems where the accumulation
is not promoted to unsigned (32 bit size_t, 64 bit off_t).
* src/tac.c (copy_to_temp): Increment an unsigned type to
avoid the subsequent signed overflow warning.
Fix similar problems in head, od, split, tac, and tail.
Reported by George Shuklin in: http://bugs.gnu.org/18621
* NEWS: Document this.
* src/head.c (elseek): Move up.
(elide_tail_bytes_pipe, elide_tail_lines_pipe): New arg
CURRENT_POS. All uses changed.
(elide_tail_bytes_file, elide_tail_lines_file):
New arg ST and remove arg SIZE. All uses changed.
* src/head.c (elide_tail_bytes_file):
* src/od.c (skip): Avoid optimization for /sys files, where
st_size is bogus and st_size == st_blksize.
Don't report error at EOF when not optimizing.
* src/head.c, src/od.c, src/tail.c: Include "stat-size.h".
* src/split.c (input_file_size): New function.
(bytes_split, lines_chunk_split, bytes_chunk_extract): New arg
INITIAL_READ. All uses changed. Use it to double-check st_size.
* src/tac.c (tac_seekable): New arg FILE_POS. All uses changed.
(copy_to_temp): Return size of temp file. All uses changed.
* src/tac.c (tac_seekable):
* src/tail.c (tail_bytes):
* src/wc.c (wc):
Don't trust st_size; double-check by reading.
* src/wc.c (wc): New arg CURRENT_POS. All uses changed.
* tests/local.mk (all_tests): Add tests/misc/wc-proc.sh,
tests/misc/od-j.sh, tests/tail-2/tail-c.sh.
* tests/misc/head-c.sh:
* tests/misc/tac-2-nonseekable.sh:
* tests/split/b-chunk.sh:
Add tests for problems with /proc and /sys files.
* tests/misc/od-j.sh, tests/misc/wc-proc.sh, tests/tail-2/tail-c.sh:
New files.
* doc/coreutils.texi (stat invocation): Add a paragraph documenting
stat's output format when the --terse option is specified, both in
normal and in --file-system mode.
Reported by Dan Jacobson <jidanni@jidanni.org>
in http://bugs.gnu.org/18624
* init.cfg (gcc_shared_): -ldl has to be positioned after the object
files that may rely upon it. This fixes tests/cp/nfs-removal-race.sh
which references dlsym() from libdl.
* src/dd.c: Report the transfer progress every second when the
new status=progress level is used. Adjust the handling and
description of the status= option so that they're treated as
mutually exclusive levels, rather than flags with implicit precedence.
* doc/coreutils.texi (dd invocation): Document the new progress
status level. Reference the new level in the description of SIGUSR1.
* tests/dd/stats.sh: Add new test for status=progress.
* tests/dd/misc.sh: Change so status=none only takes precedence
if it's the last level specified.
* NEWS: Mention the feature.
* src/dd.c (ifd_reopen): A new wrapper to ensure we
don't exit upon receiving a SIGUSR1 in a blocking open()
on a fifo for example.
(iftruncate): Likewise for ftruncate().
(iread): Process signals also after a short read.
(install_signal_handlers): Install SIGINFO/SIGUSR1 handler
even if set to SIG_IGN, as this is what the parent can easily
set from a shell script that can send SIGUSR1 without the
possiblity of inadvertently killing the dd process.
* doc/coreutils.texi (dd invocation): Improve the example to
show robust usage wrt signal races and short reads.
* tests/dd/stats.sh: A new test for various signal races.
* tests/local.mk: Reference the new test.
* NEWS: Mention the fix.
On some filesystems (BTRFS), moving a file within the filesystem may
cross subvolume boundaries and we can use a lightweight reflink copy,
similar to what cp(1) can do, which is faster than a full file copy.
This is enabled by default because it's only an optimization for
the fall back copy and does not break user expectations or usability.
* src/mv.c (cp_option_init): Set the reflink mode to AUTO.
* NEWS: Mention the improvement.
* src/stty.c (usage): Exclude unsupported options from --help,
which for example impacts the "dsusp" and "cdtrdsr" options on Linux.
Fixes http://bugs.gnu.org/18506
"E.g." stands for latin "exempli gratia" which is typically read
as "for example". "E.g." does not stand for the word "example".
As such, "for e.g." might be read as "for for example".
Fix this usage by simply replacing "e.g." with "example".
* man/local.mk (.x.1): Move the program name argument down after
the last option argument when calling $(run_help2man).
While the other way would be accepted for the GNU help2man program,
it is not for the 'dummy-man' script (called as a fallback on
systems lacking perl).
The wrong order was introduced in commit v8.21-119-gb3578fc while
adding the --info-page option.
* man/dummy-man: Fix argument count check, now only permitting
exactly 1 argument, the program name.
Reported by Andreas Schwab <schwab@linux-m68k.org>
* .gitignore: Remove reference to no longer generated make file.
* configure.ac: Don't bother generating placeholder make file.
* man/local.mk: Hardcode the man page deps list for normal builds
to be compatible with all make implementations and configure options.
Note in SINGLE_BINARY mode, all man pages will be generated on
any change to the coreutils binary, but development will generally
not be done in this mode, so this shouldn't be an issue.
Fixes http://bugs.gnu.org/18055
Following on from commit v5.92-729-g130dd06, also avoid
the erroneous directory hardlink warning with -H.
* src/copy.c (copy_internal): Also handle the -H case
for command line arguments.
* tests/cp/duplicate-sources.sh: Augment the test case.
* NEWS: Augment the news entry.
* src/copy.c (copy_internal): Handle the case where we have the
same destination directory as already encountered, which can only
be due to the corresponding source directory being specified multiple
times.
* tests/cp/duplicate-sources.sh: Add a test for the new multiply
specified directory case, and the existing multiply specified file case.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
* src/system.h (emit_ancillary_info): Take the invariant PROGRAM_NAME
as a parameter, so that consistent references are made to online docs
and texinfo nodes, when a --program-prefix is in place. Note the
man pages don't need this fix as they're generated before the program
prefix is used.
* NEWS: Mention the improvements in references to online documentation.
* src/system.h (emit_ancillary_info): For commands that don't have
a 1:1 mapping with the texinfo node names, provide a mapping to
the correct node.
* doc/coreutils.texi: Add some extra cross references noticed while
checking this.
Fixes http://bugs.debian.org/762092
On XFS, when creating the ~2G test file 'big' in a for-loop by
appending 20M each time, the file ends up using ~4G - visible in
'st_blocks'. The unused space would be reclaimed later.
This feature is called "speculative preallocation" which aims at
avoiding fragmentation.
According to the XFS FAQ [1], there are two particular aspects of
XFS speculative preallocation that are triggering this:
1. "Applications that repeatedly trigger preallocation and reclaim
cycles [after file close] can cause fragmentation.
Therefore, this pattern is detected and causes the preallocation
to persist beyond the lifecycle of the file descriptor."
2. "Preallocation sizes grow as files grow larger."
[1] http://xfs.org/index.php/XFS_FAQ
Avoid one of the above by only doing a single close (reclaim cycle).
* tests/du/2g.sh: Similar to the fix for a dd test (see commit
v8.22-65-g7c03fe2), avoid speculative preallocation by creating
the 'big' file in one go instead of appending to it in the loop.
Remove debugging statements as the output with 'set -x' is
sufficient nowadays.
Problem reported by Vincent Lefevre in: http://bugs.gnu.org/18449
* src/cat.c (main): Allow copying an empty file to itself.
* tests/misc/cat-self.sh: New test.
* tests/local.mk (all_tests): Add it.
* src/system.h (emit_ancillary_info): Add a direct reference
to the corresponding online info documentation. Corresponding
redirects were put in place on www.gnu.org to allow for concise links.
* help2man: Adjust to add the "online help" link (and subsequent
translation bugs link) to a "REPORTING BUGS" section.
Also add the concise links for further information in --help
to the "SEE ALSO" section, and dispense with the more verbose
default for that.
old form: coreutils '$cmd invocation'
new form: '(coreutils) $cmd invocation'
The old form erroneously referenced the node for the 'coreutils'
multi-call program. Now that problematic node name was renamed
in commit v8.23-18-g72e470b, but the newer less ambiguous form
also has the advantage of working with the pinfo viewer for example.
Full discussion at http://bugs.gnu.org/18428
* man/local.mk: Adjust man page references to texinfo nodes.
* src/system.h: Adjust --help references to texinfo nodes.
* src/local.mk (transform): commit v8.23-22-g6f9b018 discarded all
transformations on the libstdbuf.so name. Be more conservative and
only exclude the $(program_transform_name) portion for libstdbuf.
* src/local.mk (transform): Skip the transformation for libstdbuf
since that should not be subject to name clashes, and we need
to reference the name directly in LD_PRELOAD etc.
* configure.ac: Add a comment on the coupling of pkglibexec_PROGRAMS
to $(transform).
Issue reported at https://trac.macports.org/ticket/44922
Improved by Nick Bowler
The C standard says this isn't portable, if you include
standard include files.
* build-aux/gen-single-binary.sh:
* src/coreutils-arch.c (single_binary_main_arch)
(single_binary_main_uname):
* src/coreutils-dir.c (single_binary_main_ls)
(_single_binary_main_dir):
* src/coreutils-vdir.c (single_binary_main_ls)
(_single_binary_main_vdir):
* src/coreutils.c (SINGLE_BINARY_PROGRAM):
Remove leading _ from single_binary prefix.
* src/numfmt.c (round_style): Rename from _round. All uses changed.
(inval_style): Rename from _invalid. All uses changed.
This supports longstanding shell commands like
'info coreutils "touch invocation"'.
Problem reported by Vincent Lefevre via Bob Proulx in:
http://bugs.gnu.org/18428
* doc/coreutils.texi (Multi-call invocation):
Rename from "coreutils invocation".
* src/extent-scan.c (extent_scan_read): Following on from the flags size
adjustment in commit v8.23-13-g1505b37, verify that the internal
representation of the flags is never truncated which could happen in the
unlikely case on 32 bit if the kernel flags ever expanded to 64 bits
which is theoretically possible given the reserved space.
C11 doesn't require them, even POSIX doesn't strictly require the
64-bit versions, and it makes the code a bit clearer if they're
used only when needed.
* src/copy.c (write_zeros, extent_copy):
* src/extent-scan.h (struct extent_info.ext_length):
Use off_t, not uint64_t, for a value derived from a file offset.
* src/extent-scan.h (struct extent_info.ext_flags)
Prefer plain unsigned int to uint32_t where either will do.
(struct extent_scan.ei_count):
Use size_t, not uint32_t, for a value bounded by SIZE_MAX.
* src/factor.c (MAGIC64, MAGIC63, MAGIC65):
Remove unnecessary casts to uint64_t.
* doc/coreutils.texi (df invocation): Add a sentence that eliding
duplicate entries for the same file system is not limited to bind
mounts, but also happens for remote file systems like NFS.
v8.23 has a test failure on Fedora rawhide build servers
in tests/df/skip-duplicate.sh. This was due to no '/'
entry being output by df. That was due to an inaccurate
/proc/mounts on the build environment as stat(/mnt/point)
identified all these /proc/mounts entries as having the
same device id:
/ rootfs
/ /dev/md1
/dev devtmpfs
/run tmpfs
/boot /dev/md0
/proc/filesystems /dev/md1
Since the device name on the right changes for a given id,
that causes the entries to be continually replaced, thus
resulting in no '/' entry. I'm guessing this is due to
the mock environment bind mounting unneeded or sensitive
items to a dummy file on the host / (/dev/md1) though
have not looked into those details.
So rather than relying on an accurate /proc/mounts,
the attached patch takes a more conservative replacement
approach and only swaps a new device entry when the
mount point matches. That should handle all practical
cases while also avoiding this situation.
* src/df.c (filter_mount_list): Only replace entries with
different device names when the mount point also matches.
If the hash structures grow sufficiently large so that
the system is actively swapping, then the deallocation
can take a significant amount of time. Details at:
http://lists.gnu.org/archive/html/coreutils/2014-08/msg00012.html
* src/cp.c (main): Only call hash deallocation routines
when in lint checking mode.
* THANKS.in: Remove as now in the git author list.
Since commit v8.22-94-g99960ee, chroot(1) skips the chroot(2) syscall
for "/" arguments (and synonyms). The problem is that it also skips
the following chdir("/") call in that case. The latter breaks existing
scripts which expect "/" to be the working directory inside the chroot.
While the first part of the change - i.e., skipping chroot("/") - is
okay for consistency with systems where it might succeed for a non-root
user, the second part might be malicious, e.g.
cd /home/user && chroot '/' bin/foo
In the "best" case, chroot(1) could not execute 'bin/foo' with ENOENT,
but in the worst case, chroot(1) would execute '/home/user/bin/foo' in
the case that exists - instead of '/bin/foo'.
Revert that second part of the patch, i.e., perform the chdir("/)
in the common case again - unless the new --skip-chdir option is
specified. Restrict this new option to the case of "/" arguments.
* src/chroot.c (SKIP_CHDIR): Add enum.
(long_opts): Add entry for the new --skip-chdir option.
(usage): Add --skip-chdir option, and while at it, move the other
to options into alphabetical order.
(main): Accept the above new option, allowing it only in the case
when NEWROOT is the old "/".
Move down the chdir() call after the if-clause to ensure it is
run in any case - unless --skip-chdir is specified.
Add a 'newroot' variable for the new root directory as it is used
in a couple of places now.
* tests/misc/chroot-fail.sh: Invert the last tests which check the
working directory of the execvp()ed program when a "/"-like
argument was passed: now expect it to be "/" - unless --skip-chdir
is given.
* doc/coreutils.texi (chroot invocation): Document the new option.
Document that chroot(1) usually calls chdir("/") unless the new
--skip-chdir option is specified. Sort options.
* NEWS (Changes in behavior): Mention the fix.
(New features): Mention the new option.
* init.cfg (nonroot_has_perm_): Add chroot's new --skip-chdir option.
* tests/cp/preserve-gid.sh (t1): Likewise.
* tests/cp/special-bits.sh: Likewise.
* tests/id/setgid.sh: Likewise.
* tests/misc/truncate-owned-by-other.sh: Likewise.
* tests/mv/sticky-to-xpart.sh: Likewise.
* tests/rm/fail-2eperm.sh: Likewise.
* tests/rm/no-give-up.sh: Likewise.
* tests/touch/now-owned-by-other.sh: Likewise.
Reported by Andreas Schwab in http://bugs.gnu.org/18062
* configure.ac: Don't add stdbuf to the list of programs to build
if EXEEXT is set, as that is not handled in configure.ac for
libstdbuf.so yet (see bin_PRGRAMS handling in configure.ac).
Also the LD_PRELOAD mechanism will need to be adjusted to support
cygwin in any case, so avoid stdbuf completely in this case for now.
Problem reported by Eric Blake.
Problem reported by Sebastian Rasmussen in: http://bugs.gnu.org/18054
* gl/lib/randread.c (randread_error): Don't put multiple string
literals inside _(...), as xgettext doesn't support that.
* src/chroot.c (main): In diagnostics, don't bother to distinguish
between setting the number of supplemental group IDs to a zero or
to a nonzero value, as the underlying system call is the same
either way. This also makes the string easier to translate correctly.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
