Merge remote branch 'origin/next' into next

rm: record timing data
* README-rm-timing-compare: new file * README-rm-timing-2: New file.
2025-09-10 07:59:52 +02:00 · 2009-10-10 09:28:19 -06:00 · 2009-10-10 09:21:36 -06:00 · 2009-10-10 09:14:39 -06:00 · 2009-10-10 09:14:35 -06:00 · 2009-10-10 09:14:17 -06:00
7 changed files with 542 additions and 49 deletions
--- a/2
+++ b/2
--- a/m4/jm-macros.m4
+++ b/m4/jm-macros.m4
@@ -89,9 +89,6 @@ AC_DEFUN([coreutils_MACROS],
    tcgetpgrp \
  )

-  # for cp.c
-  AC_CHECK_FUNCS_ONCE([utimensat])
-
  dnl This can't use AC_REQUIRE; I'm not quite sure why.
  cu_PREREQ_STAT_PROG

--- a/src/README-rm-timing-2
+++ b/src/README-rm-timing-2
@@ -0,0 +1,231 @@
+#!/bin/bash
+# Demonstrate  >5X speed-up in using fts-based rm on ext4
+# Demonstrate 1.5X speed-up in using fts-based rm on reiserfs
+
+set -x
+
+a=$(printf %031d 0)
+b=$(printf %031d 1)
+(mkdir $a \
+   && cd $a \
+   && seq --format=%031g 10000 |xargs touch \
+   && seq --format=d%030g 10000 |xargs mkdir ) || framework_failure
+cp -al $a $b || framework_failure
+mkdir e || framework_failure
+mv $a $b e || framework_failure
+
+cp -a e 1
+cp -a e 2
+
+sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
+env time /p/bin/rm -rf 1
+env time /cu/src/rm -rf 2
+
+cp -a e 1
+cp -a e 2
+
+
+sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
+env time cu/src/rm -rf 2
+env time /p/bin/rm -rf 1
+
+########################################################################
+# on ext4
+# + env time /p/bin/rm -rf 1
+# 0.19user 12.09system 0:14.42elapsed 85%CPU (0avgtext+0avgdata 0maxresident)k
+# 0inputs+0outputs (0major+212minor)pagefaults 0swaps
+# + env time /cu/src/rm -rf 2
+# 0.13user 0.93system 0:01.08elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k
+# 0inputs+0outputs (0major+1235minor)pagefaults 0swaps
+# + cp -a e 1
+# + cp -a e 2
+# + env time /cu/src/rm -rf 2
+# 0.12user 0.94system 0:01.07elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
+# 0inputs+0outputs (0major+1236minor)pagefaults 0swaps
+# + env time /p/bin/rm -rf 1
+# 0.19user 12.08system 0:12.33elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
+# 0inputs+0outputs (0major+212minor)pagefaults 0swaps
+
+# on reiserfs:
+# + env time /p/bin/rm -rf 1
+# 0.30user 4.16system 0:04.49elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
+# 0inputs+0outputs (0major+168minor)pagefaults 0swaps
+# + env time /cu/src/rm -rf 2
+# 0.21user 2.47system 0:02.88elapsed 93%CPU (0avgtext+0avgdata 0maxresident)k
+# 0inputs+80outputs (0major+1199minor)pagefaults 0swaps
+# + cp -a e 1
+# + cp -a e 2
+# + env time /cu/src/rm -rf 2
+# 0.19user 2.56system 0:02.76elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
+# 0inputs+0outputs (0major+1200minor)pagefaults 0swaps
+# + env time /p/bin/rm -rf 1
+# 0.29user 4.17system 0:04.55elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k
+# 0inputs+0outputs (0major+168minor)pagefaults 0swaps
+
+########################################################################
+# on xfs, F11, slow-disk laptop: (shows slight improvement)
+# iou$ bash /cu/src/README-rm-timing-2                                          :
+# ++ printf %031d 0
+# + a=0000000000000000000000000000000
+# ++ printf %031d 1
+# + b=0000000000000000000000000000001
+# + mkdir 0000000000000000000000000000000
+# + cd 0000000000000000000000000000000
+# + xargs touch
+# + seq --format=%031g 10000
+# + seq --format=d%030g 10000
+# + xargs mkdir
+# + cp -al 0000000000000000000000000000000 0000000000000000000000000000001
+# + mkdir e
+# + mv 0000000000000000000000000000000 0000000000000000000000000000001 e
+# + cp -a e 1
+# + cp -a e 2
+# + sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
+# + env time /p/bin/rm -rf 1
+# 1.06user 12.10system 1:41.96elapsed 12%CPU (0avgtext+0avgdata 0maxresident)k
+# 16368inputs+0outputs (1major+175minor)pagefaults 0swaps
+# + env time /cu/src/rm -rf 2
+# 0.57user 5.47system 1:32.69elapsed 6%CPU (0avgtext+0avgdata 0maxresident)k
+# 20112inputs+0outputs (1major+1214minor)pagefaults 0swaps
+# + cp -a e 1
+# + cp -a e 2
+# + sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
+# + env time /cu/src/rm -rf 2
+# 0.54user 5.34system 1:28.43elapsed 6%CPU (0avgtext+0avgdata 0maxresident)k
+# 19032inputs+0outputs (1major+1198minor)pagefaults 0swaps
+# + env time /p/bin/rm -rf 1
+# 0.99user 12.36system 1:37.36elapsed 13%CPU (0avgtext+0avgdata 0maxresident)k
+# 20256inputs+0outputs (1major+174minor)pagefaults 0swaps
+
+########################################################################
+# on btrfs, F11, slow-disk laptop: (shows 7-8X speed-up)
+# iou$ env time bash /cu/src/README-rm-timing-2                                 :
+# ++ printf %031d 0
+# + a=0000000000000000000000000000000
+# ++ printf %031d 1
+# + b=0000000000000000000000000000001
+# + mkdir 0000000000000000000000000000000
+# + cd 0000000000000000000000000000000
+# + seq --format=%031g 10000
+# + xargs touch
+# + seq --format=d%030g 10000
+# + xargs mkdir
+# + cp -al 0000000000000000000000000000000 0000000000000000000000000000001
+# + mkdir e
+# + mv 0000000000000000000000000000000 0000000000000000000000000000001 e
+# + cp -a e 1
+# + cp -a e 2
+# + sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
+# + env time /p/bin/rm -rf 1
+# 0.51user 59.42system 1:09.73elapsed 85%CPU (0avgtext+0avgdata 0maxresident)k
+# 43536inputs+82536outputs (1major+425minor)pagefaults 0swaps
+# + env time /cu/src/rm -rf 2
+# 0.26user 8.01system 0:08.50elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k
+# 808inputs+54984outputs (1major+1199minor)pagefaults 0swaps
+# + cp -a e 1
+# + cp -a e 2
+# + sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
+# 0.26user 8.31system 0:09.17elapsed 93%CPU (0avgtext+0avgdata 0maxresident)k
+# 976inputs+47744outputs (1major+1197minor)pagefaults 0swaps
+# + env time /p/bin/rm -rf 1
+# 0.48user 57.83system 1:04.81elapsed 89%CPU (0avgtext+0avgdata 0maxresident)k
+# 27408inputs+75632outputs (1major+423minor)pagefaults 0swaps
+# ---
+# 5.60user 214.36system 4:14.11elapsed 86%CPU (0avgtext+0avgdata 0maxresident)k
+# 131544inputs+551320outputs (83major+15945minor)pagefaults 0swaps
+
+########################################################################
+# on reiserfs, F11, slow-disk laptop: (shows ~4X speed-up)
+# but notice total test time: just 51s, compared with 4:11 for btrfs above.
+# iou$ env time bash /cu/src/README-rm-timing-2                                 :
+# ++ printf %031d 0
+# + a=0000000000000000000000000000000
+# ++ printf %031d 1
+# + b=0000000000000000000000000000001
+# + mkdir 0000000000000000000000000000000
+# + cd 0000000000000000000000000000000
+# + seq --format=%031g 10000
+# + xargs touch
+# + seq --format=d%030g 10000
+# + xargs mkdir
+# + cp -al 0000000000000000000000000000000 0000000000000000000000000000001
+# + mkdir e
+# + mv 0000000000000000000000000000000 0000000000000000000000000000001 e
+# + cp -a e 1
+# + cp -a e 2
+# + sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
+# + env time /p/bin/rm -rf 1
+# 0.48user 9.79system 0:10.33elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
+# 256inputs+0outputs (1major+191minor)pagefaults 0swaps
+# + env time /cu/src/rm -rf 2
+# 0.25user 2.15system 0:02.50elapsed 96%CPU (0avgtext+0avgdata 0maxresident)k
+# 808inputs+0outputs (1major+1198minor)pagefaults 0swaps
+# + cp -a e 1
+# + cp -a e 2
+# + sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
+# + env time /cu/src/rm -rf 2
+# 0.26user 2.10system 0:02.46elapsed 95%CPU (0avgtext+0avgdata 0maxresident)k
+# 808inputs+0outputs (1major+1198minor)pagefaults 0swaps
+# + env time /p/bin/rm -rf 1
+# 0.44user 9.89system 0:10.41elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
+# 264inputs+0outputs (1major+176minor)pagefaults 0swaps
+# ---
+# 4.48user 42.92system 0:51.14elapsed 92%CPU (0avgtext+0avgdata 0maxresident)k
+# 21680inputs+54232outputs (86major+16962minor)pagefaults 0swaps
+
+########################################################################
+# on ext2, F11, slow-disk laptop: (shows ~4X speed-up)
+# iou$ env time bash /cu/src/README-rm-timing-2                                 :
+# ++ printf %031d 0
+# + a=0000000000000000000000000000000
+# ++ printf %031d 1
+# + b=0000000000000000000000000000001
+# + mkdir 0000000000000000000000000000000
+# + cd 0000000000000000000000000000000
+# + seq --format=%031g 10000
+# + xargs touch
+# + seq --format=d%030g 10000
+# + xargs mkdir
+# + cp -al 0000000000000000000000000000000 0000000000000000000000000000001
+# + mkdir e
+# + mv 0000000000000000000000000000000 0000000000000000000000000000001 e
+# + cp -a e 1
+# + cp -a e 2
+# + sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
+# + env time /p/bin/rm -rf 1
+# 1.02user 15.25system 0:30.76elapsed 52%CPU (0avgtext+0avgdata 0maxresident)k
+# 178888inputs+19144outputs (1major+423minor)pagefaults 0swaps
+# + env time /cu/src/rm -rf 2
+# 0.40user 2.30system 0:05.99elapsed 45%CPU (0avgtext+0avgdata 0maxresident)k
+# 83088inputs+18480outputs (1major+1199minor)pagefaults 0swaps
+# + cp -a e 1
+# + cp -a e 2
+# + sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
+# + env time /cu/src/rm -rf 2
+# 0.38user 2.83system 0:08.18elapsed 39%CPU (0avgtext+0avgdata 0maxresident)k
+# 85848inputs+16024outputs (1major+1199minor)pagefaults 0swaps
+# + env time /p/bin/rm -rf 1
+# 0.84user 15.24system 0:25.28elapsed 63%CPU (0avgtext+0avgdata 0maxresident)k
+# 178656inputs+18728outputs (1major+423minor)pagefaults 0swaps
+# ---
+# 6.73user 263.32system 5:27.69elapsed 82%CPU (0avgtext+0avgdata 0maxresident)k
+# 780296inputs+891712outputs (86major+17155minor)pagefaults 0swaps
+
+########################################################################
+# on tmpfs, F11, slow-disk laptop: (shows ~4X speed-up)
+# + sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
+# + env time /p/bin/rm -rf 1
+# 0.37user 2.47system 0:03.10elapsed 91%CPU (0avgtext+0avgdata 0maxresident)k
+# 256inputs+0outputs (1major+175minor)pagefaults 0swaps
+# + env time /cu/src/rm -rf 2
+# 0.23user 0.41system 0:00.76elapsed 85%CPU (0avgtext+0avgdata 0maxresident)k
+# 520inputs+0outputs (1major+2092minor)pagefaults 0swaps
+# + cp -a e 1
+# + cp -a e 2
+# + sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
+# + env time /cu/src/rm -rf 2
+# 0.24user 0.41system 0:00.73elapsed 89%CPU (0avgtext+0avgdata 0maxresident)k
+# 800inputs+0outputs (1major+2090minor)pagefaults 0swaps
+# + env time /p/bin/rm -rf 1
+# 0.36user 2.47system 0:02.86elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k
+# 256inputs+0outputs (1major+175minor)pagefaults 0swaps
--- a/src/README-rm-timing-compare
+++ b/src/README-rm-timing-compare
@@ -0,0 +1,297 @@
+==================================================
+[SSD: it's a toss-up, when removing a million-file (1000 per dir) tree
+on (sdd-ocz-summit-120G, F11, x86_64, lots of RAM)
+$ z-mktree --root=z --depth=2 --b=1000;env time /p/bin/rm -rf z
+0.50user 13.99system 0:17.83elapsed 81%CPU (0avgtext+0avgdata 0maxresident)k
+0inputs+0outputs (0major+10834minor)pagefaults 0swaps
+$ z-mktree --root=z --depth=2 --b=1000;env time src/rm -rf z
+0.52user 13.88system 0:17.41elapsed 82%CPU (0avgtext+0avgdata 0maxresident)k
+0inputs+8outputs (0major+10835minor)pagefaults 0swaps
+
+==================================================
+on (tmpfs, F11, x86_64, lots of RAM)
+/t, with very deep, narrow tree, new rm is >20% faster:
+$ z-mkdir 1000000
+$ env time /p/bin/rm -rf $TMPDIR/z
+6.25user 12.88system 0:19.22elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
+0inputs+0outputs (0major+70976minor)pagefaults 0swaps
+$ z-mkdir 1000000
+env time /bin/rm -rf $TMPDIR/z
+$ env time /bin/rm -rf $TMPDIR/z
+11.06user 14.32system 0:25.43elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
+56inputs+0outputs (1major+79999minor)pagefaults 0swaps
+
+*** slow*DOWN* of ~4% on a 1-1Kdir-1M-file (1k-files-per-dir) hierarchy
+again, this is on tmpfs:
+$ z-mktree --root=/t/z --depth=2 --b=1000;strace -c /p/bin/rm -rf /t/z
+vv$ z-mktree --root=/t/z --depth=2 --b=1000;env time /cu/src/rm -rf /t/z
+0.29user 2.75system 0:03.06elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
+0inputs+0outputs (0major+10834minor)pagefaults 0swaps
+$ z-mktree --root=/t/z --depth=2 --b=1000;env time /p/p/coreutils-7.5/bin/rm -rf /t/z
+0.14user 2.76system 0:02.92elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
+0inputs+0outputs (0major+184minor)pagefaults 0swaps
+
+
+==================================================
+
+[old rm]
+iou$ z-mktree --root=/t/z --depth=2 --b=1000;strace -c /p/bin/rm -rf /t/z
+% time     seconds  usecs/call     calls    errors syscall
+------ ----------- ----------- --------- --------- ----------------
+ 90.66    0.086370           0   1001002         1 unlinkat
+  8.72    0.008310           1     10002           getdents64
+  0.22    0.000214           0      2001           openat
+  0.20    0.000188           0      2021           close
+  0.09    0.000089           0      6019           fstat64
+  0.08    0.000073           0      6003           fcntl64
+  0.03    0.000028           1        29        12 open
+  0.00    0.000000           0         4           read
+  0.00    0.000000           0         1           execve
+  0.00    0.000000           0      1000           lseek
+  0.00    0.000000           0         3         3 access
+  0.00    0.000000           0         3           brk
+  0.00    0.000000           0         1           ioctl
+  0.00    0.000000           0         2           munmap
+  0.00    0.000000           0         4           mprotect
+  0.00    0.000000           0        19           mmap2
+  0.00    0.000000           0         1           lstat64
+  0.00    0.000000           0         1           set_thread_area
+  0.00    0.000000           0         1           fstatat64
+------ ----------- ----------- --------- --------- ----------------
+100.00    0.095272               1028117        16 total
+
+
+[new, fts-based rm]
+iou$ z-mktree --root=/t/z --depth=2 --b=1000;strace -c ./rm -rf /t/z
+% time     seconds  usecs/call     calls    errors syscall
+------ ----------- ----------- --------- --------- ----------------
+ 95.88    0.122730           0   1001001           unlinkat
+  3.95    0.005058           1      7007           getdents64
+  0.10    0.000126           0      2022           close
+  0.05    0.000064           0      1001           openat
+  0.02    0.000020           0      1017           fstat64
+  0.00    0.000000           0         4           read
+  0.00    0.000000           0        29        12 open
+  0.00    0.000000           0         1           execve
+  0.00    0.000000           0         3         3 access
+  0.00    0.000000           0       510           brk
+  0.00    0.000000           0         1           ioctl
+  0.00    0.000000           0         2           munmap
+  0.00    0.000000           0         4           mprotect
+  0.00    0.000000           0        19           mmap2
+  0.00    0.000000           0         1           lstat64
+  0.00    0.000000           0      3003           fcntl64
+  0.00    0.000000           0         1           set_thread_area
+  0.00    0.000000           0      1001           fstatat64
+------ ----------- ----------- --------- --------- ----------------
+100.00    0.127998               1016627        15 total
+iou$ export LC_ALL=C                                               cu:198-rm-fts
+iou$ z-mktree --root=/t/z --depth=2 --b=1000;strace -c ./rm -rf /t/z
+% time     seconds  usecs/call     calls    errors syscall
+------ ----------- ----------- --------- --------- ----------------
+ 93.49    0.110607           0   1001001           unlinkat
+  6.32    0.007475           1      7007           getdents64
+  0.10    0.000124           0      1001           fstatat64
+  0.06    0.000068           0       479           brk
+  0.02    0.000018           0      2008           close
+  0.01    0.000015           0      3003           fcntl64
+  0.00    0.000000           0         2           read
+  0.00    0.000000           0         3           open
+  0.00    0.000000           0         1           execve
+  0.00    0.000000           0         3         3 access
+  0.00    0.000000           0         1           ioctl
+  0.00    0.000000           0         1           munmap
+  0.00    0.000000           0         4           mprotect
+  0.00    0.000000           0         6           mmap2
+  0.00    0.000000           0         1           lstat64
+  0.00    0.000000           0      1003           fstat64
+  0.00    0.000000           0         1           set_thread_area
+  0.00    0.000000           0      1001           openat
+------ ----------- ----------- --------- --------- ----------------
+100.00    0.118307               1016526         3 total
+
+iou$ z-mktree --root=/t/z --depth=2 --b=1000;strace -c /p/bin/rm -rf /t/z
+% time     seconds  usecs/call     calls    errors syscall
+------ ----------- ----------- --------- --------- ----------------
+ 92.87    0.107659           0   1001002         1 unlinkat
+  6.59    0.007641           1     10002           getdents64
+  0.22    0.000252           0      2001           openat
+  0.15    0.000179           0      6005           fstat64
+  0.10    0.000113           0      6003           fcntl64
+  0.05    0.000054           0      1000           lseek
+  0.02    0.000027           0      2007           close
+  0.00    0.000000           0         2           read
+  0.00    0.000000           0         3           open
+  0.00    0.000000           0         1           execve
+  0.00    0.000000           0         3         3 access
+  0.00    0.000000           0         3           brk
+  0.00    0.000000           0         1           ioctl
+  0.00    0.000000           0         1           munmap
+  0.00    0.000000           0         4           mprotect
+  0.00    0.000000           0         6           mmap2
+  0.00    0.000000           0         1           lstat64
+  0.00    0.000000           0         1           set_thread_area
+  0.00    0.000000           0         1           fstatat64
+------ ----------- ----------- --------- --------- ----------------
+100.00    0.115925               1028047         4 total
+
+**********************************************************************
+iou$ z-mktree --root=/t/z --depth=2 --b=1000;env time /p/bin/rm -rf /t/z
+0.16user 3.35system 0:03.68elapsed 95%CPU (0avgtext+0avgdata 0maxresident)k
+0inputs+0outputs (0major+166minor)pagefaults 0swaps
+iou$ z-mktree --root=/t/z --depth=2 --b=1000;env time /p/bin/rm -rf /t/z
+0.16user 3.12system 0:03.34elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k
+0inputs+0outputs (0major+166minor)pagefaults 0swaps
+iou$ z-mktree --root=/t/z --depth=2 --b=1000;env time /p/bin/rm -rf /t/z
+0.14user 3.12system 0:03.26elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
+0inputs+0outputs (0major+167minor)pagefaults 0swaps
+iou$ z-mktree --root=/t/z --depth=2 --b=1000;env time /p/bin/rm -rf /t/z
+0.12user 2.98system 0:03.17elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k
+0inputs+0outputs (0major+167minor)pagefaults 0swaps
+
+
+iou$ z-mktree --root=/t/z --depth=2 --b=1000;env time ./rm -rf /t/z
+0.68user 3.08system 0:03.76elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
+0inputs+0outputs (0major+1566minor)pagefaults 0swaps
+iou$ z-mktree --root=/t/z --depth=2 --b=1000;env time ./rm -rf /t/z
+0.64user 3.12system 0:03.79elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
+0inputs+0outputs (0major+1565minor)pagefaults 0swaps
+iou$ z-mktree --root=/t/z --depth=2 --b=1000;env time ./rm -rf /t/z
+0.60user 3.09system 0:03.78elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k
+0inputs+0outputs (0major+1566minor)pagefaults 0swaps
+
+************************************************************************
+************************************************************************
+
+The above are relatively normal hierarchies.
+Here is one that is extreme (/t is a tmpfs file system):
+[shows that fts-based rm takes 20-25% more time than the old one]
+
+iou$ mkdir .j && (cd .j && seq 1000000|xargs touch)
+
+iou$ env time ./rm -rf /t/.j                                       cu:198-rm-fts
+0.10user 0.41system 0:00.59elapsed 87%CPU (0avgtext+0avgdata 0maxresident)k
+256inputs+0outputs (1major+4459minor)pagefaults 0swaps
+iou$ env time ./rm -rf /t/.j                                       cu:198-rm-fts
+0.82user 3.48system 0:04.35elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k
+256inputs+0outputs (1major+43130minor)pagefaults 0swaps
+iou$ env time /p/bin/rm -rf /t/.j                                  cu:198-rm-fts
+0.14user 3.02system 0:03.25elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k
+256inputs+0outputs (1major+164minor)pagefaults 0swaps
+iou$ env time strace -c /p/bin/rm -rf /t/.j                        cu:198-rm-fts
+% time     seconds  usecs/call     calls    errors syscall
+------ ----------- ----------- --------- --------- ----------------
+ 95.49    0.122568           0   1000002         1 unlinkat
+  4.51    0.005785           1      7797           getdents64
+  0.00    0.000000           0         2           read
+  0.00    0.000000           0         3           open
+  0.00    0.000000           0         7           close
+  0.00    0.000000           0         1           execve
+  0.00    0.000000           0         1           lseek
+  0.00    0.000000           0         3         3 access
+  0.00    0.000000           0         3           brk
+  0.00    0.000000           0         1           ioctl
+  0.00    0.000000           0         1           munmap
+  0.00    0.000000           0         4           mprotect
+  0.00    0.000000           0         6           mmap2
+  0.00    0.000000           0         1           lstat64
+  0.00    0.000000           0         5           fstat64
+  0.00    0.000000           0         3           fcntl64
+  0.00    0.000000           0         1           set_thread_area
+  0.00    0.000000           0         1           fstatfs64
+  0.00    0.000000           0         1           openat
+  0.00    0.000000           0         1           fstatat64
+------ ----------- ----------- --------- --------- ----------------
+100.00    0.128353               1007844         4 total
+2.96user 21.57system 0:24.80elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k
+8inputs+0outputs (0major+406minor)pagefaults 0swaps
+iou$ env time strace -c ./rm -rf /t/.j                             cu:198-rm-fts
+% time     seconds  usecs/call     calls    errors syscall
+------ ----------- ----------- --------- --------- ----------------
+ 83.46    0.160976           0   1000001           unlinkat
+ 14.54    0.028036          21      1306           brk
+  2.01    0.003873           0      7795           getdents64
+  0.00    0.000000           0         2           read
+  0.00    0.000000           0         3           open
+  0.00    0.000000           0         8           close
+  0.00    0.000000           0         1           execve
+  0.00    0.000000           0         3         3 access
+  0.00    0.000000           0         1           ioctl
+  0.00    0.000000           0         1           munmap
+  0.00    0.000000           0         4           mprotect
+  0.00    0.000000           0         6           mmap2
+  0.00    0.000000           0         1           lstat64
+  0.00    0.000000           0         3           fstat64
+  0.00    0.000000           0         3           fcntl64
+  0.00    0.000000           0         1           set_thread_area
+  0.00    0.000000           0         1           fstatfs64
+  0.00    0.000000           0         1           openat
+  0.00    0.000000           0         1           fstatat64
+------ ----------- ----------- --------- --------- ----------------
+100.00    0.192885               1009142         3 total
+3.86user 22.06system 0:26.30elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k
+0inputs+0outputs (0major+43371minor)pagefaults 0swaps
+
+======================================================
+# On a faster system, the new fts-based rm still takes a 20% perf. hit,
+# probably due to malloc (ab)use, just like above.  Yep, sure looks like it.
+# in /tmpfs, /t
+mkdir .j && (cd .j && seq 1000000|xargs touch)
+vv$ env time /p/p/coreutils-7.5/bin/rm -rf .j
+0.14user 2.81system 0:02.98elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
+264inputs+0outputs (1major+174minor)pagefaults 0swaps
+vv$ cd /t; env time /cu/src/rm -rf .j
+0.38user 3.17system 0:03.57elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
+0inputs+0outputs (0major+70489minor)pagefaults 0swaps
+
+vv$ strace -c /p/p/coreutils-7.5/bin/rm -rf .j
+% time     seconds  usecs/call     calls    errors syscall
+------ ----------- ----------- --------- --------- ----------------
+ 66.41    0.081223           0   1000002         1 unlinkat
+ 33.43    0.040887          42       978           getdents
+  0.16    0.000199         199         1           execve
+  0.00    0.000000           0         1           read
+  0.00    0.000000           0         3           open
+  0.00    0.000000           0         7           close
+  0.00    0.000000           0         6           fstat
+  0.00    0.000000           0         1           lstat
+  0.00    0.000000           0         1           lseek
+  0.00    0.000000           0         9           mmap
+  0.00    0.000000           0         3           mprotect
+  0.00    0.000000           0         1           munmap
+  0.00    0.000000           0         3           brk
+  0.00    0.000000           0         1           ioctl
+  0.00    0.000000           0         1         1 access
+  0.00    0.000000           0         3           fcntl
+  0.00    0.000000           0         1           fstatfs
+  0.00    0.000000           0         1           arch_prctl
+  0.00    0.000000           0         1           openat
+  0.00    0.000000           0         1           newfstatat
+------ ----------- ----------- --------- --------- ----------------
+100.00    0.122309               1001025         2 total
+
+
+vv$ cd /t; strace -c /cu/src/rm -rf .j
+% time     seconds  usecs/call     calls    errors syscall
+------ ----------- ----------- --------- --------- ----------------
+ 51.26    0.087638           0   1000001           unlinkat
+ 26.49    0.045291          46       976           getdents
+ 22.24    0.038024          18      2134           brk
+  0.00    0.000000           0         1           read
+  0.00    0.000000           0         3           open
+  0.00    0.000000           0         8           close
+  0.00    0.000000           0         4           fstat
+  0.00    0.000000           0         1           lstat
+  0.00    0.000000           0         9           mmap
+  0.00    0.000000           0         3           mprotect
+  0.00    0.000000           0         1           munmap
+  0.00    0.000000           0         1           ioctl
+  0.00    0.000000           0         1         1 access
+  0.00    0.000000           0         1           execve
+  0.00    0.000000           0         3           fcntl
+  0.00    0.000000           0         1           fstatfs
+  0.00    0.000000           0         1           arch_prctl
+  0.00    0.000000           0         1           openat
+  0.00    0.000000           0         1           newfstatat
+------ ----------- ----------- --------- --------- ----------------
+100.00    0.170953               1003151         1 total
--- a/src/copy.c
+++ b/src/copy.c
@@ -123,20 +123,12 @@ static char const *top_level_dst_name;
 static inline int
 utimens_symlink (char const *file, struct timespec const *timespec)
 {
-  int err = 0;
-
-#if HAVE_UTIMENSAT
-  err = utimensat (AT_FDCWD, file, timespec, AT_SYMLINK_NOFOLLOW);
+  int err = lutimens (file, timespec);
  /* When configuring on a system with new headers and libraries, and
     running on one with a kernel that is old enough to lack the syscall,
     utimensat fails with ENOSYS.  Ignore that.  */
  if (err && errno == ENOSYS)
    err = 0;
-#else
-  (void) file;
-  (void) timespec;
-#endif
-
  return err;
 }

--- a/src/touch.c
+++ b/src/touch.c
@@ -22,6 +22,7 @@
 #include <stdio.h>
 #include <getopt.h>
 #include <sys/types.h>
+#include <assert.h>

 #include "system.h"
 #include "argmatch.h"
@@ -119,11 +120,9 @@ static bool
 touch (const char *file)
 {
  bool ok;
-  struct stat sbuf;
  int fd = -1;
  int open_errno = 0;
-  struct timespec timespec[2];
-  struct timespec const *t;
+  struct timespec const *t = newtime;

  if (STREQ (file, "-"))
    fd = STDOUT_FILENO;
@@ -144,24 +143,13 @@ touch (const char *file)

  if (change_times != (CH_ATIME | CH_MTIME))
    {
-      /* We're setting only one of the time values.  stat the target to get
-         the other one.  If we have the file descriptor already, use fstat.
-         Otherwise, either we're in no-create mode (and hence didn't call open)
-         or FILE is inaccessible or a directory, so we have to use stat.  */
-      if (fd != -1 ? fstat (fd, &sbuf) : stat (file, &sbuf))
+      /* We're setting only one of the time values.  */
+      if (change_times == CH_MTIME)
+        newtime[0].tv_nsec = UTIME_OMIT;
+      else
        {
-          if (open_errno)
-            error (0, open_errno, _("creating %s"), quote (file));
-          else
-            {
-              if (no_create && (errno == ENOENT || errno == EBADF))
-                return true;
-              error (0, errno, _("failed to get attributes of %s"),
-                     quote (file));
-            }
-          if (fd == STDIN_FILENO)
-            close (fd);
-          return false;
+          assert (change_times == CH_ATIME);
+          newtime[1].tv_nsec = UTIME_OMIT;
        }
    }

@@ -171,16 +159,6 @@ touch (const char *file)
         write access to the file, but don't own it.  */
      t = NULL;
    }
-  else
-    {
-      timespec[0] = (change_times & CH_ATIME
-                     ? newtime[0]
-                     : get_stat_atime (&sbuf));
-      timespec[1] = (change_times & CH_MTIME
-                     ? newtime[1]
-                     : get_stat_mtime (&sbuf));
-      t = timespec;
-    }

  ok = (gl_futimens (fd, (fd == STDOUT_FILENO ? NULL : file), t) == 0);

@@ -195,8 +173,7 @@ touch (const char *file)
  else if (fd == STDOUT_FILENO)
    {
      /* Do not diagnose "touch -c - >&-".  */
-      if (!ok && errno == EBADF && no_create
-          && change_times == (CH_ATIME | CH_MTIME))
+      if (!ok && errno == EBADF && no_create)
        return true;
    }

@@ -429,10 +406,7 @@ main (int argc, char **argv)
      if (change_times == (CH_ATIME | CH_MTIME))
        amtime_now = true;
      else
-        {
-          gettime (&newtime[0]);
-          newtime[1] = newtime[0];
-        }
+        newtime[1].tv_nsec = newtime[0].tv_nsec = UTIME_NOW;
    }

  if (optind == argc)
--- a/tests/cp/preserve-slink-time
+++ b/tests/cp/preserve-slink-time
@@ -24,6 +24,7 @@ fi
 . $srcdir/test-lib.sh

 grep '^#define HAVE_UTIMENSAT' "$CONFIG_HEADER" > /dev/null ||
+grep '^#define HAVE_LUTIMES' "$CONFIG_HEADER" > /dev/null ||
  skip_test_ 'this system lacks the utimensat function'

 ln -s no-such dangle || framework_failure
@@ -34,6 +35,7 @@ case $(stat --format=%y dangle) in
  ??:??:??.000000000) sleep 2;;
 esac

+# Can't use --format=%x, as lstat() modifies atime on some platforms.
 cp -Pp dangle d2 || framework_failure
 stat --format=%y dangle > t1 || framework_failure
 stat --format=%y d2 > t2 || framework_failure
Author	SHA1	Message	Date
Eric Blake	6c935a0554	Merge remote branch 'origin/next' into next	2009-10-10 09:28:19 -06:00
Jim Meyering	3b7dc2b522	rm: record timing data * README-rm-timing-compare: new file * README-rm-timing-2: New file.	2009-10-10 09:21:36 -06:00
Eric Blake	aeaa6bf7ae	touch: optimize use of utimens * src/touch.c (main): Use UTIME_NOW rather than calling gettime. (touch): Use UTIME_OMIT rather than stat.	2009-10-10 09:14:39 -06:00
Eric Blake	82e344ca09	copy: allow symlink timestamp preservation on more systems * src/copy.c (utimens_symlink): Simplify by using lutimens. * m4/jm-macros.m4 (coreutils_MACROS): Drop utimensat; gnulib does this for us. * tests/cp/preserve-slink-time: Recognize lutimes support.	2009-10-10 09:14:35 -06:00
Eric Blake	2cd1577554	build: update gnulib submodule to latest, for utimens improvements	2009-10-10 09:14:17 -06:00
Eric Blake	41c9e064a6	rm: avoid compiler warning * src/remove.c (rm_fts): Don't allow fall-through when assertions are disabled.	2009-09-05 16:28:46 +02:00
Eric Blake	5f303a1293	euidaccess-stat: remove unnecessary macros * lib/euidaccess-stat.c (F_OK, R_OK, W_OK, X_OK): Delete; now guaranteed by gnulib.	2009-09-05 16:28:46 +02:00
Eric Blake	2df272914b	rm: use gnulib faccessat * bootstrap.conf (gnulib_modules): Add faccessat. Replace strdup with strdup-posix. * m4/jm-macros.m4 (coreutils_MACROS): Revert previous change, now that gnulib does it for us. * src/remove.c (write_protected_non_symlink): Use faccessat in more situations.	2009-09-05 16:28:46 +02:00
Jim Meyering	278109b9f4	NEWS: mention recent improvements in rm	2009-09-05 16:28:46 +02:00
Jim Meyering	0f3f7d495c	rm: improve efficiency of rm -r (without -f) from O(N^2) to O(N) where N is the depth of the deepest hierarchy rm is processing. * src/remove.c (write_protected_non_symlink): Use faccessat to avoid O(N)-per-entry cost of calling euidaccess. * m4/jm-macros.m4 (coreutils_MACROS): Check for faccessat.	2009-09-05 16:28:46 +02:00
Jim Meyering	4f87cc0364	build: placate gcc's new -Wskip-jump-init * remove.c (rm_fts): Put braces around each of the two offending blocks. * configure.ac: Don't turn off -Wjump-misses-init. With the rewrite of remove.c, it is no longer needed.	2009-09-05 16:28:46 +02:00
Jim Meyering	18c5fbd5dc	rm: rewrite to use fts * remove.c: Don't include "unlinkdir.h"; no longer used. Do not include <setjmp.h> or "cycle-check.h". Likewise. Include "xfts.h". (dir_name, dir_len): Remove definitions. (CONSECUTIVE_READDIR_UNLINK_THRESHOLD): Likewise. (INODE_SORT_DIR_ENTRIES_THRESHOLD, NEED_REWIND, D_TYPE): Likewise. (struct dirstack_state, Dirstack_state): Likewise. (g_buf, g_n_allocated): Remove declarations. (hash_freer, hash_compare_strings, rm_malloc): Remove functions. (rm_free, push_dir, top_dir, pop_dir, right_justify): Likewise. (full_filename0, xfull_filename, full_filename_): Likewise. (AD_stack_height, AD_stack_top, AD_stack_pop, AD_stack_clear): Likewise. (obstack_init_minimal, ds_init, ds_clear, ds_free): Likewise. (AD_pop_and_chdir, AD_ensure_initialized, AD_mark_helper): Likewise. (AD_mark_as_unremovable, AD_mark_current_as_unremovable): Likewise. (AD_push_initial, AD_push, AD_push, AD_is_removable): Likewise. (write_protected_non_symlink): Change 3rd parameter from dirstack_state "ds" to full_name. (prompt): Adjust parameters. Now, state comes from FTS/FTSENT pair. Those replace fd_cwd and "ds". Remove "filename". Remove pdirent_type in favor of new "is_dir" parameter. Rename is_empty to is_empty_p. (DO_RMDIR, DO_UNLINK): Remove definitions. (remove_entry, fd_to_subdirp, compare_ino): Remove functions. (dirent_count, dirent_inode_sort_may_be_useful): Likewise. (preprocess_dir): Likewise. (fts_skip_tree, mark_ancestor_dirs, excise, rm_fts): New functions. (remove_cwd_entries, remove_dir, rm_1): Remove functions. (rm): Rewrite as a simple loop calling fts_read and dispatching each entry via rm_fts. * src/rm.c (main): Adapt to new signature of rm(). * bootstrap.conf (gnulib_modules): Remove unlinkdir, no longer used. * src/Makefile.am (sc_tight_scope): Also recognize an extern "enum" declaration. * tests/rm/empty-name: Adjust expected output to match new diagnostic.	2009-09-05 16:28:46 +02:00
Jim Meyering	cf963e5a54	rm: record timing data * README-rm-timing-compare: new file * README-rm-timing-2: New file.	2009-09-05 16:28:46 +02:00