Currently, filefrag's "expected physical block" column expects extent
records to be physically adjacent regardless of the amount of logical
block space between the two records. This means that if we punch a
hole in a file, we get reports like this:
ext: logical_offset: physical_offset: length: expected: flags:
4: 4096.. 8343: 57376.. 61623: 4248:
5: 8345.. 10313: 61625.. 63593: 1969: 61624:
Notice how it expects 8345 to map to 61624, and scores this against
the fragmentation of the file. Flagging this as "unexpected" is
incorrect because the gap in the logical mapping is exactly the same
size as the gap in the physical extents.
Furthermore, this particular mapping leaves the door open to the
optimal mapping -- if a write to block 8344 causes it to be mapped to
61624, the entire range 4096-10313 can be mapped with a single extent.
Until that happens, there's no way to combine extents 4 and 5 because
of the gap in the logical mapping at block 8344.
Therefore, tweak the extent report to account for holes.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Also change ext2fs_symlink() so that the target parameter is a const
char *, thus promising that we will never change the incoming string.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Some temporary char buffers allocated on the stack are not properly
aligned when typecast to a structure containing __u32 or __u64 types,
and this can cause alignment warnings on ARM and other alignment
sensitive architectures, and potential slowdowns to do fixups.
Fix the buffer alignment to avoid such issues.
Addresses: https://bugzilla.redhat.com/show_bug.cgi?id=680090
Reported-by: Gordan Bobic <gordan.bobic@gmail.com>
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
This started with the fm_ext being uninitialized, but upon closer
analysis I discovered that forcing extent emulation in FIBMAP mode
was reporting an extent for every block in the file. Fix both
problems.
The Coverity bug was 1297512.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The extent count calculation works correctly with the FIBMAP ioctl in
verbose (-v) mode, but without the verbose option, the calculation was
broken because we weren't properly updating the fm_ext data structures
in non-verbose mode.
Addresses-Launchpad-Bug: #1356496
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Print filefrag_fiemap() error message to stderr instead of stdout.
Only call ioctl(EXT3_IOC_GETFLAGS) for ext{2,3,4} filesystems to
decide if the ext2 indirect block allocation heuristic shold be used.
Properly handle the the force_bmap (-B) option.
Exit with a positive error number instead of a negative one.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
ioctl(FIGETBSZ) was used to get block size earlier but 2508eaa7
(filefrag: improvements to filefrag FIEMAP handling) moved to fstatfs
f_bsize which doesn't work well for many files systems.
Block size returned using fstatfs isn't block size but "optimal
transfer block size" as per man page. Even stat st_blksize is
"preferred I/O block size" and in may file systems it may even vary
from file to file (POSIX). This patch changes filefrag to use
FIGETBSZ preferentially over f_bsize.
[ Modified by tytso to add the fallback to f_bsize if FIGETBSZ fails
for some reason ]
Signed-off-by: Rakesh Pandit <rakesh@tuxera.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
29758d2 broke -B option which is useful for filesystems not supporting
FIEMAP. Also, fix extents calculation for -B which is broken since
2508eaa7.
Signed-off-by: Rakesh Pandit <rakesh@tuxera.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
29758d2 filefrag: exit with error code if an error is hit
introduced a couple errors; in one case it missed returning
a value, and possibly picked up errno from (unchecked) close(),
and in the other used a test where it needed an
assignment. So capture the error, move perror() directly
after the failed call in both cases, and fix the assignment.
Also fix a precedence problem with:
if (fe_flags & mask == 0)
which is equivalent to:
if (fe_flags & (mask == 0))
but we need:
if ((fe_flags & mask) == 0)
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
If an error is hit during filefrag operation, it will continue to run
(if multiple files are specified on the command-line), but will exit
with a non-zero value, so that callers can determine that some error
was hit.
Clean up the printing of FIEMAP flags and print some newer flags that
were missing. Also print unknown flags as hex values.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Filefrag doesn't catch and print the shared extent flag. Add this for
users of filefrag on file systems with shared extents (such as btrfs).
Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Update the filefrag program to allow displaying the extents in
some different formats. Try and stay within 80 columns.
* add -k option to print extents in kB-sized units (like df -k)
* add -b {blocksize} to print extents in blocksize units
* add -e option to print extent format, even when FIBMAP is used
* add -X option to print extents in hexadecimal format
Internally, the FIBMAP handling code has been moved into its own
function like FIEMAP, so that the code is more modular. Extent
offsets are now handled in bytes instead of in blocks, to allow
printing extents with arbitrary block sizes. The extent header
printing also moved into its own function so that it can be shared
between the FIEMAP and FIBMAP handling routines, since it got more
complex with the different output options.
Only print error about FIBMAP being root-only a single time.
Print the filesystem type if it changes between specified files.
Add fsync() for FIBMAP if "-s" is given.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Quiet a number of simple compiler warnings:
- pointers not initialized by ext2fs_get_mem()
- return without value in non-void function
- dereferencing type-punned pointers
- unused variables
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
filefrag on a virtual fs like proc segfaults:
Floating point exception
because stat.f_blocks is 0, so the calculation of cylgroups is 0,
which leads to a divide by 0 when calculating expected extents.
Since it's only used for ext2 filesystems anyway, just move
the calculation of expected under "if (is_ext2)" to fix this.
Reported-by: Max Beikirch <maxnet@onlinehome.de>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
filefrag has several bugs:
1.
$ touch f1
$ filefrag f1
f1: 1 extent found ----> bug!
$ filefrag -v f1
Filesystem type is: ef53
File size of f1 is 0 (0 blocks, blocksize 4096)
f1: 0 extents found
2.
$ truncate -s 1m f2
$ filefrag f2
f2: 1 extent found ----> bug!
$ filefrag -v f2
Filesystem type is: ef53
File size of f2 is 1048576 (256 blocks, blocksize 4096)
f2: 0 extents found
3.
$ for i in `seq 11 -2 0`; do dd if=/dev/zero of=f4 bs=4k count=1 seek=$i conv=notrunc oflag=sync &>/dev/null; done
$ ll f4
-rw-r--r-- 1 root root 49152 Jun 9 15:09 f4
$ filefrag f4
f4: 7 extents found
$ filefrag -v f4
Filesystem type is: ef53
File size of f4 is 49152 (12 blocks, blocksize 4096)
ext logical physical expected length flags
0 1 1109993 1
1 3 1109992 1109994 1
2 5 1109991 1109993 1
3 7 1109990 1109992 1
4 9 1109989 1109991 1
5 11 1108207 1109990 1 eof
f4: 7 extents found -----------------------> but we only have 6 extents, bug!
All of these bugs come from the fact that we've made a mistake on
calculating total extents:
o we set 1 as default for 'total extents', and this will report 1
extent found even when we don't get any extent from fiemap.
o if our first extent does not start from 0(logical addr), total
extents will be one more than what it should be.
Addresses-Red-Hat-Bugzilla: #840848
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Liu Bo <liubo2009@cn.fujitsu.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Commit a00be17e47 was missing a patch hunk needed to prevent
filefrag from looping forever when it is run without the -v option.
Addresses-Debian-Bug: #644792
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
From a bug report filed by Ibragimov Rinat:
When filefrag uses FIEMAP ioctl its logic differs for ordinary and
verbose (-v) modes. ext4 returns extent on every 32768 block so on
large files it is possible that `filefrag large-file' tells about 4
extents while `filefrag -v large-file' finds only one.
Also when I tried to use generic_block_fiemap function to add
FIEMAP for reiserfs, every block was reported as a new extent
resulting in thousands "extents" for continuous files.
I think filefrag should merge adjacent extents even when -v is not
specified.
Addresses-Debian-Bug: #631498
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The DEFS line in MCONFIG had gotten so long that it exceeded 4k, and
this was starting to cause some tools heartburn. It also made "make
V=1" almost useless, since trying to following the individual commands
run by make was lost in the noise of all of the defines.
So fix this by putting the configure-generated defines in lib/config.h
and the directory pathnames to lib/dirpaths.h.
In addition, clean up some vestigal defines in configure.in and in the
Makefiles to further shorten the cc command lines.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The "count" variable is only ever set if FIBMAP is used,
due to the -B switch, or a fiemap failure. However,
we use it unconditionally to calculate "expected" for
extN files, so we can end up printing garbage.
Initialize count to 0, and unless we go through the FIBMAP
path, expected will be 0 as well, and in that case do not
print the message.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
/boot/a: 0 extents found
works properly, but
Filesystem type is: ef53
Filesystem cylinder groups is approximately 61
File size of a is 0 (0 blocks, blocksize 1024)
ext logical physical expected length flags
a: 1 extent found
yields 1 extent when it should be 0.
Fix this up by special-casing no extents returned in verbose
mode; skip printing the header for the columns too, since there
are no columns to print.
Also, in nonverbose mode we can set fm_extent_count to 0
so that FIEMAP will just query the extent count without gathering
details; clarify this with a comment.
Addresses-RedHat-Bugzilla: 653234
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
On 32-bit platforms where the file system block size is 8k or greater,
the calculation bpib*bpib*bpib* will overflow a 32-bit calculation,
leading to a divide by zero error. Fix this.
Thanks to Mikulas Patocka for pointing this out.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The extent list header gets printed before we fall back to bmap:
# filefrag -v /mnt/test/bar
Filesystem type is: 58465342
File size of /mnt/test/bar is 12288 (3 blocks, blocksize 4096)
ext logical physical expected length flags <---- HERE
Discontinuity: Block 2 is at 17 (was 16)
/mnt/test/bar: 2 extents found
so delay printing it until we know fiemap is working.
(though ideally it'd be nice to have the same verbose output
regardless of the interface we used, I think).
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The fragmentation count in the bmap case seems to be
off by one:
/mnt/test/bar: 0 extents found
Addresses-Debian-Bug: #540376
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The FIEMAP support added in e2fsprogs 1.41.6 broke the "perfection
would be XXX expects" calculation restore it.
Also fix some gcc -Wall warnings as well. (Cleaning up gcc -Wall is
what caused me to notice this regression).
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
When used with -v and the targeted file has more than 144
extents(double of the length of fm_extents array provided by buf),
filefrag_fiemap loops and calls fiemap ioctl() multiple times to
calculate the actual number of extents in a file. Each call to fiemap
ioctl() uses fm_start as the starting logical offset. The patch fixes
fm_start in each loop( except for the first one) and makes the extent
calculation correct for files with more that 144 extents.
To produce the problem, first run filefrag -v on a highly fragmented
file. Then change the buf size in filefrag_fiemap to make it large
enough to have all the extent mapped in a single loop and run filefrag
-v after recompiling. The former will produce a much smaller extent
count because of the false fm_start used in the loop. And the two will
produce different extent output since the 145th extent.
Signed-off-by: Peng Tao <bergwolf@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The FIEMAP ioctl is more efficient and doesn't require root
privileges. So if it is available, use it in preference to repeated
FIBMAP calls.
Signed-off-by: Kalpak Shah <kalpak.shah@sun.com>
Signed-off-by: Andreas Dilger <adilger@sun.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Fix calculation of the ideal number of extents needed for a file to
take into account sparse files.
In addition, suppress the "this file is extent-based" message unless
verbose mode is enabled.
Addresses-Debian-Bug: #458306
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add a new function, ext2fs_div_ceil(), which correctly calculates a division
of two unsigned integer where the result is always rounded up the next
largest integer. This is used everywhere where we might have
previously caused an overflow when the number of blocks
or inodes is too close to 2**32-1.
Based on patches from Eric Sandeen, but generalized to use this new function
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Signed-off-by: Eric Sandeen <esandeen@redhat.com>
This fixes some (but not all) of the compatibility bugs which prevented
e2fsprogs from being compiled on a Linux 2.0.35 system. There are still
some unprotected use of long long's, and apparently some type problems
with the uuid library, but these can be fixed up later.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Currently filefrag uses signed int for block numbers, thus it reporting
corrupted block number for a file on a more than 8TB ext3. The following
trivial patch replace the signed int type block number with "unsigned
long type.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o
Vitaliy Filippov