When we're expanding a directory, check to see if we're doing an
implied cluster allocation; if so, we don't need to allocate a new
block, and we certainly don't need to update the summary counts.
Reported-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If pass5 finds bitmap errors in a range of clusters, don't print each
cluster number individually when we could print only the start and end
cluster number. e2fsck already does this for the non-bigalloc case.
Reviewed-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
When the rehash process is running on a bigalloc filesystem, it
compresses all the directory entries and hash structures into the
beginning of the directory file and then uses block_iterate3() to free
the blocks off the end of the file. It seems to call
ext2fs_block_alloc_stats2() for every block in a cluster, which is
unfortunate because this function allocates and frees entire clusters
(and updates the summary counts accordingly). In this case e2fsck
writes out incorrect summary counts.
Reviewed-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Compiling with LLVM generates a large number of warnings due
to the use of _() for wrapping strings for i18n:
warning: format string is not a string literal
(potentially insecure) [-Wformat-security]
./nls-enable.h:4:14: note: expanded from macro '_'
#define _(a) (gettext (a))
^~~~~~~~~~~~
These warnings are fixed by using "%s" as the format string,
and then _() is used as the string argument.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
When we're rehashing directories, it's possible that an extent block
(or a map block) could be (silently) allocated by the underlying
libext2fs when expanding the directory. This silent allocation is not
captured in block_found_map, which is disastrous if later the rehash
process expands another directory and uses that same block from
before without realizing that it's now in use.
Therefore, if we notice that the free block count has dropped by more
than what e2fsck allocated itself during the expansion, we iterate the
directory's blocks a second time to ensure that these silent
allocations are marked in the found blocks bitmap.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
For each site where we test for a large file (> 2GB) and set the
LARGE_FILE feature, use a helper function to make the size test
consistent with the test that's in e2fsck. This fixes the fsck
complaints when we try to create a 2GB journal (not so hard with 64k
block size) and fixes the incorrect test in fileio.c.
Reviewed-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Fix the e2fsck problem comments to match the actual message printed,
so that it is possible to find the problem code when searching by
the message.
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Fix the checking of s_mmp_block in e2fsck_pass1() and
ext2fs_mmp_read() to handle the high 32 bits of s_blocks_count.
Remove redundant check of s_mmp_block in do_dump_mmp() right before
ext2fs_mmp_read() is called.
Also fix s_blocks_count_hi in check_backup_super_block(), since it
cannot use the ext2fs_blocks_count() helper easily.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If we have a 64-bit file system with extended attribute blocks, e2fsck
would not correctly handle EA blocks that were located beyond the
32-bit block number boundary. Fix this by teaching
e2fsck/ea_refcount.c to use 64-bit block numbers.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If the external journal device has exactly 1 << 32 blocks,
journal->j_maxlen would get set to zero, which would cause e2fsck to
declare the journal to be invalid.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The resize inode only works on 32-bit block numbers, so use blk_t
instead of blk64_t. This avoids some -Wconversion noise, and slims
the compiled code slightly, especially on 32-bit platforms.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
We need to store some error codes using an int to keep recovery.c as
close as possible to the recovery.c source file in the kernel.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The use of ext2fs_write_dir_block() meant that attempts to fix
deleted/unused inodes in a directory would not be fixed for file
systems with 64-bit block numbers. (And some random block with the
high 32-bits cleared would get corrupted.)
Fix a similar problem when expanding directories and when creating the
lost+found dirctory.
Signed-off-by: Kit Westneat <kwestneat@ddn.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Run sparse against source files when building e2fsprogs with 'make C=1'. If
instead C=2, it configures basic ext2 types for bitwise checking with sparse,
which can help find the (many many) spots where conversion errors are
(possibly) happening.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Since it's impossible to address all blocks of a 64bit filesystem
without extents, have e2fsck turn on the feature if it finds (64bit &&
!extents).
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Fix all the places where we should be using a blk64_t instead of a
blk_t. These fixes are more severe because 64bit values could be
truncated silently.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Commit d3f32c2db8 introduced a regression that caused e2fsck failures
in xfstests generic 013, 070, 083, 091, and 263. Uninitialized
extents created by fallocate() at the end of file with the
FALLOC_FL_KEEP_SIZE flag were identified as invalid. However,
because the file size is not increased when FALLOC_FL_KEEP_SIZE is
used, uninitialized extents can correctly contain blocks located past
the end of file.
Fix this by filtering out possible invalid extents if they are
uninitialized and extend past the block containing the end of file.
Signed-off-by: Eric Whitney <enwlinux@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Under some failure cases, we can get to fatal_error()
without even having a superblock set up. In that case,
ext2fs_mmp_stop() will segfault when it tries to dereference
fs->super.
Check for the existence of a superblock before we go
down the ext2fs_mmp_stop() path to avoid this problem.
Reported-by: Hubert Kario <hkario@redhat.com>
Addresses-Red-Hat-Bugzilla: #997972
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The function deallocate_inode() in e2fsck/pass2.c was buggy in that it
would clear out the inode's mode and flags fields before trying to
deallocate any blocks which might belong to the inode.
The good news is that deallocate_inode() is mostly used to free inodes
which do not have blocks: device inodes, FIFO's, Unix-domain sockets.
The bad news is that if deallocate_inode() tried to free an invalid
extent-mapped inode, it would try to interpret the root of the extent
node as block numbers, and would therefore mark various file system
metadata blocks (the superblock, block group descriptors, the root
directory, etc.) as free and available for allocation. This was
unfortunate.
(Try running an older e2fsck against the test file system image in the
new test f_invalid_extent_symlink, and then run e2fsck a second time
on the fs image, and weep.)
Fortunately, this kind of file system image corruption appears to be
fairly rare in actual practice, since it would require a very unlucky
set of bits to be flipped, or a buggy file system implementation.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
E2fsck was missing a check for directories with logical blocks so
large that i_size > 2GB. Without this check the test image found in
the new test f_toobig_extent_dir will cause e2fsck to die with a
memory allocation failure:
Error storing directory block information (inode=12, block=0, num=475218819): Memory allocation failed
e2fsck: aborted
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reported-by: Andrey Melnikov <temnota.am@gmail.com>
Add a test to see if the backtrace() function requires linking in a
library in /usr/lib.
Addresses-Debian-Bug: #708307
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
e2fsck does not detect extents which are outside their location in the
extent tree. This can result in a bad extent at the end of an extent-block
not being detected.
From a part of a dump_extents output:
1/ 2 37/ 68 143960 - 146679 123826181 2720
2/ 2 1/ 2 143960 - 146679 123785816 - 123788535 2720
2/ 2 2/ 2 146680 - 147583 123788536 - 123789439 904 Uninit <-bad extent
1/ 2 38/ 68 146680 - 149391 123826182 2712
2/ 2 1/ 2 146680 - 147583 18486 - 19389 904
2/ 2 2/ 2 147584 - 149391 123789440 - 123791247 1808
e2fsck does not detect this bad extent which both overlaps another, valid
extent, and is invalid by being beyond the end of the extent above it in
the tree.
This patch modifies e2fsck to detect this invalid extent and remove it.
Signed-off-by: David Jeffery <djeffery@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Perhaps the most serious fix up is a type-punning warning which could
result in miscompilation with overly enthusiastic compilers.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
When opening the external journal, use the same logic to decide
whether or not to open the file system with EXT2_FLAG_EXCLUSIVE found
in main().
Otherwise, it's not posible to use e2fsck when the root file system is
using an external journal.
Reported-by: Calvin Owens <jcalvinowens@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
e2fsck thinks that this:
# touch mnt/testfile1
# setfattr -n "user.test" mnt/testfile1
results in a filesystem with corruption:
Pass 1: Checking inodes, blocks, and sizes
Extended attribute in inode 12 has a value size (0) which is invalid
Clear? yes
but as far as I can tell, there is absolutely nothing wrong with
a 0-length value on an extended attribute. Just remove the check.
Reported-by: David Shaw <dshaw@jabberwocky.com>
Reported-by: Harald Reindl <h.reindl@thelounge.net>
Addresses-Red-Hat-Bugzilla: #557959
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Commit e3507739e4 introduced a build failure if e2fsprogs is
configured with --enable-jbd-debug. Fix this.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
In try_open_fs(), if ext2fs_open2() returns an error, do not try to
access the struct ext2_filesys. The previous check 'if (ret_fs)' was
always true, but even 'if (*ret_fs)' might be incorrect in some cases,
so check 'retval==0' instead.
Signed-off-by: Nickolai Zeldovich <nickolai@csail.mit.edu>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Previously e2fsck would corrupt memory if the log file name was longer
than 100 bytes (e.g., a long log_filename value in e2fsck.conf or a
pattern that expands out to more than 100 bytes). This was due to
incorrectly calling realloc() in append_string() on the struct string
instead of the malloc'ed char* buffer, among other problems. This
patch fixes the call to realloc() and also ensures that the buffer is
grown by sufficiently many bytes (not just by 2x).
Signed-off-by: Nickolai Zeldovich <nickolai@csail.mit.edu>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
"e2fsck: Can't allocate dx_block info array"
is only so helpful - it'd be nice to know how much it tried to allocate.
In particular, since I think malloc(0) can return NULL,
it'd be nice to know if maybe we passed in an uninitialized (or
0-initialized) size.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
There are a number of places where we multiply a dgrp_t with
s_blocks_per_group expecting that we will get a blk64_t. This
requires a cast, or using the convenience function
ext2fs_group_first_block2().
This audit was suggested by Eric Sandeen.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Quiet a number of simple compiler warnings:
- pointers not initialized by ext2fs_get_mem()
- return without value in non-void function
- dereferencing type-punned pointers
- unused variables
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Commit 789bd401c3 ("e2fsck: fix incorrect interior node logical start
values") surfaced a bug where if e2fsck finds and removed an invalid
node in the extent tree, i.e.:
Inode 12 has an invalid extent node (blk 22, lblk 0)
Clear? yes
It was possible for starting logical blocks found in the interior
nodes of the extent tree. Commit 789bd401c3 added the ability for
e2fsck to discover this problem, which resulted in the test
f_extent_bad_node to fail when the second pass of e2fsck reported the
following complaint:
Interior extent node level 0 of inode 12:
Logical start 0 does not match logical start 3 at next level. Fix? yes
This patch fixes this by adding a call to ext2fs_extent_fix_parents()
after deleting the bogus node in the extent tree.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
An index node's logical start (ei_block) should
match the logical start of the first node (index
or leaf) below it. If we find a node whose start
does not match its parent, fix all of its parents
accordingly.
If it finds such a problem, we'll see:
Pass 1: Checking inodes, blocks, and sizes
Interior extent node level 0 of inode 274258:
Logical start 3666 does not match logical start 4093 at next level. Fix<y>?
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Commit 53e3120c18 introduced a regression which would case e2fsck to
overrun an array boundary for bigalloc file systems, and most likely
crash. Fix this by correctly using blocks instead of clusters when
incrementing the loop counter in the fast path optimization case.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add a fast path optimization in e2fsck's pass 5 for the common case
where the block bitmap is correct. The optimization works by
extracting each block group's block allocation bitmap into a memory
buffer, and comparing it with the expected allocation bitmap using
memcmp(). If it matches, then we can just update the free block
counts and be on our way, and skip checking each bit individually.
Addresses-Google-Bug: #7534813
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Lukas Czerner <lczerner@redhat.com>
Optimize e2fsck pass 1 by marking entire extents as being in use at a
time, instead of block by block. This optimization only works for
non-bigalloc file systems for now (it's tricky to handle bigalloc file
systems since this code is also responsible for dealing with blocks
that are not correctly aligned within a cluster). When the
optimization works, the CPU savings can be significant: ove a full CPU
minute for a mostly full 4T disk.
Addresses-Google-Bug: #7534813
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Lukas Czerner <lczerner@redhat.com>
In e2fsck_pass4(), we were consulting inode_dir_map using
ext2fs_test_inode_bitmap2() for every single inode in the file system.
However, there were many cases where we never needed the result of the
test --- most notably if the inode is not in use.
I was a bit surprised that GCC 4.7 with CFLAGS set to "-g -O2" wasn't
able to optimize this out for us, but here is the pass 4 timing for an
empty 3T file system before this patch:
Pass 4: Memory used: 672k/772k (422k/251k), time: 3.67/ 3.66/ 0.00
and afterwards, we see a 43% improvement:
Pass 4: Memory used: 672k/772k (422k/251k), time: 2.09/ 2.08/ 0.00
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Also fix a bug caused by a stray continuation backslash which caused
the e2fsck/Makefile to fail when profiling is enabled.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Since clang uses C99 semantics by default, the main changes required
to allow clang to build e2fsprogs was to add support the C99 inline
semantics, while still allowing us to be built when the legacy (but
still default for gcc) GNU C89 inline semantics are in force.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
When passed a negative count (indicating a byte count rather than
a block count) e2fsck_handle_read_error() treats the data as a full
block, causing unix_write_blk64() (which can handle negative counts
just fine) to try to write too much. Given a faulty block device,
this resulted in a SEGV when unix_write_blk64() read past the bottom
of the stack copying the data to cache. (check_backup_super_block ->
unix_read_blk64 -> raw_read_blk -> e2fsck_handle_read_error)
Reported-by: Alex Friedman <alexfr@il.ibm.com>
Signed-off-by: Jim Keniston <jkenisto@us.ibm.com>
Signed-off-by: Dan Streetman <ddstreet@us.ibm.com>
Reviewed-by: Mingming Cao <mcao@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
In addition to the free blocks and free inodes, also print the number
of blocks and inodes in the verbose statistics.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Disks have gotten bigger, so 8 digits might not be enough. Allow for
12 digits worth of blocks, which is more than enough for 3 petabytes.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add report_time, report_verbose, and report_features options to
e2fsck.conf which enable additional, more verbose reporting by e2fsck.
This is useful for large cloud installations where there are a large
number file systems being managed, and where it may not be obvious
from the e2fsck log files exactly how a particular file system is
configured.
The report_time and report_verbose options, which are the same as the
-tt and -v command line options, respectively, are useful because they
are options specific to e2fsck, and the fsck program does not have a
way of passing certain options only to a specific /sbin/fsck.<fstype>
program.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
When checking to see whether or not a new name is unique, the code was
using the wrong length parameter, which could cause the anti-collision
loop for a long time trying to find what it thinks is a unique name.
Addresses-Sourceforge-Bug: #3540545
Reported-by: Vitaly Oratovsky <vmo@users.sourceforge.net>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Previously e2fsck would only allow a mounted file system to be checked
if it was the root file system and it was mounted read-only. Now
allow any file system mounted read-only if the -f option is specified.
This makes it easier to test how e2fsck handles checking file systems
which are mounted without having to test on the root file system.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong