e2fsck does not detect extents which are outside their location in the
extent tree. This can result in a bad extent at the end of an extent-block
not being detected.
From a part of a dump_extents output:
1/ 2 37/ 68 143960 - 146679 123826181 2720
2/ 2 1/ 2 143960 - 146679 123785816 - 123788535 2720
2/ 2 2/ 2 146680 - 147583 123788536 - 123789439 904 Uninit <-bad extent
1/ 2 38/ 68 146680 - 149391 123826182 2712
2/ 2 1/ 2 146680 - 147583 18486 - 19389 904
2/ 2 2/ 2 147584 - 149391 123789440 - 123791247 1808
e2fsck does not detect this bad extent which both overlaps another, valid
extent, and is invalid by being beyond the end of the extent above it in
the tree.
This patch modifies e2fsck to detect this invalid extent and remove it.
Signed-off-by: David Jeffery <djeffery@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Perhaps the most serious fix up is a type-punning warning which could
result in miscompilation with overly enthusiastic compilers.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
e2fsck thinks that this:
# touch mnt/testfile1
# setfattr -n "user.test" mnt/testfile1
results in a filesystem with corruption:
Pass 1: Checking inodes, blocks, and sizes
Extended attribute in inode 12 has a value size (0) which is invalid
Clear? yes
but as far as I can tell, there is absolutely nothing wrong with
a 0-length value on an extended attribute. Just remove the check.
Reported-by: David Shaw <dshaw@jabberwocky.com>
Reported-by: Harald Reindl <h.reindl@thelounge.net>
Addresses-Red-Hat-Bugzilla: #557959
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Commit 789bd401c3 ("e2fsck: fix incorrect interior node logical start
values") surfaced a bug where if e2fsck finds and removed an invalid
node in the extent tree, i.e.:
Inode 12 has an invalid extent node (blk 22, lblk 0)
Clear? yes
It was possible for starting logical blocks found in the interior
nodes of the extent tree. Commit 789bd401c3 added the ability for
e2fsck to discover this problem, which resulted in the test
f_extent_bad_node to fail when the second pass of e2fsck reported the
following complaint:
Interior extent node level 0 of inode 12:
Logical start 0 does not match logical start 3 at next level. Fix? yes
This patch fixes this by adding a call to ext2fs_extent_fix_parents()
after deleting the bogus node in the extent tree.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
An index node's logical start (ei_block) should
match the logical start of the first node (index
or leaf) below it. If we find a node whose start
does not match its parent, fix all of its parents
accordingly.
If it finds such a problem, we'll see:
Pass 1: Checking inodes, blocks, and sizes
Interior extent node level 0 of inode 274258:
Logical start 3666 does not match logical start 4093 at next level. Fix<y>?
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Optimize e2fsck pass 1 by marking entire extents as being in use at a
time, instead of block by block. This optimization only works for
non-bigalloc file systems for now (it's tricky to handle bigalloc file
systems since this code is also responsible for dealing with blocks
that are not correctly aligned within a cluster). When the
optimization works, the CPU savings can be significant: ove a full CPU
minute for a mostly full 4T disk.
Addresses-Google-Bug: #7534813
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Lukas Czerner <lczerner@redhat.com>
Since "bool" is a valid C type, declarations of the form "int bool"
will cause compiler errors if <stdbool.h> is included. Rename these
variables to avoid this name clash.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The on-disk format for interior nodes in the extent tree does not
encode the length of each entry in the interior node; instead, it is
synthesized/simulated by the extent library code in libext2fs.
Unfortunately, this simulation is not perfect; in particular it does
not work for the last extent in the extent tree if there are
uninitialized blocks allocated using fallocate with
FALLOC_FL_KEEP_SIZE, and it leads to e2fsck incorrectly complaining
about an invalid zero-length extent.
We only need to worry about the extent length for the leaves of the
tree, since it is there were we are checking an on-disk value, as
opposed to a software-generated simulation. So restrict the check of
extent length to leaf nodes in the extent tree.
Reported-by: Tao Ma <tm@tao.ma>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The operator precedence bug means that we might pay atteion to
s_grp_quota_inum even if the RO_COMPAT_QUOTA feature flag is clear.
However, fortunately, this is unlikely to happen in practice.
Cc: Aditya Kali <adityakali@google.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
We've decided to remove EOFBLOCKS_FL from the ext4 file system entirely,
because it is not actually very useful and it is causing more problems
than it solves. We're going to remove it from e2fsprogs first and then
after the new e2fsprogs version is common enough we can remove the
kernel part as well.
This commit changes e2fsck to not check for EOFBLOCKS_FL. Instead we
simply search for initialized extents past the i_size as this should not
happen. Uninitialized extents can be past the i_size as we can do
fallocate with KEEP_SIZE flag.
Also remove the EXT4_EOFBLOCKS_FL from lib/ext2fs/ext2_fs.h since it is
no longer needed.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If an extent has e_len set to zero, the kernel will oops with a
BUG_ON. Unfortunately, e2fsck wasn't catching this case. The kernel
needs to be fixed to notice this case and call ext4_error() instead of
failing an assertion check, but e2fsck should catch this case and
repair it (by deleting the errant extent).
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Now that we have multiple backend implementations of the bitmap code,
this commit teaches e2fsck to use either the most appropriate backend
for each use case.
Since we don't know for sure if we will get it all right, the default
choices can be overridden via e2fsck.conf. The various definitions
are shown here, with the current defaults (which may change as we add
more bitmap implementations and as learn what works better).
; EXT2FS_BAMP64_BITARRAY is 1
; EXT2FS_BMAP64_RBTREE is 2
; EXT2FS_BMAP64_AUTODIR is 3
[bitmaps]
inode_used_map = 2 ; pass1
inode_dir_map = 3 ; pass1
inode_reg_map = 2 ; pass1
block_found_map = 2 ; pass1
inode_bad_map = 2 ; pass1
inode_imagic_map = 2 ; pass1
block_dup_map = 2 ; pass1
block_ea_map = 2 ; pass1
inode_link_info = 2 ; pass1
inode_dup_map = 2 ; pass1b
inode_done_map = 3 ; pass3
inode_loop_detect = 3 ; pass3
fs_bitmaps = 2
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
When I create a non-extent file with the maximum size in ext4,
e2fsck detects the following error:
Pass 1: Checking inodes, blocks, and sizes
Inode 12, i_size is 4402345721856, should be 4402345721856. Fix?
As we know, e2fsck checks the size field of the inode in pass 1.
However, in case of the ext4 with the feature of ^extent and huge_file,
the maximum file size calculated in e2fsck is less than the real one.
The patch fixes this problem.
Signed-off-by: Kazuya Mio <k-mio@sx.jp.nec.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The ext2fs_file_acl_block() and ext2fs_set_file_acl_block() needs to
only check i_file_acl_high if the 64-bit flag is set. This is needed
because otherwise we will run into problems on Hurd systems which
actually use that field for h_i_mode_high.
This involves an ABI change since we need to pass ext2_filsys to these
functions. Fortunately these functions were first included in the
1.42-WIP series, so it's OK for us to change them now. (This is why
we have 1.42-WIP releases. :-)
Addresses-Sourceforge-Bug: #3379227
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
In some cases the bad block inode gets corrupted. If it looks insane,
offer to clear it before trying to interpret it does more harm than
good.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Multi-mount protection is feature that allows mke2fs, e2fsck, and
others to detect if the filesystem is mounted on a remote node (on
SAN disks) and avoid corrupting the filesystem. For e2fsprogs this
means that it checks the MMP block to see if the filesystem is in use,
and marks the filesystem busy while e2fsck is running on the system.
This is useful on SAN disks that are shared between high-availability
servers, or accessible by multiple nodes that aren't in HA pairs. MMP
isn't intended to serve as a primary HA exclusion mechanism, but as a
failsafe to protect against user, software, or hardware errors.
There is no requirement that e2fsck updates the MMP block at regular
intervals, but e2fsck does this occasionally to provide useful
information to the sysadmin in case of a detected conflict.
For the kernel (since Linux 3.0) MMP adds a "heartbeat" mechanism to
periodically write to disk (every few seconds by default) to notify
other nodes that the filesystem is still in use and unsafe to modify.
Originally-by: Kalpak Shah <kalpak@clusterfs.com>
Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The DEFS line in MCONFIG had gotten so long that it exceeded 4k, and
this was starting to cause some tools heartburn. It also made "make
V=1" almost useless, since trying to following the individual commands
run by make was lost in the noise of all of the defines.
So fix this by putting the configure-generated defines in lib/config.h
and the directory pathnames to lib/dirpaths.h.
In addition, clean up some vestigal defines in configure.in and in the
Makefiles to further shorten the cc command lines.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
old_op is set but never used, because we restore "0"
not old_op. So don't bother with it.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reserve EXT4_FEATURE_RO_COMPAT_METADATA_CSUM and
EXT2_FEATURE_COMPAT_EXCLUDE_BITMAP. Also reserve fields in the
superblock and the inode for the checksums. In the block group
descriptor, reserve the exclude bitmap field for the snapshot feature,
and checksums for the inode and block allocation bitmaps.
With this commit, the metadata checksum and exclude bitmap features
should have reserved all of the fields they need in ext4's on-disk
format.
This commit also fixes an a missing byte swap for s_overhead_blocks.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: Darrick J. Wong <djwong@us.ibm.com>
Cc: Amir Goldstein <amir73il@gmail.com>
This patch adds support for doing quota accounting during full
e2fsck scan if the 'quota' feature was set on the superblock.
If user-visible quota inodes are in use, they will be hidden
and converted to the reserved quota inodes.
Signed-off-by: Aditya Kali <adityakali@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Currently the bigalloc implementation in the kernel requires extents,
but this restriction might get relaxed in the future. Also, old
versions of mke2fs that supported bigalloc during early testing
created the root and lost+found directories without using
extent-mapped inodes. This makes it possible for e2fsck to better
support these old legacy file systems if it comes across them.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Use the EXT2_I_SIZE() macro consistently to access the inode size.
The i_size/i_size_high combination is open coded in several places.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Fix several types of compiler warnings (unused variables/labels),
uninitialized variables, etc that are hit with gcc -Wall.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
For file systems with 64-bit block numbers, we need to make sure we
correct the i_blocks_hi field as well as the i_blocks field when
setting it to the correct value.
Thanks to Justin Maggard for pointing this out.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This test, added to e2fsprogs-1.41.12, is backwards.
If EOFBLOCKS is set, then the size -should- be less than
the last physical block...
xfstests 013 caught this.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
In the case where s_first_data_block is 1, we need to explictly reject
an extent whose starting physical block is zero.
Thanks to Jiaying Zhang <jiayingz@google.com> for finding this bug.
Addresses-Google-Bug: #2573806
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If a corrupted file system causes us to want to delete an extent, and
that causes us to want to release a block in e2fsck pass #1, it would
be preferable if e2fsck didn't seg fault. This tends to get users
craky, as users are wont to do. :-)
Thanks to Dirk Reiners for reporting this bug:
e2fsck crashes fixing a corrupted 3.5 TB filesystem:
0x0000000000432002 in ext2fs_unmark_generic_bitmap (bitmap=0x0, bitno=623386749)
at gen_bitmap.c:183
183 if ((bitno < bitmap->start) || (bitno > bitmap->end)) {
(gdb) bt
bitno=623386749) at gen_bitmap.c:183
block=623386749) at ../../lib/ext2fs/bitops.h:319
inuse=-1) at alloc_stats.c:78
extent.c:1509
pb=0x7fffffffdfe0, start_block=0, ehandle=0x6dcf50) at pass1.c:1709
pb=0x7fffffffdfe0, start_block=0, ehandle=0x6dcf50) at pass1.c:1737
pctx=0x7fffffffe100, pb=0x7fffffffdfe0) at pass1.c:1842
block_buf=0x6c4330 "\373\212#") at pass1.c:1920
The source of the NULL bitmap is fs on stack frame 2:
(gdb) up 2
inuse=-1) at alloc_stats.c:78
78 ext2fs_unmark_block_bitmap(fs->block_map, blk);
Addresses-SourceForge-Bug: #2971800
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Justin reported that creating a 4T file with posix_fallocate led
to fsck errors:
e2fsck 1.41.10 (10-Feb-2009)
Pass 1: Checking inodes, blocks, and sizes
Inode 12, i_blocks is 8589935432, should be 840. Fix? yes
This looks like a 32-bit overflow.
commmit 8a8f36540b added handling of
the high i_blocks number, but we accumulate blocks in the num_blocks
field, and that's still just 32 bits.
Note: we don't need to expand max_blocks for now, that's only used
in the non-extents case, and those files have smaller max sizes.
I haven't been able to replicate the problem, oddly, but Justin
reports that this patch fixed his situation.
Reported-by: Justin Maggard <jmaggard10@gmail.com>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
There were a number of problems that were prompting the user whether
or not to ABORT, but then would abort regardless of whether the user
answered yes or no. Change those to be PROMPT_NONE, PR_FATAL.
Also, fix PR_1_RESIZE_INODE_CREATE so that it recovers appropriately
after failing to create the resize inode. This problem now uses
PROMPT_CONTINUE instead of PROMPT_ABORT, and if the user says, "no",
the code will abort.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Some kernels will crash if EOFBLOCKS_FL is set when it is it not
needed, and this if it is left set when it isn't needed, it is a sign
of a kernel bug.
Addresses-Google-Bug: #2604224
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This reverts commit 0ea910997b.
Since the Linux kernel now has support for the EXT4_EOFBLOCKS_FL flag
starting in 2.6.34, we don't need this workaround any more.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If a corrupted file system causes us to want to delete an extent, and
that causes us to want to release a block in e2fsck pass #1, it would
be preferable if e2fsck didn't seg fault. This tends to get users
craky, as users are wont to do. :-)
Thanks to Dirk Reiners for reporting this bug:
e2fsck crashes fixing a corrupted 3.5 TB filesystem:
0x0000000000432002 in ext2fs_unmark_generic_bitmap (bitmap=0x0, bitno=623386749)
at gen_bitmap.c:183
183 if ((bitno < bitmap->start) || (bitno > bitmap->end)) {
(gdb) bt
bitno=623386749) at gen_bitmap.c:183
block=623386749) at ../../lib/ext2fs/bitops.h:319
inuse=-1) at alloc_stats.c:78
extent.c:1509
pb=0x7fffffffdfe0, start_block=0, ehandle=0x6dcf50) at pass1.c:1709
pb=0x7fffffffdfe0, start_block=0, ehandle=0x6dcf50) at pass1.c:1737
pctx=0x7fffffffe100, pb=0x7fffffffdfe0) at pass1.c:1842
block_buf=0x6c4330 "\373\212#") at pass1.c:1920
The source of the NULL bitmap is fs on stack frame 2:
(gdb) up 2
inuse=-1) at alloc_stats.c:78
78 ext2fs_unmark_block_bitmap(fs->block_map, blk);
Addresses-SourceForge-Bug: #2971800
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This is the userspace side of Jiaying's EOFBLOCKS patch. With
Aneesh's patches for .33, Jiaying's patch, and this one, xfstests
013/fsstress (even with direct IO enabled) has held up through many
runs.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Pass 1 has a test to see if a special file is really a directory.
Signed-off-by: Nick Dokos <nicholas.dokos@hp.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The e2fsck_get_alloc_block() callback is used so that if the ext2fs
library needs to allocate blocks internally (most notably by the
extents functions), e2fsck's internal block usage map is consulted
since it is the only thing that can be trusted during a large part of
e2fsck's operation.
Change it to update the on-disk bitmap if it is loaded. This reduces
the number of spurious differences found in pass #5.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Directories are not allowed to be sparse; the code for scanning
extent-mapped directories was not calling ext2fs_add_dir_block() for
missing directory blocks, so we weren't catching this form of file
system corruption. Fix this.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The check that determines whether an directory needs to be have an
index added to it depends on i_size. So move it after we have fixed
up i_size so that we reliable will rehash a directory that needs it,
even if its i_size field was originally incorrect. Otherwise, a
second run of e2fsck would be needed before the directory gets an
index added.
Thanks to Mikulas Patocka for providing a sample file system which
demonstrated this problem.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The old method for detecting directories with holes depended on i_size
being correct, even though the correct value of i_size hadn't been
calculated yet. Hence, a directory inode with holes and an i_size of
0 would require two e2fsck passes to fix completely.
The replacement method for determining whether or not
ext2fs_add_dir_block() should be called is more reliable, and reduces
the size of e2fsck and makes the code more readable as a bonus.
Thanks to Mikulas Patocka for providing a sample file system which
demonstrated this problem.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
David Jeffery