old_op is set but never used, because we restore "0"
not old_op. So don't bother with it.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reserve EXT4_FEATURE_RO_COMPAT_METADATA_CSUM and
EXT2_FEATURE_COMPAT_EXCLUDE_BITMAP. Also reserve fields in the
superblock and the inode for the checksums. In the block group
descriptor, reserve the exclude bitmap field for the snapshot feature,
and checksums for the inode and block allocation bitmaps.
With this commit, the metadata checksum and exclude bitmap features
should have reserved all of the fields they need in ext4's on-disk
format.
This commit also fixes an a missing byte swap for s_overhead_blocks.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: Darrick J. Wong <djwong@us.ibm.com>
Cc: Amir Goldstein <amir73il@gmail.com>
This patch adds support for doing quota accounting during full
e2fsck scan if the 'quota' feature was set on the superblock.
If user-visible quota inodes are in use, they will be hidden
and converted to the reserved quota inodes.
Signed-off-by: Aditya Kali <adityakali@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Currently the bigalloc implementation in the kernel requires extents,
but this restriction might get relaxed in the future. Also, old
versions of mke2fs that supported bigalloc during early testing
created the root and lost+found directories without using
extent-mapped inodes. This makes it possible for e2fsck to better
support these old legacy file systems if it comes across them.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Use the EXT2_I_SIZE() macro consistently to access the inode size.
The i_size/i_size_high combination is open coded in several places.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Fix several types of compiler warnings (unused variables/labels),
uninitialized variables, etc that are hit with gcc -Wall.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
For file systems with 64-bit block numbers, we need to make sure we
correct the i_blocks_hi field as well as the i_blocks field when
setting it to the correct value.
Thanks to Justin Maggard for pointing this out.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This test, added to e2fsprogs-1.41.12, is backwards.
If EOFBLOCKS is set, then the size -should- be less than
the last physical block...
xfstests 013 caught this.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
In the case where s_first_data_block is 1, we need to explictly reject
an extent whose starting physical block is zero.
Thanks to Jiaying Zhang <jiayingz@google.com> for finding this bug.
Addresses-Google-Bug: #2573806
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If a corrupted file system causes us to want to delete an extent, and
that causes us to want to release a block in e2fsck pass #1, it would
be preferable if e2fsck didn't seg fault. This tends to get users
craky, as users are wont to do. :-)
Thanks to Dirk Reiners for reporting this bug:
e2fsck crashes fixing a corrupted 3.5 TB filesystem:
0x0000000000432002 in ext2fs_unmark_generic_bitmap (bitmap=0x0, bitno=623386749)
at gen_bitmap.c:183
183 if ((bitno < bitmap->start) || (bitno > bitmap->end)) {
(gdb) bt
bitno=623386749) at gen_bitmap.c:183
block=623386749) at ../../lib/ext2fs/bitops.h:319
inuse=-1) at alloc_stats.c:78
extent.c:1509
pb=0x7fffffffdfe0, start_block=0, ehandle=0x6dcf50) at pass1.c:1709
pb=0x7fffffffdfe0, start_block=0, ehandle=0x6dcf50) at pass1.c:1737
pctx=0x7fffffffe100, pb=0x7fffffffdfe0) at pass1.c:1842
block_buf=0x6c4330 "\373\212#") at pass1.c:1920
The source of the NULL bitmap is fs on stack frame 2:
(gdb) up 2
inuse=-1) at alloc_stats.c:78
78 ext2fs_unmark_block_bitmap(fs->block_map, blk);
Addresses-SourceForge-Bug: #2971800
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Justin reported that creating a 4T file with posix_fallocate led
to fsck errors:
e2fsck 1.41.10 (10-Feb-2009)
Pass 1: Checking inodes, blocks, and sizes
Inode 12, i_blocks is 8589935432, should be 840. Fix? yes
This looks like a 32-bit overflow.
commmit 8a8f36540b added handling of
the high i_blocks number, but we accumulate blocks in the num_blocks
field, and that's still just 32 bits.
Note: we don't need to expand max_blocks for now, that's only used
in the non-extents case, and those files have smaller max sizes.
I haven't been able to replicate the problem, oddly, but Justin
reports that this patch fixed his situation.
Reported-by: Justin Maggard <jmaggard10@gmail.com>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
There were a number of problems that were prompting the user whether
or not to ABORT, but then would abort regardless of whether the user
answered yes or no. Change those to be PROMPT_NONE, PR_FATAL.
Also, fix PR_1_RESIZE_INODE_CREATE so that it recovers appropriately
after failing to create the resize inode. This problem now uses
PROMPT_CONTINUE instead of PROMPT_ABORT, and if the user says, "no",
the code will abort.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Some kernels will crash if EOFBLOCKS_FL is set when it is it not
needed, and this if it is left set when it isn't needed, it is a sign
of a kernel bug.
Addresses-Google-Bug: #2604224
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This reverts commit 0ea910997b.
Since the Linux kernel now has support for the EXT4_EOFBLOCKS_FL flag
starting in 2.6.34, we don't need this workaround any more.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If a corrupted file system causes us to want to delete an extent, and
that causes us to want to release a block in e2fsck pass #1, it would
be preferable if e2fsck didn't seg fault. This tends to get users
craky, as users are wont to do. :-)
Thanks to Dirk Reiners for reporting this bug:
e2fsck crashes fixing a corrupted 3.5 TB filesystem:
0x0000000000432002 in ext2fs_unmark_generic_bitmap (bitmap=0x0, bitno=623386749)
at gen_bitmap.c:183
183 if ((bitno < bitmap->start) || (bitno > bitmap->end)) {
(gdb) bt
bitno=623386749) at gen_bitmap.c:183
block=623386749) at ../../lib/ext2fs/bitops.h:319
inuse=-1) at alloc_stats.c:78
extent.c:1509
pb=0x7fffffffdfe0, start_block=0, ehandle=0x6dcf50) at pass1.c:1709
pb=0x7fffffffdfe0, start_block=0, ehandle=0x6dcf50) at pass1.c:1737
pctx=0x7fffffffe100, pb=0x7fffffffdfe0) at pass1.c:1842
block_buf=0x6c4330 "\373\212#") at pass1.c:1920
The source of the NULL bitmap is fs on stack frame 2:
(gdb) up 2
inuse=-1) at alloc_stats.c:78
78 ext2fs_unmark_block_bitmap(fs->block_map, blk);
Addresses-SourceForge-Bug: #2971800
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This is the userspace side of Jiaying's EOFBLOCKS patch. With
Aneesh's patches for .33, Jiaying's patch, and this one, xfstests
013/fsstress (even with direct IO enabled) has held up through many
runs.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Pass 1 has a test to see if a special file is really a directory.
Signed-off-by: Nick Dokos <nicholas.dokos@hp.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The e2fsck_get_alloc_block() callback is used so that if the ext2fs
library needs to allocate blocks internally (most notably by the
extents functions), e2fsck's internal block usage map is consulted
since it is the only thing that can be trusted during a large part of
e2fsck's operation.
Change it to update the on-disk bitmap if it is loaded. This reduces
the number of spurious differences found in pass #5.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Directories are not allowed to be sparse; the code for scanning
extent-mapped directories was not calling ext2fs_add_dir_block() for
missing directory blocks, so we weren't catching this form of file
system corruption. Fix this.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The check that determines whether an directory needs to be have an
index added to it depends on i_size. So move it after we have fixed
up i_size so that we reliable will rehash a directory that needs it,
even if its i_size field was originally incorrect. Otherwise, a
second run of e2fsck would be needed before the directory gets an
index added.
Thanks to Mikulas Patocka for providing a sample file system which
demonstrated this problem.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The old method for detecting directories with holes depended on i_size
being correct, even though the correct value of i_size hadn't been
calculated yet. Hence, a directory inode with holes and an i_size of
0 would require two e2fsck passes to fix completely.
The replacement method for determining whether or not
ext2fs_add_dir_block() should be called is more reliable, and reduces
the size of e2fsck and makes the code more readable as a bonus.
Thanks to Mikulas Patocka for providing a sample file system which
demonstrated this problem.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
E2fsck was not properly printing the i_blocks field in filesystem
corruption messages, and it was not properly checking i_blocks_hi and
i_blocks_lo, either. This commit fixes this.
Thanks to Felipe Conteras for pointing this out.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If multiple blocks of a block group's inode table overlaps with other
file system blocks, only ask once for each block group.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If the filesystem feature FLEX_BG is enabled, the inode table and
bitmap blocks can be located anywhere in the inode table. So for
FLEX_BG filesystems, new_table_block() now tries allocate in the block
group's flex_bg first, and if there is no space in the local flex_bg,
then try to allocate from the whole filesystem.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Previously e2fsprogs interpreted 0 for a rec_len of 65536 (which could
occur if the directory block is completely empty in 64k blocksize
filesystems), while the kernel interpreted 65535 to mean 65536. The
kernel will accept both to mean 65536, and encodes 65535 to be 65536.
This commit changes e2fsprogs to match.
We add the encoding agreed upon for 128k and 256k filesystems, but we
don't enable support for these larger block sizes, since they haven't
been fully tested.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The patch below adds a function, ext2fs_extent_open2(), that behaves
as ext2fs_extent_open(), but will use the user-supplied inode
structure when opening an extent instead of reading the inode from
disk. It also changes several of the calls to extent_open() to use
this enhancement.
Signed-off-by: Nic Case <number9652@yahoo.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
in the case of ! defined RESOURCE_TRACK, so that we can clean up #ifdef
throughout e2fsck source.
Signed-off-by: Ken Chen <kenchen@google.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If ext2fs_extent_open() fails due to a corrupt extent header, and the
user declines to clear the inode, check_blocks_extents() should bail
out; otherwise, it will cause a core dump due a null pointer
dereference.
Addresses-Sourceforge-Bug: #2791794
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
A corrupted interior node in an extent tree would cause e2fsck to
crash with the error message:
Error1: Corrupt extent header on inode 107192
Aborted (core dumped)
Handle this and related failures when scanning an inode's extent tree
more robustly.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Report whether a fragmented inode is a directory or a file, as this is
highly useful for determining what is going on with an ext4 filesystem.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Track the number of non-contiguous files and directories so we can
give more detailed information in verbose mode.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Some of these could affect filesystems between 2^31 and 2^32-1 blocks.
Thanks to Valerie Aurora Henson for pointing out the problems in
lib/ext2fs/alloc_tables.c, which led me to do a "make gcc-wall" scan
over the source tree.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Some recent changes had caused diet libc support to bitrot. Fix up
missing header files and other portability fixups needed for dietlibc.
(Many of these changes also improve general portability.)
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The rec_len field in the directory entry is 16 bits, so if the
filesystem is completely empty, rec_len of 0 is used to designate
65536, for the case where the directory entry takes the entire 64k
block.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
For inodes with blocks preallocated with FALLOC_FL_KEEP_SIZE, e2fsck
complained about i_size being too small. Fix this.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
A misunderstanding C's precedence rules and the meaning of
s_log_block_size meant that we were capping the maximum size of
extent-based files at 8GB instead of the 64TB that it should be for
filesystems with 4k block sizes.
Addresses-Kernel-Bugzilla: #11341
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Also added support for "e2fsck -E fragcheck" which issues a
comprehensive report of discontiguous file extents.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This:
Truncating bigfile to 14680064000000
Pass 1: Checking inodes, blocks, and sizes
Inode 49154, i_size is 14680064000000, should be 0. Fix<y>?
is a bit unexpected. It's because the size is being checked against
the max sizes for bitmap files, not extent-based files.
Nick saw this with his 14TB file.
Patch below applies different size limits to the different file
formats.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The logical block numbers must be monotonically increasing, and there
must not be any overlapping extents. If any are found, report them as
filesystem corruption.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Wire up callback functions for ext2fs_alloc_block() and
ext2fs_block_alloc_stats() so that we use the ctx->block_found_map
block bitmap to determine which new block we should allocate, and then
to update the block_found_map bitmap if the extent functions need to
allocate or release blocks.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
ext2fs_extent_delete() will leave the extent handle pointing at the
next extent --- except if the last extent in the node. To deal with
this last case, call ext2fs_get_extent_info() and stop scanning after
processing info->num_entries extents.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
ext2fs_delete_extent() deletes the current extent and moves to the
next extent (if present). So we need to skip moving to the next
extent and get the (new) current extent and check it before moving on.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
pass1 was checking that an "extent's" start+len did not extend
past the last filesystem block, but unless we are at a leaf
block, the physical block is that of a node in the tree, and
the length may include sparseness. The test is only valid
for leaf blocks.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Extent data is shared with the i_block[] space in the inode,
but it is always swapped on access, not when the inode is read.
In e2fsck/pass1.c we must be careful when checking validity
of the extents flag on the inode. If the flag was set when
the inode was read & swapped, then the extents data itself
(in ->i_block[]) was NOT swapped, so testing for a valid
extent header requires some swapping first. Then, if we
ultimately set the extents flag, all of i_block[] must be
re/un-swapped.
This passes the f_extent regression test on both ppc & x86.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The ext2fs_extent_get() function was not OR-ing together UNINIT
and LEAF flags in the case where an extent was both; so if we
had an extent which was both uniint and leaf, pass1 would bail
out where depth == max_depth but was not marked as leaf, and
e2fsck (from the next branch) would abort with:
e2fsck 1.40.8 (13-Mar-2008)
Pass 1: Checking inodes, blocks, and sizes
Error1: No 'down' extent
Aborted
Also, if the error is encountered again, print the inode number
to aid debugging until it's properly handled, at least.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If a directory's i_size is bigger than the number of blocks, don't try
to allocate extra empty blocks to the end of the directory; there's no
real point to do that. Also, if a directory's i_size is not a
multiple of the blocksize, flag that as a mistake so it can be fixed.
This more elegantly addresses the problem which was found on Bas van
Schaik's filesystem.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add vertificaton of the in-inode EA information, and allow in-inode
EA's to have a checksum.
Signed-off-by: Andreas Dilger <adilger@clusterfs.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Factor out code to clear a bogus inode and update e2fsck's internal
data structures accordingly into a common routine,
e2fsck_clear_inode(). This saves about 200 bytes in the compiled x86
e2fsck executable, and makes the code more maintainable in the
long-term.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Avoid pointer cast and call e2fsck_write_inode_full() the same way
as check_inode_extra_space() does.
Signed-off-by: Dmitry V. Levin <ldv@altlinux.org>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Allow files to be preallocated on-disk up to the next multiple of the
system's page size without complaining about extra blocks.
Signed-off-by: Andreas Dilger <adilger@clusterfs.com>
Signed-off-by: Girish Shilamkar <girish@clusterfs.com>
Signed-off-by: Kalpak Shah <kalpak@clusterfs.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This patch instruments the libext2fs unix I/O manager and adds bytes
read/written and data rate to e2fsck -tt pass/overall timing output.
Signed-off-by: Jim Garlick <garlick@llnl.gov>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add an extra validity test in check_ext_attr(). If an attribute's
e_value_size is zero the current code does not allocate a region for it
and as a result the e_value_offs value is not verified. However, if
e_value_offs is very large then the later call to
ext2fs_ext_attr_hash_entry() can dereference bad memory and crash
e2fsck.
Signed-off-by: Andreas Dilger <adilger@clusterfs.com>
Signed-off-by: Jim Garlick <garlick@llnl.gov>
This patch removes a code snippet from check_ea_in_inode() in pass1 which checks
if the EA values in the inode are sorted or not. The comments in fs/ext*/xattr.c
state that the EA values in the external EA block are sorted but those in the
inode need not be sorted. I have also attached a test image which has unsorted
EAs in the inodes. The current e2fsck wrongly clears the EAs in the inode.
Signed-off-by: Kalpak Shah <kalpak@clusterfs.com>
Fix a typo which could cause e2fsck to throw an I/O error while doubling
checking whether or not a special device file was really an inode.
Also, don't do this tests on symbolic links since for filesystems with a
large numbers of symlinks it could degrade performance and increases the
risk for false positives.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add better ehandler_operation() markers so it is clearer what e2fsck was
doing when an I/O error is reported.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If e2fsck.conf configures a scratch_files directory which is available,
and the number of directories exceeds scratch_files.numdirs_threshold,
then try to use the tdb library to store the inode count abstraction.
This allows us to check very large filesystems without needing as much
physical memory.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This is probably only useful in artificial test cases, but it will be
useful if we ever do the "inodes in directory" idea for ext4.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Don't assume that a special device is bogus just because i_blocks is
non-zero. The i_blocks field could get adjusted later, and if this
happens it will confuse the e2fsck_process_bad_inode() in pass 2. In
practice true garbage inodes will have random non-zero in
i_blocks[4..15], so there's no point doing the check for an illegal
i_blocks value.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
- EXT4_FEATURE_RO_COMPAT_HUGE_FILE (0x0008) - change i_blocks to be
in units of s_blocksize units instead of 512-byte sectors, use
l_i_frag and l_i_fsize as i_blocks_hi (could also be part of 64BIT).
E2fsck and debugfs changed to support i_blocks_hi instead of l_i_frag and
l_i_fsize.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Create new ext2fs library inline functions in order to calculate
the starting and ending blocks in a block group.
Signed-off-by: Eric Sandeen <esandeen@redhat.com>
Don't do a structure copy via an assignment in e2fsck's pass #1 when
it is a no-op in order to avoid false positives from valgrind.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Eric Sandeen