Add a couple of tests to verify that writing to and recovering from
an external journal work properly.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Test e2fsck' recovery of commit blocks with (a) only a corrupt
checksum and (b) an obviously incorrect tid.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Test e2fsck' ability to deal with (a) revoke blocks with a bad
checksum and (b) revoke blocks with an obviously bad block number.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Test e2fsck' ability to deal with corrupt journal superblock checksum
and a bad magic.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Add tests to ensure that we know how to recover journals with the
csum_v2 feature set.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Test that we can write and replay transactions with the old journal
checksum algorithm.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Simple tests for the 32bit journal transaction creation code when
journal and metadata_csum are enabled. We test the following:
(a) writing and replaying transactions with multiple
descriptor blocks
(b) same, but with multiple revoke blocks.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Simple tests for the 64bit journal transaction creation code when
journal and metadata_csum are enabled. We test writing (bad) block
bitmaps out through the journal and replaying them via fsck, with a
few twists:
(a) All bitmaps are committed (fs errors reported)
(b) All the bitmap blocks are revoked (no errors)
(c) The transaction is never committed (no errors)
(d) Same as (a), but debugfs gets to do the replay.
We also test:
(a) writing and replaying transactions with multiple
descriptor blocks
(b) same, but with multiple revoke blocks.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Simple tests for the journal transaction creation code. We test
writing (bad) block bitmaps out through the journal and replaying them
via fsck, with a few twists:
(a) All bitmaps are committed (fs errors reported)
(b) All the bitmap blocks are revoked (no errors)
(c) The transaction is never committed (no errors)
(d) Same as (a), but debugfs gets to do the replay.
We also test:
(a) writing and replaying transactions with multiple
descriptor blocks
(b) same, but with multiple revoke blocks.
(c) adding the 64bit flag to a journal
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Display the feature flags of an external journal.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: TR Reardon <thomas_reardon@hotmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Spit out a more specific error if someone tries to modify an
external journal device.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: TR Reardon <thomas_reardon@hotmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Verify the (ext4) superblock checksum of an external journal device
and prompt to correct the checksum if nothing else is wrong with the
superblock.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Cc: TR Reardon <thomas_reardon@hotmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Enable mke2fs to create an external journal device with a superblock
checksum.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Cc: TR Reardon <thomas_reardon@hotmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
When creating a journal inode, check the return value from
block_iterate3() because otherwise we fail to capture errors such as
being unable to allocate an extent tree block, which leads to e2fsck
creating broken journals.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Also add the convenience macro $CLEAN_OUTPUT in test_config which can
be used to run the "sed -e $cmd_dir/filter.sed" command to clean up
e2fsprogs command output before comparing with the expected golden
output.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Add to ext2fs_symlink the ability to create inline data symlinks.
[ Modified by tytso to add more logging to the test script ]
Suggested-by: Pu Hou <houpu.hp@alibaba-inc.com>
Cc: Pu Hou <houpu.hp@alibaba-inc.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The following tests were using md5sum: i_e2image, u_mke2fs, and
u_tune2fs. Convert them to use crcsum for better portability (not all
environments have md5sum; some might have sha1sum instead :-)
For our purposes crcsum is quite sufficient.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
ext2fs_flush2() unconditionally writes the block group descriptors to
disk even if the underlying FS isn't marked dirty. This causes the
following error message on a fsck -n run:
e2fsck 1.43-WIP (09-Jul-2014)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Error writing block 2 (Attempt to write block to filesystem resulted in short write). Ignore error? no
Error writing block 2 (Attempt to write block to filesystem resulted in short write). Ignore error? no
Error writing file system info: Attempt to write block to filesystem resulted in short write
Since ext2fs_close2() only calls flush if the dirty flag is set,
modify e2fsck to exhibit the same behavior so that we don't spit out
write errors for a read only check.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Add a regression test to ensure that previous patches' fixes to e2fsck
do not revert.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If we come across an inode with the inline data and extents inode flag
set, try to figure out the correct flag settings from the contents of
i_block and i_size. If i_blocks looks like an extent tree head, we'll
make it an extent inode; if it's small enough for inline data, set it
to that. This leaves the weird gray area where there's no extent
tree but it's too big for the inode -- if /could/ be a block map,
change it to that; otherwise, just clear the inode.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Since fifo, socket, and device inodes cannot have inline data or
extents, strip off these flags if we find them.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Ensure that the various blobs in the in-inode EA region do not overlap.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Fix up the rest of the inline data code not to complain if there's no
EA, since it's possible that there's no EA because we're in the
process of creating an inline data file. Also, don't return an error
code when removing a nonexistent EA, because there's no reason to.
Furthermore, if we write less than 60 bytes of inline data, remove the
EA to avoid wasting space.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
In pass 3, convert the "delete files and re-run e2fsck" message to a
proper error code for more consistent error reporting and to make
translation easier.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
This test checks to make sure resize2fs can properly handle a file
system which started life as a normal ext4 file system and then was
grown to a size where meta_bg was enabled, and then shrunk back below
the point where the meta_bg format is still needed.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The test verifies that e2fsck can properly fix a file system where the
value of s_first_meta_bg in the superblock is larger than the number
of block group descriptors in the file system. E2fsck will fix this
by clearing the meta_bg feature.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Place the allocation bitmaps and inode table blocks so they are
adjacent, even in the last flexbg.
Previously, after running "mke2fs -t ext4 DEV 286720", the layout of
the last few block groups would look like this:
Group 32: (Blocks 262145-270336) [INODE_UNINIT, ITABLE_ZEROED]
Block bitmap at 262145 (+0), Inode bitmap at 262161 (+16)
Inode table at 262177-262432 (+32)
Group 33: (Blocks 270337-278528) [INODE_UNINIT, BLOCK_UNINIT, ITABLE_ZEROED]
Block bitmap at 262146 (bg #32 + 1), Inode bitmap at 262162 (bg #32 + 17)
Inode table at 262433-262688 (bg #32 + 288)
Group 34: (Blocks 278529-286719) [INODE_UNINIT, ITABLE_ZEROED]
Block bitmap at 262147 (bg #32 + 2), Inode bitmap at 262163 (bg #32 + 18)
Inode table at 262689-262944 (bg #32 + 544)
Now, they look like this:
Group 32: (Blocks 262145-270336) [INODE_UNINIT, ITABLE_ZEROED]
Block bitmap at 262145 (+0), Inode bitmap at 262148 (+3)
Inode table at 262151-262406 (+6)
Group 33: (Blocks 270337-278528) [INODE_UNINIT, BLOCK_UNINIT, ITABLE_ZEROED]
Block bitmap at 262146 (bg #32 + 1), Inode bitmap at 262149 (bg #32 + 4)
Inode table at 262407-262662 (bg #32 + 262)
Group 34: (Blocks 278529-286719) [INODE_UNINIT, ITABLE_ZEROED]
Block bitmap at 262147 (bg #32 + 2), Inode bitmap at 262150 (bg #32 + 5)
Inode table at 262663-262918 (bg #32 + 518)
This reduces the free space fragmentation in a freshly created file
system. It also allows the following mke2fs command to succeed:
mke2fs -t ext4 -b 4096 -O ^resize_inode -G $((2**20)) DEV 2130483
(Note that while this allows people to run mke2fs with insanely large
flexbg sizes, this is not a recommended practice, as the kernel may
refuse to resize such a file system while mounted, since it currently
tries to allocate an in-memory data structure based on the size of the
flexbg, and so a file system with a very large flexbg size will cause
the memory allocation to fail. This will hopefully be fixed in a
future kernel release, but if the goal is to force all of the metadata
blocks to be at the beginning of the file system, it's better to use
the packed_meta_blocks configuration parameter in mke2fs.conf.)
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Add regression tests to e2fsck to examine how it deals with inode
table blocks which (a) have been zero'd; (b) have been one'd; (c) have
corrupt inodes with obvious problems; and (d) have inodes with
non-obvious problems.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Add tests to examine how e2fsck deals with (a) the block bitmap being
corrupt; (b) the inode bitmap being corrupt; (c) the bitmap checksums
being incorrect (but the bitmaps are fine); and (d) the group
descriptor checksum itself is incorrect.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Add regression tests to examine how e2fsck deals with random
superblock corruption such as obviously wrong fields and the checksum
itself being incorrect.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Add regression tests to examine how e2fsck deals with MMP blocks with
(a) a bad magic number; and (b) an incorrect checksum.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Add some regression tests to examine how e2fsck handles directory
entry blocks and htree blocks with (a) malformed directory entries;
(b) incorrect checksums; or (c) obviously garbage entries.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Add some regression tests to examine how e2fsck deals with (a) extent
blocks with only a bad checksum; (b) extent blocks with a bad magic
number; and (c) extent entries with corruption.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Add regression tests for e2fsck dealing with (a) EA block with a bad
checksum; (b) EA block with a bad magic number; and (c) EA block with
damage that isn't otherwise noticeable.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If an inode fails checksum verification during pass 1 and the user
doesn't fix or clear the inode as part of the regular inode checks,
ensure that e2fsck remembers to ask the user if he simply wants to
correct the checksum.
We weren't capturing all the ways out of an interation of the inode
scanning loop, which means that not all errors were caught. Also,
we might as well clear the 'failed csum' flag if we write the inode
directly from the inode scanning loop.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If an inode fails checksum verification, don't stuff a copy of it in
the inode cache, because this can cause the library to fail to return
the "corrupt inode" error code.
In general, this happens if ext2fs_read_inode_full() is called twice
on an inode with an incorrect checksum. If fs->flags has
EXT2_FLAG_IGNORE_CSUM_ERRORS set during the first call and *unset*
during the second call, the cache hit during the second call fails to
return EXT2_ET_INODE_CSUM_INVALID as you'd expect. This happens
during fsck because the first read_inode call happens as part of
check_blocks and the second call happens during inode checksum
revalidation. A file system with a slightly corrupt non-extent inode
will trigger this.
While we're at it, make the inode read function consistent with the
rest of libext2fs -- copy the metadata object into the caller's buffer
even if it fails checksum verification. This will help e2fsck avoid a
double re-read later on down the line.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If we're totally unable to allocate a lost+found directory, ask the
user if he would like to dump orphaned files in the root directory.
Hopefully this enables the user to delete enough files so that a
subsequent run of e2fsck will make more progress. Better to cram lost
files in the rootdir than the current behavior, which is to fail at
linking them in, thereby leaving them as lost files.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The f_badcluster output format depends on how libreadline formats
and outputs the commands read from stdin. Instead of trying to
handle these differences, use an input command file, which does
not depend on external components to be consistent.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
This should have been part of commit 9a1d614df2 ("e2fsck: fix
rule-violating lblk->pblk mappings on bigalloc filesystems") but it
accidentally got dropped when the patch was applied.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Fix the routine that adds dirent checksum structures to the directory
block to handle oddball situations a bit more robustly.
First, when we're walking the entry array, we might encounter an
entry that ends exactly one byte before where the checksum entry needs
to start, i.e. there's space for the tail entry, but it needs to be
reinitialized. When that happens, we should proceed until d points to
that space so that the tail entry can be initialized.
Second, it's possible that we've been fed a directory block where the
entries end just short of the end of the block. In this case, we need
to adjust the size of the last entry to point exactly to where the
dirent tail starts. The current code requires that entries end
exactly on the block boundary, but this is not always the case with
damaged filesystems.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If we trash the root directory block, e2fsck will find inode 11 (the
old lost+found) and try to attach it to l+f. The lost+found checker
also fails to find l+f and tries to add one to the root dir. The root
dir is not found but is recreated with incorrect checksums, so linking
in the l+f dir fails and the l+f '..' entry isn't set. Since both
dirs now fail checksum verification, they're both referred to rehash
to have that fixed, but because l+f doesn't have a '..' entry, rehash
crashes because l+f has < 2 entries.
On a checksumming filesystem, the routines in e2fsck that recreate
/lost+found and / must write the new directory block *after* the inode
has been written to disk because the checksum depends on i_generation.
Add a regression test while we're at it.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If we think we're going to need to repair either the root directory or
the lost+found directory, reserve a block at the end of pass 1 to
reduce the likelihood of an e2fsck abort while reconstructing
root/lost+found during pass 3.
If / and/or /lost+found are corrupt and duplicate processing in pass
1b allocates all the free blocks in the FS, fsck aborts with an
unusable FS since pass 3 can't recreate / or /lost+found. If either
of those directories are missing, an admin can't easily mount the FS
and access the directory tree to move files off the injured FS and
free up space; this in turn prevents subsequent runs of e2fsck from
being able to continue repairs of the FS.
(One could migrate files manually with debugfs without the help of
path names, but it seems easier if users can simply mount the FS and
use regular FS management tools.)
[ Fixed up an obvious C trap: const char * and const char [] are not
the same thing when you are taking the size of the parameter.
People, run your regression tests! Like spinach, it's good for you. :-)
-- tytso ]
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Provide an API to set i_size in an inode and take care of all required
feature flag modifications. Refactor the code to use this new
function.
[ Moved the function to lib/ext2fs/blk_num.c, which is the rest of
these sorts of functions live, and renamed it to be
ext2fs_inode_size_set() instead of ext2fs_inode_set_size() to be
consistent with the other functions in in blk_num.c -- tytso ]
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
We rely on a nasty hack to adjust the free block count where we pass
signed value into ext2fs_free_blocks_count_add(), which takes an
64-bit unsigned value, and relies on overflow and C's signed->unsigned
semantics to do the subtraction. This works, so long as a 64-bit
signed value is used.
Unfortunately, ext2fs_block_alloc_stats2() and
ext2fs_block_alloc_stats_range(), this is not true, so on a 64-bit
file system, the free blocks accounting can get screwed up.
A simple way to demonstrate the problem is:
mke2fs -F -t ext4 -O 64bit /tmp/foo.img 1M
e2fsck -fy /tmp/foo.img
... which will result in the following e2fsck complaint:
Pass 5: Checking group summary information
Free blocks count wrong (4294968278, counted=982).
Fix? yes
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Port tune2fs' -e flag to mke2fs so that we can set error behavior at
format time, and introduce the equivalent errors= setting into
mke2fs.conf.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Directories can't have uninitialized extents, so offer to clear the
uninit flag when we find this situation. The actual directory blocks
will be checked in pass 2 and 3 regardless of the uninit flag.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Currently, directories cannot be fallocated, which means that the only
way they get bigger is for the kernel to append blocks one by one.
Therefore, if we encounter a logical block offset that is too big, we
needn't bother adding it to the dblist for pass2 processing, because
it's unlikely to contain a valid directory block. The code that
handles extent based directories also does not add toobig blocks to
the dblist.
Note that we can easily cause e2fsck to fail with ENOMEM if we start
feeding it really large logical block offsets, as the dblist
implementation will try to realloc() an array big enough to hold it.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Always iterate logical block 0 in a directory, even if no physical
block has been allocated. Pass 2 will notice the lack of mapping and
offer to allocate a new directory block; this enables us to link the
directory into lost+found.
Previously, if there were no logical blocks mapped, we would fail to
pick up even block 0 of the directory for processing in pass 2. This
meant that e2fsck never allocated a block 0 and therefore wouldn't fix
the missing . and .. entries for the directory; subsequent e2fsck runs
would complain about (yet never fix) the problem.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If we notice a hole in the block map of an extent-based directory,
offer to collapse the hole by decreasing the logical block # of the
extent. This saves us from pass 3's inefficient strategy, which fills
the holes by mapping in a lot of empty directory blocks.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
In the loop in ext2fs_get_free_blocks2, we ask the bitmap if there's a
range of free blocks starting at "b" and ending at "b + num - 1".
That quantity is the number of the last block in the range. Since
ext2fs_blocks_count() returns the number of blocks and not the number
of the last block in the filesystem, the check is incorrect.
Put in a shortcut to exit the loop if finish > start, because in that
case it's obvious that we don't need to reset to the beginning of the
FS to continue the search for blocks. This is needed to terminate the
loop because the broken test meant that b could get large enough to
equal finish, which would end the while loop.
The attached testcase shows that with the off by one error, it is
possible to throw e2fsck into an infinite loop while it tries to
find space for the inode table even though there's no space for one.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If we encounter an inode with IND/DIND/TIND blocks or internal extent
tree blocks that point into critical FS metadata such as the
superblock, the group descriptors, the bitmaps, or the inode table,
it's quite possible that the validation code for those blocks is not
going to like what it finds, and it'll ask to try to fix the block.
Unfortunately, this happens before duplicate block processing (pass
1b), which means that we can end up doing stupid things like writing
extent blocks into the inode table, which multiplies e2fsck'
destructive effect and can render a filesystem unfixable.
To solve this, create a bitmap of all the critical FS metadata. If
before pass1b runs (basically check_blocks) we find a metadata block
that points into these critical regions, continue processing that
block, but avoid making any modifications, because we could be
misinterpreting inodes as block maps. Pass 1b will find the
multiply-owned blocks and fix that situation, which means that we can
then restart e2fsck from the beginning and actually fix whatever
problems we find.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
When we're dumping a fast symlink inode, we print some odd things to
stdout. To clean this up, first don't print inline data EA, since the
inode dump doesn't display file and directory contents. Then, teach
the inode dump function how to print out either an inline data fast
symlink or a non-inline data fast symlink.
(This is a follow-up to the earlier patch "debugfs: Only print the
first 60 bytes from i_block on a fast symlink")
[ Modified by tytso so that the d_inline_dump test works when build
directory is different from the source directory --- i.e., when
doing a VPATH build. ]
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The -t option is documented as deprecated in GNU's mktemp, and
FreeBSD's mktemp doesn't support it at all.
Replace it with the construct "mktemp ${TMPDIR:-/tmp}/foo.XXXXXX"
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
In addition, incorporate the test name into the e2fsprogs-tmp to make
it easier to debug left-over temp files in the future.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
It can be made simpler because there is no need to differentiate between
having an internal journal inode and having an external journal device.
Signed-off-by: Benno Schulenberg <bensberg@justemail.net>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The default configuration still has quota disabled, but
runs the f_quota test unconditionally, so we fail by
default.
Fix that...
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The strptime() function does not update fields in struct tm that are
not specified in the input format. The glibc implementation sets the
tm_yday field (%j) when any of the year (%Y), month (%m), or day (%d)
fields are changed, but the MacOS strptime() does not set tm_yday in
this case. This caused string_to_time() to calculate the wrong Unix
epoch on MacOS. If tm_yday is unset, compute it in string_to_time().
This also fixes test regression failures for FreeBSD.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
We no longer need to reference https://ext4.wiki.kernel.org/index.php/Quota
since we've fixed the nasty bugs associated with e2fsck and the quota
feature. The wiki page will be updated once we've done a release that
includes these fixes indicated the verison which these problems have
been fixed.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Aditya Kali <adityakali@google.com>
The logdump command doesn't know how to deal with revoke tables in
64bit journals, so teach it to do this.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Fix some minor bugs relating to passing CFLAGS to cppcheck, and
package the cppcheck output into nicer looking reports.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The previous commit exposed bugs in the calculation for flex_bg file
systems. The problem is that since (by default) we keep the metadata
blocks for the flex_bg in the first block group of the flex_bg, and
because we don't want to overwrite metadata blocks used by the
original file system with data blocks make life easier in case the
resize is aborted for some reason, we need to treat all of the
metadata blocks in the existing flex_bg has in use for the purposes of
calculate_minimum_resize_size().
Even though this means we need to reserve more data blocks to avoid
running out of space, the net result of these two commits is a net
savings in how much we can shrink a file system.
Using the following test sequence:
mke2fs -F -t ext4 /tmp/foo.img 2T
resize2fs -M /tmp/foo.img
resize2fs -M /tmp/foo.img
resize2fs -M /tmp/foo.img
Here is the comparison in the resulting file systems between the old
and new resize2fs (units are in 4k blocks):
resize #1 resize #2 resize #3
old resize2fs 1117186 45679 43536
new resize2fs 48784 37413 37392
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Old distros may not have the "truncate" tool, so use "dd" instead.
If tmpfs cannot handle a 2GB temp file (e.g. old RHEL5 and SLES 11
kernels) then skip the test instead of failing it. If this fails,
try to report better error messages instead of failing silently.
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
When pass1 finds blocks that are mapped to multiple files, it will
print every duplicated block. If there are long sequences of
duplicate blocks (e.g. the e_pblk field is wrong in an extent), this
can cause a gigantic flood of output when a range could convey the
same information. Therefore, teach pass1b to print ranges when
possible.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Enhance debugfs to be able to display and modify extended attributes, and
create some simple tests for the extended attribute editing functions.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
In this test, inode flag is some random data, and after we apply inline
data patch set we should need to handle it.
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Use the new extended attribute APIs to display all extended attributes
(current code does not look in the EA block) and display full names
(current code ignores name index too).
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
To check the coverage of e2fsprogs's regression test, do the
following:
configure --enable-gcov
make -j8 ; make -j8 check ; make coverage.txt
The coverage information will be the coverage.txt and *.gcov files in
the build directories.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The "mkswap" program is not available on MacOS, so just use the
existing swap0.img.bz2 and swap1.img.bz2 files directly.
Because MacOS HFS+ doesn't support sparse files (welcome to the 80's)
the m_bigjournal test takes forever to zero out the whole 42GB test
filesystem. Skip this test for Darwin kernels for now.
Unfortunately, neither "df -T" nor "stat -f -c %T" is available on
MacOS to directly determine the filesystem type, and I'm too lazy
to parse the output of "mount" and match it to the path of the test
directory in shell, so it just checks the kernel type and assumes
the filesystem type is HFS and skips the test.
Since this test runs on Linux the majority of the time, the loss of
test coverage is minimal. If MacOS should ever get a real filesystem,
this can be revisited.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Now that libext2fs marks group metadata in the fs block bitmap, adjust
the expected test output to reflect expanded use of block_uninit and
the fact debugfs no longer prints block bitmap data that fails to
account for group data blocks.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The dietlibc doesn't support the TZ environment variable, which is
required by the standard. Work around this so that we can run the
regression test suite when building with dietlibc. (This is useful
for finding problems.)
With this change, the only thing which doesn't work as far as dietlibc
is concerned is the posix_memalign test, and the MMP support tests
(because posix_memalign isn't provided by dietlibc, sigh.)
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Newer versions of autoconf pull in AC_PROG_GCC as part of
AC_CANONICAL_HOST. So we need check for WITH_DIET_LIBC earlier in
configure.in.
Also, e2fsprogs now needs functions which are found in diet libc's
compat library. So add support for autoconf's LIBS function, and
automatically set libs to include -lcompat.
Finally, disable compiling e4defrag by deault if --with-diet-libc is
specified because the program has too many glibc dependencies.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Previously, this message used 8193 as the example alternate
superblock. But for most file systems, the backup superblock is
located at 32768 (since most file systems have a block size of 4k, and
not 1k).
Addresses-Debian-Bug: #719185
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If given at least one offset and only one file, assume source
and dest are the same, and do an in place move.
Signed-off-by: Phillip Susi <psusi@ubuntu.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Fix the f_extent_oobounds test so that it uses binaries built in the
tree, instead of the binaries in the system PATH (which might not
exist in a chroot environment) when creating the test image.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The s_desc_size in the superblock specifies the group descriptor
size in bytes, but in various places the EXT4_FEATURE_INCOMPAT_64BIT
flag implies that the descriptor size is EXT2_MIN_DESC_SIZE_64BIT
(64 bytes) instead of checking the actual size. In other places,
the s_desc_size field is used without checking for INCOMPAT_64BIT.
In the case of ext2fs_group_desc() the s_desc_size was being ignored,
and assumed to be sizeof(struct ext4_group_desc), which would result
in garbage for any but the first group descriptor. Similarly, in
ext2fs_group_desc_csum() and print_csum() they assumed that the
maximum group descriptor size was sizeof(struct ext4_group_desc).
Fix these functions to use the actual superblock s_desc_size if
INCOMPAT_64BIT.
Conversely, in ext2fs_swap_group_desc2() s_desc_size was used
without checking for INCOMPAT_64BIT being set.
The e2fsprogs behaviour is different than that of the kernel,
which always checks INCOMPAT_64BIT, and only uses s_desc_size to
determine the offset of group descriptors and what range of bytes
to checksum.
Allow specifying the s_desc_size field at mke2fs time with the
"-E desc_size=NNN" option. Allow a power-of-two s_desc_size
value up to s_blocksize if INCOMPAT_64BIT is specified. This
is not expected to be used by regular users at this time, so it
is not currently documented in the mke2fs usage or man page.
Add m_desc_size_128, f_desc_size_128, and f_desc_bad test cases to
verify mke2fs and e2fsck handling of larger group descriptor sizes.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Since commit 5ad07acad if $TMP cannot hold large test filesystems
for resize testing the resize_test creates temporary test files
in the local working directory. Since it overrides TMPFILE locally
the calling program does not delete the generated file correctly.
Delete the large $TMPFILE within resize_test if it passes, but leave
it for debugging if the test failed.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The new resize tests create 2TB test files, but tmpfs in kernels
before 3.1 have a max file size of 256GB. Ext3 may also have
a size limit for smaller blocksize filesystems.
Fix the resize_test script to verify that $TMPFILE can be resized
to the final test size, and if that fails try creating the file on
the local filesystem instead of in $TMPDIR. If that cannot hold
the large filesystem, skip the test.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Tested-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Update all superblock copies when disabling the quota feature.
Added basic tests for the quota feature.
Signed-off-by: Niu Yawei <niu@whamcloud.com>
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Run sparse against source files when building e2fsprogs with 'make C=1'. If
instead C=2, it configures basic ext2 types for bitwise checking with sparse,
which can help find the (many many) spots where conversion errors are
(possibly) happening.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Eric Sandeen reported that Fedora's mke2fs when compiled for ppc was
creating a file system which caused problems with resize2fs -M.
Closer examination showed that the problem was file system which
looked like this:
Filesystem features: ext_attr dir_index filetype sparse_super
Inode count: 512
Block count: 1247
...
Group 0: (Blocks 1-1024)
Primary superblock at 1, Group descriptors at 2-2
Block bitmap at 66 (+65), Inode bitmap at 67 (+66)
Inode table at 68-99 (+67)
Group 1: (Blocks 1025-1246)
Backup superblock at 1025, Group descriptors at 1026-1026
Block bitmap at 1090 (+65), Inode bitmap at 1091 (+66)
Inode table at 1092-1123 (+67)
It's not obvious to me why Fedora's ppc mke2fs is creating file
systems like this (I can't reproduce this on debian ppc systems), but
resize2fs -M should be able to deal with such file systems, which is
what this test is designed to check.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Commit d3f32c2db8 was intended to detect extents found outside their
proper location in the extent tree, including invalid extents at the
end of an extent block. However, it incorrectly reported legal
uninitialized extents created by fallocate() at the end of file with
the FALLOC_FL_KEEP_SIZE flag as false positives. xfstests
generic/263 (among others) caught this problem, while the e2fsprogs
test f_uninit_ext_past_eof did not. The latter test failed to
detect the problem in part because it uses a test file whose i_size
is 0.
Add a test derived from the fsx-based test case in xfstests
generic/263 consisting of a file with non-zero length, more than
four extents total, and two uninitialized extents past EOF to
reliably reproduce commit d3f32c2db8's false positive behavior.
Signed-off-by: Eric Whitney <enwlinux@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The function deallocate_inode() in e2fsck/pass2.c was buggy in that it
would clear out the inode's mode and flags fields before trying to
deallocate any blocks which might belong to the inode.
The good news is that deallocate_inode() is mostly used to free inodes
which do not have blocks: device inodes, FIFO's, Unix-domain sockets.
The bad news is that if deallocate_inode() tried to free an invalid
extent-mapped inode, it would try to interpret the root of the extent
node as block numbers, and would therefore mark various file system
metadata blocks (the superblock, block group descriptors, the root
directory, etc.) as free and available for allocation. This was
unfortunate.
(Try running an older e2fsck against the test file system image in the
new test f_invalid_extent_symlink, and then run e2fsck a second time
on the fs image, and weep.)
Fortunately, this kind of file system image corruption appears to be
fairly rare in actual practice, since it would require a very unlucky
set of bits to be flipped, or a buggy file system implementation.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
E2fsck was missing a check for directories with logical blocks so
large that i_size > 2GB. Without this check the test image found in
the new test f_toobig_extent_dir will cause e2fsck to die with a
memory allocation failure:
Error storing directory block information (inode=12, block=0, num=475218819): Memory allocation failed
e2fsck: aborted
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reported-by: Andrey Melnikov <temnota.am@gmail.com>
The environment variable EXT2FS_NO_MTAB_OK will suppress the error
code EXT2_NO_MTAB_FILE when the /etc/mtab file can not be found. This
allows the e2fsprogs regression test suite to be run in chroots which
might not have an /etc/mtab file.
By default will still want to complain if the /etc/mtab file is
missing, since we really don't want to discourage distributions and
purveyors of embedded systems from running without an /etc/mtab file.
But if it's missing it only results in a missing sanity check that
might cause file system corruption if the file system is mounted when
programs such as e2fsck, tune2fs, or resize2fs is running, so there is
no potential security problems that might result if this environment
variable is set inappropriately.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Don't change the root directory's UID/GID automatically just because
mke2fs was run as a non-root user. This can be confusing for users,
and is not flexible for non-root installation tools that need to
create a filesystem with different ownership from the current user.
Add the "-E root_owner[=uid:gid]" option to mke2fs so that the user
and group can be explicitly specified for the root directory. If
the "=uid:gid" argument is not specified, the current UID and GID
are extracted from the running process, as was done in the past.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
A minor cleanup to order the command-line option parsing in
alphabetical order, except for "-E" and "-R", which need to
be co-located.
Print a message that the "-R" option is deprecated. It has
been deprecated since 2005 (commit c6a44136b9).
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The sed filters for test outputs that are used to remove build and
test specific information (such as version strings, dates, times,
UUIDs) were unconditionally deleting the first line of output. This
would normally contain the tool version string, but in some cases
contained other information that was being lost. This can lead to
difficulty debugging test failures.
The sed filtering has been changed to only remove the actual version
strings. As well, similar filter strings were duplicated throughout
many scripts, and "sed" and "tr" were often called multiple times in
a pipeline. These have been consolidated into a single filter.sed
file to avoid having to maintain these filters in multiple places.
In a few cases, accidentally deleted messages have been restored to
the expect output for the tests. In other cases, trivial whitespace
has been changed in the expect files.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
This tests creates a file system where the last entry in one leaf
block overlaps with logical block range in the first entry of the next
leaf block.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
E2fsck previously was complaining with zero-length extended attributes
if they appeared in the in-inode xattr space. Test to make sure
e2fsck is now happy with such xattrs.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The only checksum program which we can reliably count upon being
installed on all systems is "sum", which is not a particular robust
checksum. The problem with using md5sum or sha1sum is it hat it may
not be installed on all systems. So create a crcsum program which is
used so we can validate that a data file on a resized file system has
not been corrupted.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Throttle updates for the "Allocating Groups" progress updates to once
a second as well. We now do this throttling in libext2fs, so we don't
have to do this for each of mke2fs's progress updates, and because the
updates from ext2fs_allocate_tables() come from within libext2fs
anyway.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If the f_mmp test is interrupted during its test run, then it can
leave debugfs busy-looping in the background. Since f_mmp is a
relatively long-running test, and is likely to be running during
a parallel test run, this can happen fairly often.
Set a signal trap for the f_mmp test script being killed, so that
the background debugfs command will always be killed by the test.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Add a configuration knob so the regression tests can disable progress
reporting. This fixes a potential lack of predictability since the
progress reports are now time based (once a second) which is
problematic for regression tests which are comparing the expected
output of mke2fs.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The MMP tests need to be run on a real disk instead of tmpfs, since
the MMP block access is using O_DIRECT. As such, they create their
own test files in the local testing directory instead of using the
temporary file created in /tmp by the test_one script. Delete the
tempfs file before clobbering TMPFILE, otherwise it will leave the
unused file in /tmp after the test is completed.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The "mktemp" program requires a template on OS/X. Allow the test
TMPFILE to be created in the local /tmp directory for both OS/X
and Linux.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Add two tests, f_orphan_indirect_inode, and f_orphan_extents_inode,
which tests the bug fixes in the two previous commits:
e2fsck: update global free blocks/inodes count when truncating orphan inodes
libext2fs: fix block iterator for extents when truncating inodes
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong