The "mktemp" program requires a template on OS/X. Allow the test
TMPFILE to be created in the local /tmp directory for both OS/X
and Linux.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Add two tests, f_orphan_indirect_inode, and f_orphan_extents_inode,
which tests the bug fixes in the two previous commits:
e2fsck: update global free blocks/inodes count when truncating orphan inodes
libext2fs: fix block iterator for extents when truncating inodes
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If all of the extents in the last extent tree block (ETB) in a
non-trivial extent tree contain uninitialized extents which are after
the end of the file as defined by i_size, the hueristics will
incorrectly estimate the last entry (and hence the node's e_len field)
in the last entry of each level of the extent tree.
As Tao Ma has noted, since e2fsck was requiring that the length
(e_len) field of interior nodes be non-zero, this was causing false
failures where e2fsck would declare that the extent tree was
corrupted.
This was fixed in commit 9c40d14841: "e2fsck: only check for
zero-length leaf extents". Add a regression test case to ensure that
this issue remains fixed.
Signed-off-by: Tao Ma <boyu.mt@taobao.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Change the e2fsck/mke2fs/tune2fs/e2image/debugfs regression tests to
be driven by Makefile rules instead of by a script loop. This allows
the tests to be run in parallel like a build and reduces testing time
significantly.
One major change to the tests themselves is to printing the test name,
description, and status together after the test has passed or failed,
to avoid mixing lines from the tests. The other major change is to
use unique temporary filenames for each test, which was mostly handled
already via b4db1e4c74, but in some
cases temporary files are changed to use $test_name.tmp to avoid any
collision between running tests.
On my old 2-CPU system it reduced the testing time from 160s to 40s.
Much of the savings is from the MMP test delays running in parallel.
It still takes the time of the slowest test, f_mmp_garbage, though
there will be ongoing benefit in the future as more tests are added
since the wallclock time will not increase linearly for each test.
Tests were run with various combinations of "make -j", and "make -j2"
through "make -j44" repeatedly without any test failures.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The previous image did not have a 64-bit journal so it wouldn't
properly test the 64-bit journal format.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
These break the self-test suite on *BSD, and on some Linux distros
where /bin/sh is not implemented by bash.
Signed-off-by: Matthias Andree <matthias.andree@gmx.de>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Add regression tests which make sure e2fsprogs understands the current
32-bit and 64-bit journal format. If a patch breaks the on-disk
format, these tests should warn us about that.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Since we have changed the eofblocks handling in e2fsck (by removing
the need for the EOFBLOCKS_FL flag), we should have a test to check
out how we handle uninitialized and initialized blocks which are exist
beyond i_size, with files that have (and don't have) the EOFBLOCKS_FL.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
We've decided to remove EOFBLOCKS_FL from the ext4 file system entirely,
because it is not actually very useful and it is causing more problems
than it solves. We're going to remove it from e2fsprogs first and then
after the new e2fsprogs version is common enough we can remove the
kernel part as well.
This commit changes e2fsck to not check for EOFBLOCKS_FL. Instead we
simply search for initialized extents past the i_size as this should not
happen. Uninitialized extents can be past the i_size as we can do
fallocate with KEEP_SIZE flag.
Also remove the EXT4_EOFBLOCKS_FL from lib/ext2fs/ext2_fs.h since it is
no longer needed.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Print the actual errors returned by ext2fs_open2() and
ext2fs_check_desc() before we fall back to the backup block group
descriptors so that it's easier to see if there is some obscure
failure that is causing e2fsck to think that it should use the backup
block group descriptors.
Addresses-Google-Bug: #6208183
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The "make testnew" target in the tests/ subdirectory (used to
help create new test cases) should use the locally-built mke2fs
binary instead of the installed system binary, to avoid cases
where the system binary does not support some new functionality
being tested.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If an extent has e_len set to zero, the kernel will oops with a
BUG_ON. Unfortunately, e2fsck wasn't catching this case. The kernel
needs to be fixed to notice this case and call ext4_error() instead of
failing an assertion check, but e2fsck should catch this case and
repair it (by deleting the errant extent).
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If we have to read the backup group descriptor checksums, the UNINIT
flags are cleared to ensure that all of the inodes in the filesystem
are scanned. However, the code that reset the UNINIT flags did not
reset the group checksum, and this produced many spurious error
messages in e2fsck.
Group descriptor 0 checksum is invalid. FIXED.
Group descriptor 1 checksum is invalid. FIXED.
:
:
Recompute checksums after modifying group descriptors to avoid these
error messages. Remove expected error messages in f_illitable_flexbg.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The function ext2fs_get_pathname() used to return EXT2_ET_NO_DIRECTORY
if one of the directories in an inode's pathname is not a directory.
This is not very useful in an emergency, when the file system is
corrupted. This commit will cause ext2fs_get_pathname() to return a
partial pathname, which should help system administrators trying to
use debugfs to investigate a corrupted file system.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Several of the mmp tests were failing on an s390 guest because
direct IO cannot be done on sub-sector sizes, and they were
doing 1k IOs to files on a 4k logical/physical device.
A few tests could just be changed to 4k, but others
needed fixed up output as well.
With this I have all tests passing on s390 again.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The ext2fs_file_acl_block() and ext2fs_set_file_acl_block() needs to
only check i_file_acl_high if the 64-bit flag is set. This is needed
because otherwise we will run into problems on Hurd systems which
actually use that field for h_i_mode_high.
This involves an ABI change since we need to pass ext2_filsys to these
functions. Fortunately these functions were first included in the
1.42-WIP series, so it's OK for us to change them now. (This is why
we have 1.42-WIP releases. :-)
Addresses-Sourceforge-Bug: #3379227
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If e2fsck modifies certain superblock fields which the kernel doesn't
look at, mark the superblock as dirty without marking the file system
as changed. This will avoid e2fsck signalling the init scripts that a
reboot is necessary. This is safe, because the kernel doesn't
actually look at these superblock fields.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
In some cases the bad block inode gets corrupted. If it looks insane,
offer to clear it before trying to interpret it does more harm than
good.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add tests for the MMP feature - creating a filesystem with mke2fs
and MMP enabled, enable/disable MMP with tune2fs, disabling the
e2fsck MMP flag with tune2fs after a failed e2fsck, and e2fsck
checking and fixing a corrupt MMP block.
The MMP tests need to be run from a real disk, not tmpfs, because
tmpfs doesn't support O_DIRECT reads, which MMP uses to ensure
that reads from the MMP block are not filled from the page cache.
Using a local disk does not slow down the tests noticably, since
they wait to detect if the MMP block is being modified.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Continue to remove the external journal device even if the device
cannot be found.
Add a test to verify that the journal device/UUID are actually removed
from the superblock. It isn't possible to use a real journal device
for testing without loopback devices and such (it must be a block device)
and this would invite complexity and failures in the regression test.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The DEFS line in MCONFIG had gotten so long that it exceeded 4k, and
this was starting to cause some tools heartburn. It also made "make
V=1" almost useless, since trying to following the individual commands
run by make was lost in the noise of all of the defines.
So fix this by putting the configure-generated defines in lib/config.h
and the directory pathnames to lib/dirpaths.h.
In addition, clean up some vestigal defines in configure.in and in the
Makefiles to further shorten the cc command lines.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Currently we need to grep, list or just search for failed tests when
running 'make check' which is annoying. This commit simply prints out
the list of failed test names at the end of the output.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If e2fsprogs tools (mke2fs, e2fsck) is run on regular file instead of
on block device, we can use punch hole instead of regular discard
command which would not work on regular file anyway. This gives us
several advantages. First of all when e2fsck is run with '-E discard'
parameter it will punch out all ununsed space from the image, hence
trimming down the file system image. And secondly, when creating an
file system on regular file (with '-E discard' which is default), we
can use punch hole to clear the file content, hence we can skip inode
table initialization, because reads from sparse area returns zeros. This
will result in faster file system creation (without the need to specify
lazy_itable_init) and smaller images.
This commit also fixes some tests that would fail due to mke2fs showing
discard progress, hence the output would differ.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
We are doing ext2fs_flush() twice right now at the end of the mke2fs.
First by directly calling ext2fs_flush() which is intended to write
superblock and fs accounting information. And then it is invoked again
when we are calling ext2fs_close(), only this time, because the fs is
not dirty, we are writing out only superblock.
I think it is bad to call it twice because even when writing only super
block it takes some time on bigger file systems and moreover
ext2fs_close() can fail without any reasonable explanation for the user.
Also ext2fs_flush() is printing out progress and it is confusing for the
users.
Fix all this by removing the ext2fs_flush() and leaving it all to
ext2fs_close(). However we need to introduce new variables to store
check interval and max mount count, because fs structure is freed on
ext2fs_close() and we really want to print those information as the last
info for the user.
[ Fixed type mismatch in a printf format statement -tytso]
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Since e2image can be optionally compiled out, we tested to see if
e2image was built; but using "test -x $E2IMAGE" fails if e2image is
something like "valgrind --simhints=lax-ioctls ../misc/e2image".
Define and use $E2IMAGE_EXE, much like we have done with e2undo and
resize2fs.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The /tmp directory is often a memory based file system, and using this
can speed up running the regression test suite.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Commit 2a77a784a3 (firest released in e2fsprogs 1.33) compared
superblock summary free blocks and inode counts with the allocation
bitmap counts before starting the file system check proper, and if
they differed, set the superblock and marked it as dirty. If no other
file systme changes were required, this would cause a "*** FILE SYSTEM
WAS MODIFIED ***" message without any explanation of what e2fsck had
changed.
We fix this by only setting the superblock summary free block/inodes
counts if we are skipping a full check, and in non-preen mode, e2fsck
will now print an explicit message stating how the superblock had been
updated.
In a full check, any updates to the superblock free blocks/inodes
fields will be noted in pass5.
This change requires changing a few test results (essentially
reversing the changes made in commit 2a77a784a3).
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Ignore files generated by "make" or "make check" in "git status".
Ignore backup files from editing files and generated tags files.
Delete a temporary file in tests/d_loaddump/script at test cleanup.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
All of the regression tests in e2fsprogs still use a block-mapped
journal (if any journal at all). Add a simple regression test that
tests extent-mapped journals for both mke2fs and e2fsck.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Add test for internal journal over 4GB in size, using the
lazy_journal_init and lazy_itable_init features. Otherwise
the filesystem metadata would be too large to reliably run on
test systems, and take too long to create/check the filesystem.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
This commit adds new regression test called i_e2image which should
validate expected behaviour of e2image raw and qcow2 image creation
and qcow2 -> raw image conversion. You can run it with "make check" as
the rest of regression tests.
Testing is performed on three filesystem images with different block
sizes (1024, 2048, 4096). It creates raw and qcow2 images from the
original and then convert qcow2 image back to raw image. The results are
checksummed and compared with pre-prepared results. All md5sums should
stay the same and raw image created from original and qcow2 image should
be the same as well (just for the record).
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
This patch adds support for specifying 'reserved_ratio' (percent blocks
reserved for super user, same as '-m' command line option) in mke2fs.conf.
It adds profile_get_double function in profile.c that allows reading
floating point values from profile files.
Signed-off-by: Aditya Kali <adityakali@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
User namespace xattrs are generally useful, and I think extN
is the only filesystem requiring a special mount option to
enable them, when xattrs are otherwise available. So this
change sets that mount option into the defaults, via a
mke2fs.conf option.
Note that if xattrs are config'd off, this will lead to a
mostly-harmless:
EXT4-fs (sdc1): (no)user_xattr options not supported
message at mount time...
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The forced fsck often comes at unexpected and inopportune moments,
and even enterprise customers are often caught by surprise when
this happens. Because a filesystem with an error condition will
be marked as requiring fsck anyway, I submit that the time-based
and mount-based checks are not particularly useful, and that
administrators can schedule fscks on their own time, or tune2fs
the enforced intervals if they so choose. This patch disables the
intervals by default, and I've added a new mkfs.conf option to
turn on the old behavior of random, unexpected, time-consuming
fscks at boot time. ;)
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If the file system has a blocksize less than 64k, then don't use the
extended rec_len encoding, to be consistent with what the kernel will
do.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The getopt() function returns an int, not a char. On systems where the
default char is unsigned (like ppc), we get weird behavior where -1 is
truncated to 0xff but compared to (int)-1.
Also fix this same bug for two test programs, test_rel and iscan,
which aren't currently used at the moment.
Addresses-Gentoo-Bug: #299386
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Use a standard, fixed mke2fs.conf file so that if downstream
distributions want to change the mke2fs.conf which is distributed in
the RPM or dpkg file, it won't screw up the regression tests.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add test to make sure e2fsck doesn't end up detaching and sending to
lost+found inodes which are in an bg_itable_unused_region (e.g. if
there was a power failure and bg_itable_unused wasn't updated because
journaling wasn't enabled).
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Some kernels will crash if EOFBLOCKS_FL is set when it is it not
needed, and this if it is left set when it isn't needed, it is a sign
of a kernel bug.
Addresses-Google-Bug: #2604224
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This tests e2fsck's handling of both direct and extent-mapped
directoris containing holes when the i_size is 0.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Duplicate directory entries were not necessarily getting found and
fixed for non-indexed directories, since we were sorting these
directories by inode number, and the duplicate entry code assumed the
entries were getting sorted by name or directory name hash.
Addresses-Sourceforge-Bug: #2862551
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Non-expert users get confused when they see messages like this:
Illegal block #-1 (2291965952) in inode 176. CLEARED.
So change it to be something a little bit more understandable:
Illegal indirect block (2291965952) in inode 176. CLEARED.
Addresses-SourceForge-Bug: #2871782
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The previous commit adds a new line printed by mke2fs with the
filesystem/device topology information. Update the regression test's
expected output to account for this.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
With 64-bit file systems, mke2fs can take a long time to do things
other than write inode tables. I exported the mke2fs numeric progress
meter and used it for allocating group tables and the final file
system flush.
Signed-off-by: Valerie Aurora (Henson) <vaurora@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Some people don't want to see the concise "kernel-style" make output.
This configure option allows build engines that want to see the full
set of commands executed by the makefile to get what they want. Most
people will find this more distracting than useful, unless they need
to debug the Makefiles.
(It is not necessary to rerun configure to enable this verbose make
output temprarily; if a developer wants to do a quick debug of a
directory's makefile, he or she can simply edit the definition of the
$(E) and $(Q) variables in the Makefile; instructions can be found in
the MCONFIG file which is included in at the beginning of every
Makefile.)
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The e2fsprogs makefiles were using the same Makefile variable
LIBCOM_ERR for the link-line arguments as well as the dependencies.
Since LIBCOM_ERR can now include non-file arguments such as
"-lpthread", we need to use a separate DEPLIBCOM_ERR variable that
only has build file dependencies.
Do the same thing for STATIC_LIBCOM_ERR and PROFILED_LIBCOM_ERR.
Addresses-Sourceforge-Patches: #2813809
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Previously e2fsprogs interpreted 0 for a rec_len of 65536 (which could
occur if the directory block is completely empty in 64k blocksize
filesystems), while the kernel interpreted 65535 to mean 65536. The
kernel will accept both to mean 65536, and encodes 65535 to be 65536.
This commit changes e2fsprogs to match.
We add the encoding agreed upon for 128k and 256k filesystems, but we
don't enable support for these larger block sizes, since they haven't
been fully tested.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The Lifetime writes field will probably not be stable as we add new
features to e2fsprogs, so filter it out to avoid spurious test failures.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Gcc is too stupid to realize that:
const char *usage="String which has no percent signs";
com_err(progname, 0, usage);
is OK. I refuse to bow to stupidity with:
com_err(progname, 0, "%s", usage);
but I will use the string directly for the sake of people who like to
build with -Werror=format-security.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The ext4 filesystem uses journals too, so remove "ext3" from the
problem descriptions involving journals.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This test case is designed to test e2fsck's ability to deal with a
corrupted interior node in an extent tree.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If the directory is packed with no slack space, as soon as any new
directory entries are added, leaf nodes end up getting split and
directory ends up getting very inefficient.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Some recent changes had caused diet libc support to bitrot. Fix up
missing header files and other portability fixups needed for dietlibc.
(Many of these changes also improve general portability.)
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Fix error message to print the depth of a corrupt htree directory.
Signed-off-by: Andreas Dilger <adilger@sun.com>
Signed-off-by: Kalpak Shah <kalpak.shah@sun.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Also added support for "e2fsck -E fragcheck" which issues a
comprehensive report of discontiguous file extents.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
On Solaris setbuf() will discard any pending output to the stream, so
make we call fflush() before calling setbuf().
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
E2fsck could to do more damage to a filesystem by trying to relocate
inode tables due to corrupted block group descriptors, and the
relocation could seriously damage the filesystem.
This patch enhances ext2fs_check_desk() so it detects more
self-inconsistent block group descriptors, including the cases where
e2sck might be tempted to relocate the inode table, and reports the
block group descriptors as invalid; this will cause e2fsck to attempt
to use the backup superblocks, which hopefully have not been trashed.
Addresses-Sourceforge-Bug: #1840291
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
When calculating the number reserved blocks, use floating point for
better accuracy, since for big filesystems it really makes a
difference. In addition, mke2fs and tune2fs accepts a floating point
number from the user, so they should provide that level of accuracy.
Addresses-Debian-Bug: #452639
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
With the new mke2fs changes the output of the
command differs if we run mke2fs on a device that
already have the file system. So erase the file system
before running mke2fs so that output remain as expected.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The combination of meta_bg and resize_inode leads to a corrupt
filesystem, and it's not really clear it makes any logical sense.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This simplifies the code, and using the uninit_bg with the inode table
lazily initialized is just as good.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Andreas Dilger