Fix a potential memory leak reported by Li Xi. In addition, there
were possible error cases where the file descriptor would not be
properly closed, so fix those as well while we're at it.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reported-by: Li Xi <pkuelelixi@gmail.com>
Make sure the s_mmp_update_interval super block field is set
from the file system parameters block which is passed into the
ext2fs_initialize() function.
Addresses-Lustre-Bug: LU-1888
Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Commit 53e3120c18 introduced a regression which would case e2fsck to
overrun an array boundary for bigalloc file systems, and most likely
crash. Fix this by correctly using blocks instead of clusters when
incrementing the loop counter in the fast path optimization case.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
One of these fixes was triggering failures when running:
./test_scripts --valgrind r_move_itable r_inline_xattr r_resize_inode
It should be a false positive, but it fixing this makes it easier to
see real problems.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Commit 44a2cca35e introduced a compile-time failure if --enable-quota
is not passed to the configure script.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If we haven't turned --enable-quota on at config time,
I don't think tune2fs should know about the feature either.
Today we can actually tune2fs -O quota even if not
configured on, and then the rest of the tools will
refuse to touch it:
# tune2fs -O quota /dev/sda1
# tune2fs -O ^quota /dev/whatever complains
tune2fs 1.42.3 (14-May-2012)
tune2fs: Filesystem has unsupported read-only feature(s) while trying to open /dev/sda1
# fsck /dev/sda1
fsck from util-linux 2.21.2
e2fsck 1.42.3 (14-May-2012)
/dev/sda1 has unsupported feature(s): quota
e2fsck: Get a newer version of e2fsck!
Ok, so turn it off?
# tune2fs -O ^quota /dev/whatever complains
tune2fs 1.42.3 (14-May-2012)
tune2fs: Filesystem has unsupported read-only feature(s) while trying to open /dev/sda1
Nope. Debugfs? Nope.
# debugfs -w /dev/sda1
debugfs 1.42.3 (14-May-2012)
/dev/sda1: Filesystem has unsupported read-only feature(s) while opening filesystem
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reported-by: Bert DeKnuydt <Bert.Deknuydt@esat.kuleuven.be>
Addresses-Red-Hat-Bugzilla: #880596
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Add a fast path optimization in e2fsck's pass 5 for the common case
where the block bitmap is correct. The optimization works by
extracting each block group's block allocation bitmap into a memory
buffer, and comparing it with the expected allocation bitmap using
memcmp(). If it matches, then we can just update the free block
counts and be on our way, and skip checking each bit individually.
Addresses-Google-Bug: #7534813
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Lukas Czerner <lczerner@redhat.com>
This optimizies the CPU utilization of the rb_get_bmap_range()
function when most of the bitmap is allocated.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Lukas Czerner <lczerner@redhat.com>
This simplifies the rb_get_bmap_range() function and speeds it up for
the case where most of the bitmap is zero.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Lukas Czerner <lczerner@redhat.com>
This function efficiently counts the number of bits in a block of
memory.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Lukas Czerner <lczerner@redhat.com>
Optimize e2fsck pass 1 by marking entire extents as being in use at a
time, instead of block by block. This optimization only works for
non-bigalloc file systems for now (it's tricky to handle bigalloc file
systems since this code is also responsible for dealing with blocks
that are not correctly aligned within a cluster). When the
optimization works, the CPU savings can be significant: ove a full CPU
minute for a mostly full 4T disk.
Addresses-Google-Bug: #7534813
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Lukas Czerner <lczerner@redhat.com>
This speeds up reading bitmaps from disk for very large (and full)
disks by significant amounts (i.e., up to two CPU minutes for a 4T
file system).
Addresses-Google-Bug: #7534813
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Lukas Czerner <lczerner@redhat.com>
Restructure the ext2fs_get_device_size() and blkid_get_dev_size()
code to localize the variables used for different device probing
methods. This at least reduces the #ifdef mess to only one part
of the code for each method, and avoids "unused variable" compiler
warnings added when variables are declared without being #ifdef'd.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Change the license of the mmp.c file to LGPL to match the license
of other files in the libext2fs library.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The "Reading directory block XXX..." message was not being sent
through the pager, which would result in confusing output.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
In e2fsck_pass4(), we were consulting inode_dir_map using
ext2fs_test_inode_bitmap2() for every single inode in the file system.
However, there were many cases where we never needed the result of the
test --- most notably if the inode is not in use.
I was a bit surprised that GCC 4.7 with CFLAGS set to "-g -O2" wasn't
able to optimize this out for us, but here is the pass 4 timing for an
empty 3T file system before this patch:
Pass 4: Memory used: 672k/772k (422k/251k), time: 3.67/ 3.66/ 0.00
and afterwards, we see a 43% improvement:
Pass 4: Memory used: 672k/772k (422k/251k), time: 2.09/ 2.08/ 0.00
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Also fix a bug caused by a stray continuation backslash which caused
the e2fsck/Makefile to fail when profiling is enabled.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Profiling shows that rb_test_bit() is now calling ext2fs_rb_next() a
lot, and this function is now the hot spot when running e2freefrag.
If we cache the results of ext2fs_rb_next(), we can eliminate those
extra calls, which further speeds up both e2freefrag and e2fsck by
reducing the amount of CPU time spent in userspace.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The code was previously allocating a single 4 or 8 byte pointer for
the rcursor and wcursor fields in the ext2fs_rb_private structure;
this added two extra memory allocations (which could fail), and extra
indirections, for no good reason. Removing the extra indirection also
makes the code more readable, so it's all upside and no downside.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Optimize testing for a bit in an rbtree-based bitmap for the case
where the calling application is scanning through the bitmap
sequentially. Previously, we did this for a set of bits which were
inside an allocated extent, but we did not optimize the case where
there was a large number of bits after an allocated extents which were
not in use.
1111111111111110000000000000000000
^ optimized ^not optimized
In my tests of a roughly half-filled file system, the run time of
e2freefrag was halved, and the cpu time spent in userspace was during
e2fsck's pass 5 was reduced by a factor of 30%.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Lukas Czerner <lczerner@redhat.com>
Enable the use of 64-bit bitmaps, so e2freefrag will work on file
systems with the 64-bit feature enabled. In addition, enable the
rbtree-based bitmaps, which significantly saves the amount of memory
required (from 97 megs to 1.7 megs for an empty 3T file system) at the
cost of additional CPU overhead (but we will claw back some of the
additional CPU overhead in the next commit).
Addresses-Google-Bug: 7269948
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Throttle updates for the "Allocating Groups" progress updates to once
a second as well. We now do this throttling in libext2fs, so we don't
have to do this for each of mke2fs's progress updates, and because the
updates from ext2fs_allocate_tables() come from within libext2fs
anyway.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
We print "extents" for the feature, so we should document "extents" in
the mke2fs's man page, even though we accept both "extent" and
"extents".
Addresses-Sourceforge-Bug: #3559210
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Remove dc from the list of dependencies, since it's not used during
the build anymore.
Addresses-Debian-Bug: #677497
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If the f_mmp test is interrupted during its test run, then it can
leave debugfs busy-looping in the background. Since f_mmp is a
relatively long-running test, and is likely to be running during
a parallel test run, this can happen fairly often.
Set a signal trap for the f_mmp test script being killed, so that
the background debugfs command will always be killed by the test.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Add a configuration knob so the regression tests can disable progress
reporting. This fixes a potential lack of predictability since the
progress reports are now time based (once a second) which is
problematic for regression tests which are comparing the expected
output of mke2fs.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
With lazy itable initialization, the progress updates for writing the
inode table happens so quickly that on a serial console, the time to
write the progress updates can be the bottleneck. Fix this by only
updating the progress indicator once a second.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If the uninit_bg feature is enabled and the kernel supports
lazy_itable_init, skip zeroing the inode table so that the resize
operation can go much more quickly. Also set the itable_unused fields
so that the first e2fsck after the resize will run faster.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Enhance the online resizing code to be more nuanced about resizing
restrictions. If the kernel supports meta_bg resizing, then we can
skip all of the restrictions. If the kernel does not support meta_bg
resizing, check more carefully to make sure there are enough reserved
gdt blocks, so that the user gets a clearer error message.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Resize2fs can't handle resizing flex_bg file systems that do not have
the resize inode, but when the kernel adds support for resizing using
the meta_bg layout, we should allow it be able to resize the file
system.
So move the flex_bg/resize_inode check to the just before we start
doing the off-line resize, instead of doing it earlier where it would
prohibit these file systems for both on-line and off-line resizes.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The file system overhead calculation in calculate_minimum_resize_size
was incorrect meta_bg file systems. This caused the minimum size to
underflow for very large file systems, which threw resize2fs into a
loop generally lasted longer than the user's patience.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The 16TB limit must be enforced regardless of whether the new size is
specified on the command line or implied by the size of the device,
but only if the file system does not support 64-bit block sizes, or
the kernel does not advertise support of meta_bg resizing.
Previously we were unconditionally enforcing it when it was implied by
the device size, but not if the new size was specified on the command
line.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Since clang uses C99 semantics by default, the main changes required
to allow clang to build e2fsprogs was to add support the C99 inline
semantics, while still allowing us to be built when the legacy (but
still default for gcc) GNU C89 inline semantics are in force.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
mke2fs -m option can set reserved blocks ratio up to 50%. But if the
last block group is not big enough to support the necessary data
structures, it gets dropped, we have to recalculate the number of
reserved blocks so that the reserved blocks matches the requested
percentage.
It also avoids a problem where if the user specifies a reserved blocks
of 50%, and after the last partial block group was dropped, if the
number of reserved blocks is greater than 50%, e2fsck will complain.
Steps to reproduce:
1. Create a FS which has the overhead for the last BG
and specify 50 % for reserved blocks ratio
# mke2fs -m 50 -t ext4 DEV 1025M
mke2fs 1.42.5 (29-Jul-2012)
warning: 256 blocks unused.
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
656640 inodes, 2621440 blocks
1310848 blocks (50.00%) reserved for the super user
~~~~~~~ <-- Reserved blocks exceed 50% of FS blocks count!
2. e2fsck outputs filesystem corruption
# e2fsck DEV
e2fsck 1.42.5 (29-Jul-2012)
Corruption found in superblock. (r_blocks_count = 1310848).
The superblock could not be read or does not describe a correct ext2
filesystem. If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 32768 <device>
Signed-off-by: Akira Fujita <a-fujita@rs.jp.ne.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The configure option --enable-relative-symlinks was incorrectly
specified in configure.in, as --enable-symlink-relative-symlinks. Fix
the configure script so that --enable-relative-symlinks works, as well
as previous incorrect command line option. We will keep the older,
incorrect --enable-symlink-relative-symlinks for at least two years
before removing it.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Most places respect this define, but this one doesn't.
Reported-by: Dmitri Bogomolov <4glitch@gmail.com>
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
When passed a negative count (indicating a byte count rather than
a block count) e2fsck_handle_read_error() treats the data as a full
block, causing unix_write_blk64() (which can handle negative counts
just fine) to try to write too much. Given a faulty block device,
this resulted in a SEGV when unix_write_blk64() read past the bottom
of the stack copying the data to cache. (check_backup_super_block ->
unix_read_blk64 -> raw_read_blk -> e2fsck_handle_read_error)
Reported-by: Alex Friedman <alexfr@il.ibm.com>
Signed-off-by: Jim Keniston <jkenisto@us.ibm.com>
Signed-off-by: Dan Streetman <ddstreet@us.ibm.com>
Reviewed-by: Mingming Cao <mcao@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Commit a7c17431b9 attempted to fix a problem where the system
libraries might get used instead of local libraries for things like
-lcom_err. It tried to accomplish this by moving $(ELF_OTHER_LIBS) to
before $(LDFLAGS).
Unfortunately, this was the wrong fix; $(ELF_OTHER_LIBS) *MUST* be
after the object files, or the linker might not pull in the necessary
library and not include it into the DT_NEEDED section of the shared
library. The proper fix is to add a -L$(LIB) before $(LDFLAGS), and
then remove the -L option from all of the ELF_OTHER_LIBS definitions
in the library Makefiles.
Addresses-Sourceforge-Bug: #3554345
Cc: Olivier Blin <olivier.blin@softathome.com>
Reported-by: Mike Frysinger <vapier@gentoo.org>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
In addition to the free blocks and free inodes, also print the number
of blocks and inodes in the verbose statistics.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Disks have gotten bigger, so 8 digits might not be enough. Allow for
12 digits worth of blocks, which is more than enough for 3 petabytes.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add report_time, report_verbose, and report_features options to
e2fsck.conf which enable additional, more verbose reporting by e2fsck.
This is useful for large cloud installations where there are a large
number file systems being managed, and where it may not be obvious
from the e2fsck log files exactly how a particular file system is
configured.
The report_time and report_verbose options, which are the same as the
-tt and -v command line options, respectively, are useful because they
are options specific to e2fsck, and the fsck program does not have a
way of passing certain options only to a specific /sbin/fsck.<fstype>
program.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The following commands:
dd if=/dev/zero of=/tmp/foo count=1 ibs=$(( 256 * 1024 * 1024 ))
mke2fs -N 256 -t ext4 /tmp/foo
... will cause mke2fs to write until it fills the device. The cause
for this is that the explicit request for 256 inodes causes the number
of inodes per block group to be 8. The ext2fs_initialize() function
assumed that all of the reserved inodes would be in the first block
group, which is not true in this case. This caused the number of
uninitialized inodes in the first block group to be negative, which
then resulted in mke2fs trying to zero out a very large number of
blocks. Oops.
Addresses-Sourceforge-Bug: #3528892
Reported-by: Mike Frysinger <vapier@gentoo.org>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>