Run sparse against source files when building e2fsprogs with 'make C=1'. If
instead C=2, it configures basic ext2 types for bitwise checking with sparse,
which can help find the (many many) spots where conversion errors are
(possibly) happening.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If the file system is being shrunk, and a block group's inode table
falls beyond the end of the inode table, we need to try to relocate
the inode table blocks.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If the file system's inode table blocks in the last block group are
located in the middle or the end of the block group, it's possible for
resize2fs -M to use a size which will require relocating the inode
table blocks in the last block group. This can lead to all sorts of
problems, so solve it by simply guaranteeing that we will never do
that.
Reported-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
free_gdp_blocks needs to be taught to use 64-bit fields and the appropriate
getters, otherwise it'll truncate high block numbers (when, say, resizing a
>16T fs) and mark the low numbered group descriptor blocks as free. Yikes.
Reported-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
It is possible to have a flex_bg filesystem with block groups
which have inode & block bitmaps at some point well past the
start of the group.
If an offline shrink puts the new size somewhere between
the start of the block group and the (old) location of
the bitmaps, they can be left beyond the end of the filesystem,
i.e. result in fs corruption.
Check each remaining block group for whether its bitmaps
are beyond the end of the new filesystem, and reallocate
them in a new location if needed.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
When doing an off-line resize2fs of an initially very small file
system, it's possible to run out of reserved gdt blocks (which are
reserved via the resize inode). Once we run out, we need to move the
allocation bitmaps and inode table out of the way to grow the gdt
blocks. Unfortunately, when moving these metadata blocks, it was
possible that a block that had been just been newly allocated for a
new block group could also get allocated for a metadata block for an
existing block group that was being moved.
To prevent this, after we grow the gdt blocks and allocate the
metadata blocks for the new block groups, make sure all of these
blocks are marked as reserved.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reported-by: John Jolly <john.jolly@gmail.com>
Fixes resize2fs so it correctly calculates the number of free clusters
in each block group for file systems with the bigalloc feature
enabled.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This should be made into a more formal, automated test case, but for
now, save this as script since it's useful for validating resize2fs's
handling of very large file systems.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The old online resize ioctl interfaces are still present in the
kernel, and we want to make it easy to test both the kernel code for
those older interfaces, and resize2fs's use of those interfaces in a
relatively easy manner.
To do this, resize2fs will now check the environment variable
RESIZE2FS_KERNEL_VERSION. If the version given is less than 3.3, then
do not try using the new resizing ioctl, but instead use the resizing
ioctls that were used before Linux version 3.3.
If the version given is less than 3.7, then emulate sanity checks
which get done to protect against the fact that the new resizing ioctl
prior to 3.7 did not handle meta_bg resizing. (This was previously
tested via the presence of the RESIZE2FS_NO_META_BG_RESIZE environment
variable. But the new environment variable, RESIZE2FS_KERNEL_VERISON,
is more general.)
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Carlos Maiolino <cmaiolino@redhat.com>
There are a number of places where we multiply a dgrp_t with
s_blocks_per_group expecting that we will get a blk64_t. This
requires a cast, or using the convenience function
ext2fs_group_first_block2().
This audit was suggested by Eric Sandeen.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Use ext2fs_[un]mark_block_range2() functions to reduce the CPU
overhead of resizing large file systems by 45%, primarily by
reducing the time spent in fix_uninit_block_bitmaps().
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add a new debug flag which prints how much time is consumed by the
various parts of resize2fs's processing.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This caused the free blocks count in the superblock to be incorrect
after resizing a 64-bit file system if the number of free blocks
overflowed a 32-bit value.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Fix a 32-bit overflow bug caused by a missing blk64_t cast which can
cause the block bitmap to get corrupted when doing an off-line resize
of a 64-bit file system.
This problem can be reproduced as follows:
rm -f foo.img; touch foo.img
truncate -s 8T foo.img
mke2fs -F -t ext4 -O 64bit foo.img
e2fsck -f foo.img
truncate -s 21T foo.img
resize2fs foo.img
e2fsck -fy foo.img
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Now that we are reserving all of the bg-specific metadata before we
try to allocate the metadata for the new block groups, we don't have
to temporarily disable the flex_bg feature flag while we allocate the
new metadata blocks --- this allows the newly created block groups to
have a much more optimized layout, instead of fragmenting the inode
table and block/inode bitmaps in sepraate block groups.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
With the bug fixes from the last two commits, resize2fs can now fully
support off-line resizing of file systems with flex_bg even if the
resize_inode feature is not present; so we no longer need to disallow
this combination.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
With flex_bg file systems, bg-specific metadata (i.e., bitmaps and the
inode table blocks) can be located in another block group. Hence,
when we grow the number of block group descriptors, we need to check
if we need to relocate metadata blocks not just for the block group
where the bgd blocks are located, but in all block groups.
This change fixes the following test case:
rm -f foo.img; touch foo.img
truncate -s 32G foo.img
mke2fs -F -t ext4 -E resize=12582912 foo.img
e2fsck -f foo.img
truncate -s 256G foo.img
./resize2fs foo.img
e2fsck -fy foo.img
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
For flex_bg file systems, if we need to relocate an allocation bitmap
or inode table, we need to make sure that all metadata blocks have
been reserved, lest we end up overwriting a metadata block belonging
to a different block group.
This change fixes the following test case:
rm -f foo.img; touch foo.img
truncate -s 32G foo.img
mke2fs -F -t ext4 -E resize=12582912 foo.img
e2fsck -f foo.img
truncate -s 64G foo.img
./resize2fs foo.img
e2fsck -fy foo.img
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This is the first commit to add support for off-line resizing using
flex_bg without the assist of using the resize_inode to reserve gdt
blocks. This functionality has been broken up into separate commits
which are hopefully obviously correct to make them easier to review
for correctness.
In this first step, we break up the for loop at the end of
blocks_to_move() so that we first mark all of the metadata blocks
which don't need to be moved in the reserve_blocks bitmap, and then
try to allocate the metadata blocks are new or which need to moved
second.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
One of these fixes was triggering failures when running:
./test_scripts --valgrind r_move_itable r_inline_xattr r_resize_inode
It should be a false positive, but it fixing this makes it easier to
see real problems.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If the uninit_bg feature is enabled and the kernel supports
lazy_itable_init, skip zeroing the inode table so that the resize
operation can go much more quickly. Also set the itable_unused fields
so that the first e2fsck after the resize will run faster.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Enhance the online resizing code to be more nuanced about resizing
restrictions. If the kernel supports meta_bg resizing, then we can
skip all of the restrictions. If the kernel does not support meta_bg
resizing, check more carefully to make sure there are enough reserved
gdt blocks, so that the user gets a clearer error message.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Resize2fs can't handle resizing flex_bg file systems that do not have
the resize inode, but when the kernel adds support for resizing using
the meta_bg layout, we should allow it be able to resize the file
system.
So move the flex_bg/resize_inode check to the just before we start
doing the off-line resize, instead of doing it earlier where it would
prohibit these file systems for both on-line and off-line resizes.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The file system overhead calculation in calculate_minimum_resize_size
was incorrect meta_bg file systems. This caused the minimum size to
underflow for very large file systems, which threw resize2fs into a
loop generally lasted longer than the user's patience.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The 16TB limit must be enforced regardless of whether the new size is
specified on the command line or implied by the size of the device,
but only if the file system does not support 64-bit block sizes, or
the kernel does not advertise support of meta_bg resizing.
Previously we were unconditionally enforcing it when it was implied by
the device size, but not if the new size was specified on the command
line.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Also fixed a number of other minor nits in the resize2fs and e2image
man pages.
Addresses-Debian-Bug: #674453, #674694
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
By passing EXT2_FLAG_64BITS to ext2fs_open2() we can avoid some
unnecessary redirection in critical paths. While resize2fs does not
currently otherwise support so big filesystems that this would matter,
passing this flag is entirely harmless and only tells libext2fs that
the caller has been recompiled against current headers.
With this change the CPU time needed to shrink a 100G filesystem drops
by 20%.
Signed-off-by: Sami Liedes <sami.liedes@iki.fi>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Clean up some compile warnings related to fstat64(), which is
verbosely deprecated on OSX.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
calculate_minimum_resize_size() forgot to account s_first_data_block
into minimum filesystem size. Thus in case the size of filesystem was
such that the last group had the minimal size (50 blocks + metadata
overhead), the code in adjust_fs_info() decided the group is unneeded,
removed it, and in some cases the resizing then failed with ENOSPC.
Fix the issue by properly accounting for s_first_data_block in
calculate_minimum_resize_size().
Signed-off-by: Jan Kara <jack@suse.cz>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Linux's compat_sys_ioctl() function, which is run when executing a
ioctl using a 32-bit binary on a 64-bit kernel, returns EINVAL when an
inode does not exist. Sigh. See /usr/src/linux/fs/compat_ioctl.c.
This is probably a kernel bug, but work around it for now.
Addresses-Debian-Bug: #644989
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The ext2fs_file_acl_block() and ext2fs_set_file_acl_block() needs to
only check i_file_acl_high if the 64-bit flag is set. This is needed
because otherwise we will run into problems on Hurd systems which
actually use that field for h_i_mode_high.
This involves an ABI change since we need to pass ext2_filsys to these
functions. Fortunately these functions were first included in the
1.42-WIP series, so it's OK for us to change them now. (This is why
we have 1.42-WIP releases. :-)
Addresses-Sourceforge-Bug: #3379227
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This commit fixes a failure when running the commands:
dd if=/dev/zero of=fs bs=1k count=100k; mke2fs fs; resize2fs -Mp fs
We should not try truncating the file system if there is only a single
block group in the file system.
Addresses-Sourceforge-Bug: #3404051
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Commit 9f6ba888f0 added support for new online resize ioctl
EXT4_IOC_RESIZE_FS. It is also trying to avoid failure when this
ioctl() is not supported by the kernel however it is checking wrong
error code (EINVAL).
When the ioctl does not exist, errno is set to ENOTTY, so we should
check for that, rather than EINVAL which means that ioctl arguments
are not valid. So change the code to check for ENOTTY and allow
resize2fs to try to use the old approach. Also add some comments.
Addresses-Red-Hat-Bugzilla: #746284
Addresses-Debian-Bug: #644989
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Also remove the _("<foo>") marker from a string that was all numbers
and hence didn't need punctuation.
Thanks to Philipp Thomas and Goeran Uddeborg for reporting these
buglets.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
For those e2fsprogs programs which use libcom_err and are
internationalized, pass the gettext() function to libcom_err during
program initialization.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
On-line resizing has been broken in the 1.42 series for two reasons:
(a) the call to the new EXT4_IOC_RESIZE_FS ioctl checked for ENOTTY to
indicate that the ioctl does not exist, when in fact EINVAL is what is
returned if the ioctl doesn't exist. (b) resize2fs was passing in a
pointer to a 64-bit value, when the ioctl expected a 32-bit value.
This was OK on little-endian systems, but it wouldn't work at all on
big-endian systems.
Fix both problems.
Addresses-Debian-Bug: #451388
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The DEFS line in MCONFIG had gotten so long that it exceeded 4k, and
this was starting to cause some tools heartburn. It also made "make
V=1" almost useless, since trying to following the individual commands
run by make was lost in the noise of all of the defines.
So fix this by putting the configure-generated defines in lib/config.h
and the directory pathnames to lib/dirpaths.h.
In addition, clean up some vestigal defines in configure.in and in the
Makefiles to further shorten the cc command lines.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Code to count the number of blocks in the last partial
group is cut and pasted around the e2fsprogs codebase
a few times.
Making this a helper function should improve matters.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
In many places we are using #ifdef HAVE_OPEN64 to determine if we can
use open64() but that's ugly. This commit creates two new helpers
ext2fs_open_file() for open() and ext2fs_stat() for stat(). Also we need
new typedef ext2fs_struct_stat for struct stat.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
This is needed to support online resizing for > 32-bit file systems
Signed-off-by: Yongqiang Yang <xiaoqiangnk@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The write_journal_inode() code is only setting the low 32-bit i_size
for the journal size, even though it is possible to specify a journal
up to 10M blocks in size. Trying to create a journal larger than 2GB
will succeed, but an immediate e2fsck would fail. Store i_size_high
for the journal inode when creating it, and load it upon access.
Use s_jnl_blocks[15] to store the journal i_size_high backup. This
field is currently unused, as EXT2_N_BLOCKS is 15, so it is using
s_jnl_blocks[0..14], and i_size is in s_jnl_blocks[16].
Rename the "size" argument "num_blocks" for the journal creation functions
to clarify this parameter is in units of filesystem blocks and not bytes.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong