After cleaning up ext2fs_bg_flag_set() and ext2fs_bg_flag_clear(),
we're left with ext2fs_bg_flag_test(). Convert it to
ext2fs_bg_flags_test().
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The ext2fs_bg_flag* functions were confusing.
Currently we have this:
void ext2fs_bg_flags_set(ext2_filsys fs, dgrp_t group, __u16 bg_flags);
void ext2fs_bg_flags_clear(ext2_filsys fs, dgrp_t group,__u16 bg_flags);
(_set (unused) sets exactly bg_flags; _clear clears all and ignores bg_flags)
and these, which can twiddle individual bits in bg_flags:
void ext2fs_bg_flag_set(ext2_filsys fs, dgrp_t group, __u16 bg_flag);
void ext2fs_bg_flag_clear(ext2_filsys fs, dgrp_t group, __u16 bg_flag);
A better interface, after the patch below, is just:
ext2fs_bg_flags_zap(fs, group) /* zeros bg_flags */
ext2fs_bg_flags_set(fs, group, flags) /* adds flags to bg_flags */
ext2fs_bg_flags_clear(fs, group, flags) /* clears flags in bg_flags */
and remove the original ext2fs_bg_flags_set / ext2fs_bg_flags_clear.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
When flex_bg is on, calculate_minimum_resize_size() should add more meta
blocks for newly added flex_bg.
Addresses-RedHat-Bugzilla: #519131
Signed-off-by: Peng Tao <bergwolf@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If the resize operation fails in the middle of the operation, mark the
filesystem as needing to be checked, and tell the user that they
should run e2fsck -fy on the device.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This was reported in Fedora, since the livecd creator does
a lot of resizing.
If we've moved the journal blocks during resize (more likely now,
due to the journal being in the middle) the backup blocks in the
superblock don't get updated, and a subsequent e2fsck will find
issues:
e2fsck 1.41.6 (30-May-2009)
Backing up journal inode block information.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/mnt/test/img: ***** FILE SYSTEM WAS MODIFIED *****
/mnt/test/img: 11/16000 files (0.0% non-contiguous), 17789/38400 blocks
This can be shown in a simple test:
# dd if=/dev/zero of=img bs=1 count=0 seek=3000M
# mke2fs -t ext4 -F img
# resize2fs img 150M
# e2fsck -f img
(thanks to the Fedora reporter Mads Kiilerich for the testcase!
https://bugzilla.redhat.com/show_bug.cgi?id=506105#c2)
So, update the backup journal in the superblock before resize2fs exits.
Addresses-RedHat-Bugzilla: #505339
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The extra padding added to the minimum size calculations:
/*
* We need to reserve a few extra blocks if extents are
* enabled, in case we need to grow the extent tree. The more
* we shrink the file system, the more space we need.
*/
if (fs->super->s_feature_incompat & EXT3_FEATURE_INCOMPAT_EXTENTS)
blks_needed += (fs->super->s_blocks_count - blks_needed)/500;
can go quite wrong if we've already added up more "blks_needed"
than our current size, and the above subtraction wraps. This can
easily happen for a filesystem which is almost completely full.
In this case, just return the current fs size as the minimum and
be done with it.
With this fix we could probably call calculate_minimum_resize_size()
for each resize2fs invocation and refuse to resize smaller than that?
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Due to a fencepost bug, when skipping a block group whose block bitmap
was uninitialized (and hence could not contain any blocks eligible for
relaocation), the block immediately following the block group wasn't
checked as well. If it was in use and required relocation, it
wouldn't get properly relocated, with the result that an inode using
such a block would end up, post resize, with a pointer to a block now
outside the bounds of the filesystem.
This commit fixes this fencepost error.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If we need to shrink the inode table, we need to make sure the inodes
contained in the part of the inode table we are vacating don't get
reused as part of the filesystem shrink operation. This wasn't a
problem with ext3 filesystems, since the inode table was located in
the block group that was going away, so that location was not eligible
for reallocation.
However with ext4 filesystems with flex_bg enabled, it's possible for
a portion of the inode table in the last flex_bg group to be
deallocated, but in a part of the filesystem which could be used as
data blocks. So we must mark those blocks as reserved to prevent
their reuse, and adjust the minimum filesystem size calculation to
assure that we don't shrink a filesystem too small for the resize
operation to succeed.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
When allocating a new set of block group metadata as part of growing
the filesystem, the resize2fs code assumes that the bitmap and inode
table blocks are in their own block group; an assumption which is
changed by the flex_bg feature. This commit works around the problem
by temporarily turning off flex_bg while allocating the new block
group metadata, to avoid potentially overwriting previously allocated
data blocks.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Some extra blocks may be needed to expand some extent allocation trees
while we are shrinking the filesystem. We don't know exactly how
much, so we use a hueristic.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Previously resize2fs assumed that bitmap and inode table blocks were
always located in their respective block group. However, this is no
longer true with flex_bg. So it is necessary to check all of the
block groups which will be truncated to see if they have metadata
blocks that need to be marked as no longer being in use in the new,
shrunk filesystem.
This bug fixes resize2fs -M, which would otherwise fail because
without the released blocks, there would not be enough space in the
filesystem. This bug also avoids (mostly harmless) filesystem
corruptions reported by e2fsck regarding blocks marked in use but not
actually used (these being the bitmap and inode table blocks
associated with the truncated block groups).
Note: in theory it is possible to have block group N utilize bitmap
and inode table blocks in block group N+X with flex_bg. At the moment
neither mke2fs nor e2fsck will create filesystems like this, which is
good, because resize2fs doesn't handle this case correctly.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
In the function blocks_to_move(), when checking to see if a block
group's block bitmap is initialized, we need to check the old_fs's
block group descriptors, not the new file system's (already truncated)
group descriptor data structures. Otherwise we will end up
derferencing past the end of the array boundary, and the resulting
garbage value may indicate that the bitmap is uninitialized, and so
all of the blocks in that block group will be skipped, resulting in
some blocks not getting marked as needing relocation.
This showed up in the following test case:
mke2fs -t ext4 -b 1024 test.img 1048576
resize2fs test.img 80000
The journal inode after the resize operation looked like this:
debugfs: stat <8>
Inode: 8 Type: regular Mode: 0600 Flags: 0x80000
...
BLOCKS:
(IND):35385, (0-5836):2356-8192, (5837-21959):8454-24576, (21960-32506):24838-35
384, (32507-32767):434177-434437
TOTAL: 32769
The blocks 434177-434437 were not moved because block group 53 was
wrongly thought to have an unitialized block group.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Some of these could affect filesystems between 2^31 and 2^32-1 blocks.
Thanks to Valerie Aurora Henson for pointing out the problems in
lib/ext2fs/alloc_tables.c, which led me to do a "make gcc-wall" scan
over the source tree.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This fixes a cosemtic issue where we don't complete the progress bar
and issue a newline before printing the final resize successful
message.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
In the rare case where new blocks are needed while mutating an extent
tree, supply a specialized block allocator so that extent_node_split()
allocates valid blocks for the interior nodes of the extent tree.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If the filesystem has the uninit_bg feature, then parts of the block
and inode bitmap may not be initialized. Teach resize2fs how to deal
with these case appropriately. (Most of these fixes were fortunately
not necessary for the common case where the resize_inode is present to
reserve space, and where the filesystem is being expanded instead of
being shrunk.)
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
When moving directories into new block groups (which would only happen
when shrinking a filesystem), resize2fs would increase the directory
in-use count by 2 times the necessary value, due to a change in
ext2fs_inode_alloc_stats() made in e2fsprogs 1.26. This is largely
harmless, but it does result in a filesystem corruption for e2fsck to
fix.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If the filesystem is grown to the point where the resize_inode is no
longer needed, clean it up properly so e2fsck doesn't have to.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
When calculating the number reserved blocks, use floating point for
better accuracy, since for big filesystems it really makes a
difference. In addition, mke2fs and tune2fs accepts a floating point
number from the user, so they should provide that level of accuracy.
Addresses-Debian-Bug: #452639
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add the -P option to print the minimum filesystem size and exit.
Add the -M option to force resizing the filesystem to the minimum
filesystem size.
Signed-off-by: Josef Back <jbacik@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Use ext2fs_get_next_inode_full() in resize2fs and clean up large inode
handling; previous attempt was not properly handling all cases, and
was incorrectly setting i_extra_isize. This caused some extended
attributes to get removed or randomly assigned to other inodes as a
result of the resize, which can be unfortunate on systems using
SELinux.
The previous commit didn't fix things completely on big-endian systems
like PowerPC.
Addresses-Red-Hat-Bugzilla: #434893
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
inode_scan_and_fix() in resize2fs needs to do read/write of the full
inode to be sure it gets all data from larger (>128 byte) inodes.
Addresses-Red-Hat-Bugzilla: #434893
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add some additional checks, primarily in resize2fs and in the rarely
used (and soon to-be-deprecated) e2fsck byte-swap filesystem function.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add macros to support variable-length group descriptors for ext4.
Signed-off-by: Valerie Clement <valerie.clement@bull.net>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This patch changes ext2fs_open() to set EXT2_FLAG_MASTER_SB_ONLY by
default. This avoids some problems in e2fsck (reported by Jim Garlick)
where a corrupt journal can end up writing the bad superblock to the
backups. In general, only e2fsck (after the filesystem is clean),
tune2fs, and resize2fs should change the backup superblocks by default.
Most callers of ext2fs_open() should not be touching anything where the
backups should be touched. So let's change the defaults to avoid
potential problems.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Found 2 of the three places where a return code for ext2fs_write_inode() was
not being checked.
The second fix in e2fsck/emptydir.c is basically just to shut coverity up even
though it really is unnecessary.
Coverity ID: 1: Checked Return
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add a new functiom, e2p_percent(), which correct calculates the percentage
of a number based on a given percentage, without worrying about overflow
issues. This is used where we calculate the number of reserved blocks using
a percentage of the total number of blocks in a filesystem.
Based on patches from Eric Sandeen, but generalized to use this new function.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Signed-off-by: Eric Sandeen <esandeen@redhat.com>
For loops such as:
for (i=1; i <= fs->super->s_blocks_count; i++) {
<do_stuff>
}
if i is an int and s_blocks_count is (2^32-1), the condition is never false.
Change these loops to:
for (i=1; i <= fs->super->s_blocks_count && i > 0; i++) {
<do_stuff>
}
to stop the loop when we overflow i
Signed-off-by: Eric Sandeen <esandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add a new function, ext2fs_div_ceil(), which correctly calculates a division
of two unsigned integer where the result is always rounded up the next
largest integer. This is used everywhere where we might have
previously caused an overflow when the number of blocks
or inodes is too close to 2**32-1.
Based on patches from Eric Sandeen, but generalized to use this new function
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Signed-off-by: Eric Sandeen <esandeen@redhat.com>
Change the format string(%d, %ld) for a block number and inode number
to %u or %lu.
Signed-off-by: Takashi Sato <sho@tnes.nec.co.jp>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
There was a off-by-one fencepost error in the logic used to check if
we avoid copying zero-filled blocks when moving an inode table down by
a block or two. Thanks to valgrind for catching it. As far as I know
this fencepost error wasn't causing any actual problems, but it was
definitely a bug.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
non-empty bad block list. Resize2fs now discards any blocks on the
badblock list which are no longer part of the filesystem as the result
of a filesystem shrink. (Note: this means that shrinking and then
enlarging a filesystem is no longer a reversible operation;
information about bad blocks in the part of the filesystem
which is to be chopped off will be lost.)
moving an inode, set the ctime field so that people using
dump/restore will backup the changed inode. Also update
the mtime and ctime of directories which get updated when
we need to move an inode.
release the bmap table. This significantly speeds up
resize2fs when shrinking or expanding a filesystem by a
very small number of blocks (which EVMS will do).
than the requested size, then we will attempt to
transparently extend the filesize in a sparse fashion by
writing a block at the end of the requested part of the
filesystem.
main.c (main), resize2fs.c (resize_fs), resize2fs.h: Change the
function prototype of resize_fs() so that it can modify
the new_size parameter with the actual new size of the
filesystem after the resize operation. (This can
sometimes be less than the requested new size if there
isn't enough space to create the necessary block group
metadata for that last bit of disk space.) Resize2fs now
prints the actual new size of the filesystem when it finishes.
Update copyright of files in preparation for release of resize2fs.
Also removed the (obsolete) beta-test check that had been in the code,
as well as the old spec files used to package the separate
distribution of resize2fs.
Makefile.in: No longer build resize2fs.static, since there's no real
point in it any more.
Modified Files:
ChangeLog Makefile.in extent.c main.c resize2fs.8.in
resize2fs.c resize2fs.h sim_progress.c test_extent.c
Removed Files:
e2fsprogs-1.12.spec pq.gif resize2fs.spec version.h
ext2fs.h, bitops.h: Add support for the Watcom C compiler to do inline
functions.
ext2fs.h, dosio.c: Use asm/types.h instead of linux/types.h to evade a
potential problem with glibc's header files trying to spike out
linux/types.h.
ext2fs.h (ext2fs_resize_mem): Change the function prototype to include
the old size of the memory, which is needed for some braindamaged
memory allocation systems that don't support realloc().
badblocks.c (ext2fs_badblocks_list_add):
bb_inode.c (clear_bad_block_proc):
dblist.c (ext2fs_add_dir_block):
icount.c (insert_icount_el):
irel_ma.c (ima_put):
rs_bitmap.c (ext2fs_resize_generic_bitmap): Update functions to pass
the old size of the memory to be resized to ext2fs_resize_mem().
ChangeLog, dirinfo.c:
dirinfo.c (e2fsck_add_dir_info): Update function to pass the old size
of the memory to be resized to ext2fs_resize_mem().
ChangeLog, extent.c, resize2fs.c:
resize2fs.c (adjust_superblock):
extent.c (ext2fs_add_extent_entry): Update functions to pass the old
size of the memory to be resized to ext2fs_resize_mem().
main.c (main): Check to make sure we can really resize this
filesystem. If there are compatible feature sets that we don't
support, we need to abort.
resize2fs.c: Change to use the new prototype required by
block_iterate2 (which uses blkcnt_t for the blockcount).
version.h, RELEASE-NOTES:
Interim commit of changes to date.
resize2fs.c (inode_scan_and_fix): If we are expanding the filesystem,
temporarily set rfs->old_fs->super->s_blocks_count to ultimate size of
the filesystem, to avoid catching an error on the block iterator.
resize2fs.c (blocks_to_move): Create the meta_bmap using the old
filesystem, not the new filesystem.
(get_new_block): Don't try checking the old block bitmap if the block
we're testing is beyond the boundaries of the old filesystem.
main.c (check_mount, main): Resize2fs now checks the size of the
partition, and uses this as a default new_size of the partition if one
is not specified. Resize2fs will not let you resize a mounted
partition.
resize2fs.c: Change how the progress function for the INODE_SCAN pass
is performed, so that the maximum size is never zero.
resize2fs.8.in: Updated man page.
Major reoganization of how resizing works. Functions in
ext2_block_move.c, and ext2_inode_move.c moved into resize2fs.c.
Multiple passes in those two files also combined into a single inode
scanning pass. Made the inode table moving function better handling
the case where it needs to abort mid-operation. When moving blocks
around, made the block allocation function preferentially avoid the
old group descriptor blocks, to make resize2fs more robust.
resize2fs.h: If EXT2_FLAT_INCLUDES is defined, then assume all
of the ext2-specific header files are in a flat directory.
ext2_block_move.c, ext2_inode_move.c, extent.c, resize2fs.c,
resize2fs.h: Rename variables named "new" to "new_block",
"new_inode", or "new_loc" to avoid C++ reserved word
clash.
ext2_block_move.c, ext2_inode_move.c, extent.c, resize2fs.c,
sim_progress.c: Use ext2fs_get_memory(),
ext2fs_free_memory(), et. al., instead of malloc() and
free().
ext2_block_move.c, ext2_inode_move.c, extent.c: Explicitly cast
all assignments from void * to be compatible with C++.
banalysis.c, banalysis.h, ext2_inode_move.c, ext2_block_move.c:
Change private to priv_data to avoid C++ namespace clash.
ChangeLog, badblocks.8.in:
badblocks.8.in: Add documentation for the -s option.
resize2fs.c, ext2_block_move.c ext2_inode_move.c: Only include printf
statements if RESIZE2FS_DEBUG is defined.
main.c: Don't read in the bitmaps since resize2fs.c does that.
e2label.c, mke2fs.c:
Adjust header files.
Allow people to set the version.h to something like 1.10-PLUS.
.del-inodemap.c~24510e64, main.c, resize2fs.c, resize2fs.h:
More interim work. All is functioning except progress meter.
bmove.c (ext2fs_move_blocks): New function which takes a bitmap of
blocks which need to be moved, and moves those blocks to another
location in the filesystem.
rs_bitmap.c (ext2fs_resize_generic_bitmap): When expanding a bitmap,
make sure all of the new parts of the bitmap are zero.
bitmaps.c (ext2fs_copy_bitmap): Fix bug; the destination bitmap wasn't
being returned to the caller.
alloc_tables.c (ext2fs_allocate_group_table): Add new function
ext2fs_allocate_group_table() which sets the group tables for a
particular block group. The relevant code was factored out of
ext2fs_allocate_tables().
dblist.c (make_dblist): Adjust the initial size of the directory block
list to be a bit more realize (ten plus twice the number of
directories in the filesystem).
Check in interim work.
configure.in (rmakefile): Added (optional) private directory for resize2fs.
Makefile.in: Change recursive descent rules to check to see if a
directory exists before trying to make it.
Many files:
Initial checkin of work done to date.
Valerie Aurora Henson