Commit Graph

191 Commits (6e4fa6e28151a3bffea2d73c22ea3b7f2338f0fd)

Author SHA1 Message Date
Vitaliy Filippov 6e4fa6e281 Fix ea remapping 2016-09-29 01:51:04 +03:00
Vitaliy Filippov b331d80fba Fix itable_unused adjustment - fixes shrink FS + shrink inode tables and some other cases 2016-09-27 23:53:58 +03:00
Vitaliy Filippov f99c7a2e12 One way to fix non-contiguous inode table allocations... 2016-09-26 00:02:00 +03:00
Vitaliy Filippov b9b963d427 Fix the rest of block bitmap differences 2016-09-26 00:02:00 +03:00
Vitaliy Filippov f58c260dc4 Remap blocks before moving inode tables (fixes big flex_bg extend),
check blocks to move even when shrinking inode tables (fixes big flex_bg shrink),
mark inode tables during allocation (fixes bigalloc shrink)
2016-09-25 23:48:01 +03:00
Vitaliy Filippov 152890aa45 Split inode_scan_and_fix into two parts.
- Basic extend and shrink works
- Extend on bigalloc fs works
- Shrink on bigalloc fs gives errors
- Extend on big flex_bg gives errors
2016-09-25 23:48:01 +03:00
Vitaliy Filippov 63039f4bac Implement inode table resizing (not done yet, should split inode_scan_and_fix()) 2016-09-25 23:47:54 +03:00
Theodore Ts'o 3d6fc97483 resize2fs: fix crash when there is an ea block and no blocks to migrate
This fixes a bug introduced in 1.43 by commit fb47b94fffc: "resize2fs:
rewrite extent/dir/ea block checksums when migrating".  If there is an
extended attribute block and there are no blocks that need to migrate,
we will crash.

The bug was caused by a botched De Morgan's transformation.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2016-08-10 16:00:17 -04:00
Theodore Ts'o 81f965a04d tune2fs,resize2fs: special case HURD only when testing s_creator_os
The s_creator_os field was a mistake, given how Hurd has been
ab(using) certain file system fields.  We should skip support for
certain advanced features (64-bit support, metadata checksums) for
file systems created on the Hurd OS only, instead of only supporting
them for file systems created on Linux.  This fixes various regression
test failures for FreeBSD.

(The regression tests are probably mostly hopeless for Hurd anyway,
but given the HURD abuse's of various file system fields, the HURD is
going to have to be given second-class treatment in any case, given
what they decided to do with ext2 support, which locks them out of the
more advanced file system features, anyway.)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2016-06-07 11:47:40 -04:00
Theodore Ts'o 62f9bd0e4f e2fsck,resize2fs: fix gcc -Wall nits
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2016-05-29 13:36:20 -04:00
Theodore Ts'o aaa1ae0856 resize2fs, libext2fs: fix bugs in sparse_super2 support
Fix resize2fs so that the location of the backup superblocks when the
sparse_super2 feature is enabled is properly set when growing the file
system from a single block group to larger file system sizes.

Also fix a bug where the block group summary statistics in some cases
when exapnding a sparse_super2 file system.

Finally, accurately calculate the file system metadata overhead of the
last block group in sparse_super2 file systems.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2016-05-10 21:06:15 -04:00
Eric Sandeen f3745728bc resize2fs: clear uninit BG if allocating from new group
If resize2fs_get_alloc_block() allocates from a BLOCK_UNINIT group, we
need to make sure that the UNINIT flag is cleared on both file system
structures which are maintained by resize2fs.  This causes the
modified bitmaps to not get written out, which leads to post-resize2fs
e2fsck errors; used blocks in UNINIT groups, not marked in the block
bitmap.  This was seen on r_ext4_small_bg.

This patch uses clear_block_uninit() to clear the flag,
and my problem goes away.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2016-03-06 21:51:23 -05:00
Theodore Ts'o 6d4bb11487 Merge branch 'maint' into next 2016-03-06 21:30:07 -05:00
Andreas Dilger 539d3ae3da misc: replace comparison macros with inline functions
Replace the EXT2FS_RB_EMPTY_ROOT(), EXT2FS_RB_EMPTY_NODE(),
EXT2FS_RB_CLEAR_NODE(), IS_BLOCK_BM(), IS_INODE_BM(), and
IS_INODE_TB() macros with static inline functions to avoid
suprious compiler warnings with clang:

	pass1.c:618:28: warning: equality comparison with extraneous
		parentheses [-Wparentheses-equality]
	if ((*((__u32 *)(entry)) == 0UL)) {
             ~~~~~~~~~~~~~~~~~~~~^~~~~~
	pass1.c:618:28: note: remove extraneous parentheses
		around the comparison to silence this warning
	if ((*((__u32 *)(entry)) == 0UL)) {
				 ^~
	pass1.c:618:28: note: use '=' to turn this equality
		comparison into an assignment
	if ((*((__u32 *)(entry)) == 0UL)) {
				 ^~
				 =

The static inline functions should compile identically, and allow
some extra compile-time checking for the arguments over macros.

Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2016-03-06 21:23:40 -05:00
Darrick J. Wong 21e37be65b resize2fs: clean up feature test macros with predicate functions
Create separate predicate functions to test/set/clear feature flags,
thereby replacing the wordy old macros.  Furthermore, clean out the
places where we open-coded feature tests.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2015-10-24 01:23:30 -04:00
Theodore Ts'o 25f291c9b3 Eliminate unused parameter warnings from Android build
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2015-07-13 09:12:23 -04:00
Darrick J. Wong f21c30d90c resize2fs: fix regression test to not depend on ext4.ko being loaded
The behavior of the r_fixup_lastbg_big test varies depending on
whether or not ext4.ko is loaded and supports lazy_itable_init.  This
makes checking the bg flags after resize2fs hard to predict, so put in
a way to force resize2fs to zero the inode tables, and compare the
output based on lazy_itable_init == 0.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2015-01-27 13:10:58 -05:00
Theodore Ts'o 560080272f Merge branch 'maint' into next 2015-01-19 16:37:04 -05:00
Darrick J. Wong c916e5248b Fix clang warning and a resource leak
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2015-01-19 16:31:49 -05:00
Darrick J. Wong fe12931f8f resize2fs: convert fs to and from 64bit mode
resize2fs does its magic by loading a filesystem, duplicating the
in-memory image of that fs, moving relevant blocks out of the way of
whatever new metadata get created, and finally writing everything back
out to disk.  Enabling 64bit mode enlarges the group descriptors,
which makes resize2fs a reasonable vehicle for taking care of the rest
of the bookkeeping requirements, so add to resize2fs the ability to
convert a filesystem to 64bit mode and back.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Cc: TR Reardon <thomas_reardon@hotmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-12-15 12:45:05 -05:00
Darrick J. Wong 538ef36326 resize2fs: don't play stupid games with the block count
While it may be true that playing games with old_fs' block count
during a grow operation shuts up a bunch of warnings, resize2fs
doesn't actually expand the group descriptor array to match the size
we're artificially stuffing into old_fs, which means that if we
actually need to allocate a block out of the larger fs (i.e. we're in
desperation mode), ext2fs_block_alloc_stats2() scribbles on the heap,
leading to crashes if you're lucky and FS corruption if not.

So, rip that piece out and turn off com_err warnings properly and add
a test case to deal with growing a nearly full filesystem.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-12-14 22:13:09 -05:00
Darrick J. Wong 1862ef7200 resize2fs: set bg flags and unused inode count when resizing
Recalculate the unused inode count and the block/inode uninit flags
when resizing a filesystem.  This can speed up future e2fsck runs
considerably and will reduce mount times.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-12-14 21:12:37 -05:00
Darrick J. Wong 8e98899b2f resize2fs: use old_fs to detect per-bg metadata blocks to free
When shrinking a filesystem, resize2fs wants to free per-bg metadata
blocks that are no longer needed.  This behavior is gated on whether
there's a superblock in the group as told by new_fs.  The check really
should be against old_fs, since we're effectively freeing blocks out
of old_fs in the transition to new_fs, but prior to sparse_super2 this
didn't matter since superblocks didn't move, so it didn't matter.

Under sparse_super2, however, there's a superblock in the last group,
so now we need to change the test to use old_fs as it should.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-12-14 21:09:29 -05:00
Darrick J. Wong fb30384777 resize2fs: quickly rewrite extent blocks when moving an inode w/ metadata_csum
When we're moving an inode on a metadata_csum filesystem, we need to
rewrite the checksum of all interior nodes of the extent tree.  The
current code does this inefficiently via set_bmap, but we can do this
more efficiently through direct iteration of the extent tree.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-12-13 22:08:18 -05:00
Darrick J. Wong 4495c5a0eb resize2fs: don't exit if shrinking sparse_super2 fs to one bg
If we're shrinking a sparse_super2 filesystem to a single block group,
the superblock will be in block 0.  This is perfectly valid (for block
group 0 with a blocksize > 1024) so don't exit.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-12-13 22:06:15 -05:00
Darrick J. Wong 180f376b04 misc: fix compiler warnings and minor build errors
Fix some gcc-4.8 warnings and other problems that broke the build.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-11-07 21:23:41 -05:00
Darrick J. Wong 08c8e319e3 libext2fs/e2fsck: refactor everyone who writes zero blocks to disk
Convert all call sites that write zero blocks to disk to use
ext2fs_zero_blocks2() since it can use Linux's zero out feature to do
the writes more quickly.  Reclaim the zero buffer at freefs time and
make the write-zeroes fallback use a larger buffer.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-10-13 04:31:17 -04:00
Theodore Ts'o 68083287ae Merge branch 'maint' into next
Conflicts:
	e2fsck/unix.c
2014-08-10 19:33:31 -04:00
Theodore Ts'o c82815e509 resize2fs: disable the meta_bg feature if necessary
When shrinking a file system, if the number block groups drops below
the point where we started using the meta_bg layout, disable the
meta_bg feature and set s_first_meta_bg to zero.  This is necessary to
avoid creating an invalid/corrupted file system after the shrink.

Addresses-Debian-Bug: #756922

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reported-by: Marcin Wolcendorf <antymat+debian@chelmska.waw.pl>
Tested-by: Marcin Wolcendorf <antymat+debian@chelmska.waw.pl>
2014-08-10 16:21:08 -04:00
Theodore Ts'o 50972e1f7d Merge branch 'maint' into next
Conflicts:
	debugfs/debugfs.c
	e2fsck/pass5.c
2014-07-26 09:46:50 -04:00
Theodore Ts'o 1e33a8b408 Fix 32/64-bit overflow when multiplying by blocks/clusters per group
There are a number of places where we need convert groups to blocks or
clusters by multiply the groups by blocks/clusters per group.
Unfortunately, both quantities are 32-bit, but the result needs to be
64-bit, and very often the cast to 64-bit gets lost.

Fix this by adding new macros, EXT2_GROUPS_TO_BLOCKS() and
EXT2_GROUPS_TO_CLUSTERS().

This should fix a bug where resizing a 64bit file system can result in
calculate_minimum_resize_size() looping forever.

Addresses-Launchpad-Bug: #1321958

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-07-26 07:40:36 -04:00
Theodore Ts'o 7f7d1cb462 Merge branch 'maint' into next
Conflicts:
	debugfs/set_fields.c
	tests/f_mmp/script
	tests/f_mmp_garbage/script
	tests/m_mmp/script
	tests/t_mmp_1on/script
	tests/t_mmp_2off/script
2014-07-06 00:09:27 -04:00
Lukas Czerner 47fee2ef6a e2fsprogs: introduce ext2fs_close_free() helper
Currently there are many uses of ext2fs_close() which might be wrong.
First of all ext2fs_close() does not set the ext2_filsys pointer to NULL
so the caller is responsible for clearing it, however there are some
cases there we do not do it.

Second of all very small number of users of ext2fs_close() actually
check the return value. If there is a problem in ext2fs_close() it will
not even free the ext2_filsys structure, but majority of users expect it
to do so.

To fix both problems this commit introduces a new helper
ext2fs_close_free() which will not only check for the return value and
free the ext2_filsys structure if the call to ext2fs_close2() failed,
but it will also set the ext2_filsys pointer to NULL.

Replace every use of ext2fs_close() in e2fsprogs tools with
ext2fs_close_free() - there is no real reason to keep using
ext2fs_close().

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
2014-07-05 21:06:48 -04:00
Theodore Ts'o ba08cb996a Merge branch 'maint' into next
Conflicts:
	debugfs/Makefile.in
	debugfs/debug_cmds.ct
	debugfs/debugfs.c
	debugfs/debugfs.h
	e2fsck/Makefile.in
	misc/Makefile.in
	misc/mke2fs.c
2014-05-13 11:01:07 -04:00
Darrick J. Wong 1244cacc70 resize2fs: fix sanity check in reserve_sparse_super2_last_group()
In reserve_sparse_super2_last_group, the old_desc check should only be
performed if ext2fs_super_and_bgd_loc2() gave us a location -- a
return value of 0 means that there is no old-style GDT block.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-05-13 10:13:40 -04:00
Theodore Ts'o c9bc7484cc Merge branch 'maint' into next
Conflicts:
	e2fsck/pass1.c
2014-05-11 18:30:11 -04:00
Theodore Ts'o cd84e9a337 resize2fs: fix inode table move for the backwards move case
In the case where the new location of the inode table is before the
old inode table, the optimization which tries to optimize zero block
moves breaks.  Fix it.

This fixes a bug that was tickled by the reproduction described in the
previous commit.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-05-04 23:15:49 -04:00
Theodore Ts'o 45a78b88db resize2fs: refine minimum required blocks for flex_bg file systems
The previous commit exposed bugs in the calculation for flex_bg file
systems.  The problem is that since (by default) we keep the metadata
blocks for the flex_bg in the first block group of the flex_bg, and
because we don't want to overwrite metadata blocks used by the
original file system with data blocks make life easier in case the
resize is aborted for some reason, we need to treat all of the
metadata blocks in the existing flex_bg has in use for the purposes of
calculate_minimum_resize_size().

Even though this means we need to reserve more data blocks to avoid
running out of space, the net result of these two commits is a net
savings in how much we can shrink a file system.

Using the following test sequence:

	mke2fs -F -t ext4 /tmp/foo.img 2T
	resize2fs -M /tmp/foo.img
	resize2fs -M /tmp/foo.img
	resize2fs -M /tmp/foo.img

Here is the comparison in the resulting file systems between the old
and new resize2fs (units are in 4k blocks):

                resize #1  resize #2   resize #3
old resize2fs    1117186     45679       43536
new resize2fs      48784     37413       37392

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-05-04 23:15:49 -04:00
Theodore Ts'o b4f30fcfe6 resize2fs: fix overly-pessimistic calculation of minimum size required
For extent-mapped file systems, we need to reserve some extra space in
case we need to grow the extent tree.  Calculate the safety margin
more intelligently, so we don't overestimate the amount of space
required.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reported-by: Dmitry Monakhov <dmonakhov@openvz.org>
2014-05-04 23:15:49 -04:00
Darrick J. Wong d05c9c7ace resize2fs: add inline dirs for remapping
When we're looking for directory blocks for the inode remapping step,
we need to include inline_data directories in the remap process.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-14 09:31:04 -04:00
Theodore Ts'o 3ba59bf649 Merge branch 'maint' into next 2014-02-23 23:07:38 -05:00
Darrick J. Wong 118d3f0b58 resize2fs: don't free in-use clusters when moving blocks
When we're moving blocks around the filesystem, ensure that freeing
the old blocks only frees the clusters if they're not in use by other
metadata.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-02-23 20:54:58 -05:00
Darrick J. Wong 01d6aa9d87 resize2fs: during shrink, don't free in-use bg data clusters
When freeing a block group descriptor block, be careful not to free
metadata clusters belonging to other groups!

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-02-23 20:52:24 -05:00
Theodore Ts'o dc084ab7b1 Merge branch 'maint' into next 2014-02-20 21:39:29 -05:00
Lukas Czerner 585bca68f8 Set pointer to NULL after ext2fs_free
ext2fs_free() does not set the ext2_filsys pointer to null so the
caller is responsible to setting it himself if it is needed.

This patch fixes some places where caller did not set ext2_filsys
pointer to NULL after ext2fs_free() which might result in use after
free.  Fix it.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-02-20 20:54:34 -05:00
Theodore Ts'o f0996c12d5 Merge branch 'maint' into next
Conflicts:
	lib/ext2fs/alloc.c
	misc/mke2fs.c
2014-01-30 19:05:16 -05:00
Theodore Ts'o 65c6c3e06f Add support for new compat feature "sparse_super2"
In practice, it is **extremely** rare for users to try to use more
than the first backup superblock located at the beginning of block
group #1.  (i.e., at block number 32768 for file systems with a 4k
block size).  This new compat feature restricts the backup superblock
to block group #1 and the last block group in the file system.

Aside from reducing the overhead of the file system by a small number
of blocks, by eliminating the rest of the backup superblocks, it
allows us to have a much more flexible metadata layout.  For example,
we can force all of the allocation bitmaps and inode table blocks to
the beginning of the disk, which allows most of the disk to be
exclusively used for contiguous data blocks.

This simplifies taking advantage of certain HDD specific features,
such as Shingled Magnetic Recording (aka Shingled Drives), and the
TCG's OPAL Storage Specification where having a simple mapping between
LBA block ranges and the data blocks used by the file system can make
life much simpler.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-01-30 13:58:18 -05:00
Theodore Ts'o 2ffccc82c4 Merge branch 'maint' into next
Conflicts:
	lib/ext2fs/csum.c
2013-12-26 00:30:56 -05:00
Theodore Ts'o f404167dda Clean up sparse warnings
Mostly by adding static and removing excess extern qualifiers.  Also
convert a few remaining non-ANSI function declarations to ANSI.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2013-12-16 18:56:36 -05:00
Theodore Ts'o e4681bca17 Merge branch 'maint' into next
Conflicts:
	debugfs/debugfs.8.in
2013-12-16 01:35:56 -05:00