Commit Graph

1964 Commits (bbd2f78cf63ab4c635d76073605d6fb1a30c277c)

Author SHA1 Message Date
Darrick J. Wong f99143146a libext2fs: avoid pointless EA block allocation
Use qsort to move the inlinedata attribute to the front of the list
and the empty entries to the end.  Then we can use handle->count to
decide if we're done writing xattrs, which helps us to avoid the
situation where we're midway through the attribute list, so we
allocate an EA block to store more, but have no idea that there's
actually nothing left in the list.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2015-01-27 13:09:52 -05:00
Darrick J. Wong 366d299fe7 libext2fs: initialize i_extra_isize when writing EAs
If i_extra_isize is zero when we try to write extended attributes,
we'll end up writing the EA magic into the i_extra_isize field, which
causes a subsequent crash on big endian systems (when we try to write
0xEA02 bytes past the inode!).  Therefore when the field is zero, set
i_extra_isize to the desired extra_isize size, zero those bytes, and
write the EAs after the end of the extended inode.

v2: Don't bother if we have 128b inodes, and ensure that the value
is 32b-aligned so that the EA magic starts on a 32b boundary.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2015-01-27 10:59:19 -05:00
Theodore Ts'o 22f22ab1d2 Reserve the codepoints for the new INCOMPAT feature ENCRYPT
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2015-01-26 10:27:41 -05:00
Enrico Scholz 1ceb8093dc buildsystem: use 'chmod a-w' instead of 'chmod -w'
'chmod -w' is not portable and can break the build:

| chmod: chmod: ss_err.h: new permissions are r--rw-r--, not r--r--r--
| ss_err.h: new permissions are r--rw-r--, not r--r--r--
| chmod: ss_err.c: new permissions are r--rw-r--, not r--r--r--
| make[2]: *** [ss_err.h] Error 1

This happens because 'chmod -w' is affected by umask. Issue can be
reproduced e.g. by

$ mkdir /tmp/foo
$ setfacl -m dⓂ️rwx /tmp/foo

$ umask 022
$ touch /tmp/foo/x
$ chmod -w /tmp/foo/x
chmod: /tmp/foo/x: new permissions are r--rw-r--, not r--r--r--

Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2015-01-23 12:05:19 -05:00
Theodore Ts'o 560080272f Merge branch 'maint' into next 2015-01-19 16:37:04 -05:00
Darrick J. Wong c916e5248b Fix clang warning and a resource leak
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2015-01-19 16:31:49 -05:00
Theodore Ts'o 9a32411732 Merge branch 'maint' into next
Conflicts:
	lib/ext2fs/inode.c
2014-12-25 23:43:10 -05:00
Theodore Ts'o 13f450addb libext2fs: add sanity check for an invalid itable_used value in inode scan code
If the number of unused inodes is greater than number of inodes a
block group, this can cause an e2fsck -n run of the file system to
crash.

We should add more checks to e2fsck to detect this case directly, but
this will at least protect progams (tune2fs, dump, etc.) which use the
inode_scan abstraction from crashing on an invalid file system.

Addresses-Debian-Bug: #773795

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-12-25 23:29:19 -05:00
Darrick J. Wong 413b5c76d8 libext2fs: speed up the max extent depth api call
The maximum extent tree depth really only depends on the filesystem
block size, so cache the last result if possible.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-12-15 12:26:57 -05:00
Darrick J. Wong ffe1b28dea libext2fs: add a way to check the theoretical maximum extent tree depth
Add an API so that client programs can discover a reasonable maximum
extent tree depth.  This will eventually be used by e2fsck as one of
the criteria to decide if an extent-based file should have its extent
tree rebuilt.

Turn some related magic numbers into constants while we're at it.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-12-13 21:13:40 -05:00
Darrick J. Wong 6509eebb63 libext2fs: set interior tree block goal more intelligently
When we're splitting an extent node, try to allocate the new interior
tree block just prior to the first extent in the block we're trying to
split.  The previous logic only set a goal block if we had to split
both the current node and its parent, which is somewhat infrequent.
When that would happen, the goal would start at zero, leading to poor
locality.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-12-13 20:14:14 -05:00
Darrick J. Wong 7b486ec08c libext2fs: find inode goal when allocating blocks
Try to be a little smarter about where we go to allocate blocks for a
inode.  For a given inode and logical offset, set the goal as if the
file were physically continuous.  If it's bmapped, just start looking
at wherever lblk 0 is.  If that's not possible (the file has no
lblk>pblk mappings, inline data, etc.) then start looking in the
inode's block group.

[ Fixed memory leak --tytso ]

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-12-13 20:07:13 -05:00
Theodore Ts'o bc57b123d6 libext2fs: use block_buf in ext2fs_alloc_block2() if it is provided
If the caller supplies a buffer to ext2fs_alloc_block2(), use it
instead of calling ext2fs_zero_blocks2().

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-12-12 22:12:45 -05:00
Darrick J. Wong 0a92af260d libext2fs: use a dynamically sized block zeroing buffer
Dynamically grow the block zeroing buffer to a maximum of 4MB, and
allow callers to provide their own zeroed buffer in
ext2fs_zero_blocks2().

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-12-12 19:28:35 -05:00
Dmitry Monakhov e50e985d6a ext2fs: fix integer overflow in rb_get_bmap_range
bmap_rb_extent is defined as __u64:blk __u64:count.  So count can
exceed INT_MAX on populated filesystems.

TESTCASE: xfstest ext4/004

Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-12-11 17:57:35 -05:00
Darrick J. Wong dc7b8dad99 libext2fs: file IO routines should handle uninit blocks
The file IO routines do not handle uninit blocks at all.  The read
method should check for the uninit flag and return a buffer of zeroes,
and the write routine should convert unwritten extents.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-12-02 22:57:14 -05:00
Darrick J. Wong 3548bb64b5 libext2fs: refactor extent head creation
Don't open-code the creation of the extent tree header, since
ext2fs_extent_open2() knows how to take care of this.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-12-02 22:55:04 -05:00
Darrick J. Wong 54f6faf7f2 libext2fs: don't report garbage inodes with really large inodes
If the inode size is large enough that there are fewer than two inodes
per block, don't report an inode checksum failure as a garbage inode
during the scan because the "more than half are broken" criteria that
we use to decide if a block of inodes is garbage doesn't really apply.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-12-02 22:17:10 -05:00
Theodore Ts'o bbf29ce6e9 Merge branch 'maint' into next 2014-12-02 22:15:25 -05:00
Darrick J. Wong c9d6c22ded libext2fs: don't allow alloc_stats on bad inode/block numbers
Don't allow callers to feed bad block/inode numbers to
ext2fs_*_alloc_stats2, because evil callers (<cough>resize2fs<cough>)
can corrupt library state this way, leading to a crash.

(There will be a subsequent patch to resize2fs to fix its bad
behavior.)

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-11-17 17:59:42 -05:00
Darrick J. Wong c0ff3a21b6 libext2fs: set BLOCK_UNINIT for non-last blockgroups if all blocks are free
Set BLOCK_UNINIT in any group whose blocks are all unused, so long as
it isn't the last group.  This helps us speed up future e2fsck runs
and mounts because we don't need to read or checksum block bitmaps for
these groups.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-11-17 17:46:13 -05:00
Darrick J. Wong 407916f5af libext2fs: fix endian handling error; reduce fragmentation some
If we're going to read the "nr - 1" entry in an indirect block for use
as a "goal" input to the block allocator, we need to byteswap the
entry.  While we're at it, if we're allocating blocks for the zeroth
entry in the indirect block, we might as well use the indirect block
as the starting point to try to reduce fragmentation.

(d_fallocate_blkmap will test this...)

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-11-07 21:27:53 -05:00
Darrick J. Wong 180f376b04 misc: fix compiler warnings and minor build errors
Fix some gcc-4.8 warnings and other problems that broke the build.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-11-07 21:23:41 -05:00
Darrick J. Wong 12406b37b2 libext2fs: fix endian checking bits
Commit 3e683eef93 ("define bitwise types and annotate conversion
routines") broke the build on various platforms.  Turns out that
crossing our fingers wasn't such a good idea, so just define it
separately.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-11-05 11:08:32 -05:00
Darrick J. Wong bab25cb7a7 libext2fs: zero the EA block buffer before filling it
When writing an extended attribute (EA) block, it's quite possible
that the EA formatting code will not write the entire buffer.
Therefore, we must zero the buffer beforehand to avoid writing random
heap contents to disk.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: Sami Liedes <sami.liedes@iki.fi>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-11-04 11:47:30 -05:00
Theodore Ts'o dfa667dab6 Merge branch 'maint' into next
Conflicts:
	lib/ext2fs/dir_iterate.c
2014-11-04 11:46:55 -05:00
Darrick J. Wong 8d5324c43f libext2fs: don't memcpy identical pointers when writing a cache block
Sami Liedes found a scenario where we could memcpy incorrectly:

If a block read fails during an e2fsck run, the UNIX IO manager will
call the io->read_error routine with a pointer to the internal block
cache.  The e2fsck read error handler immediately tries to write the
buffer back out to disk(!), at which point the block write code will
try to copy the buffer contents back into the block cache.  Normally
this is fine, but not when the write buffer is the cache itself!

So, plumb in a trivial check for this condition.  A more thorough
solution would pass a duplicated buffer to the IO error handlers, but
I don't know if that happens frequently enough to be worth the extra
point of failure.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: Sami Liedes <sami.liedes@iki.fi>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-11-04 11:43:08 -05:00
Darrick J. Wong dab7435917 libext2fs: directory iteration mustn't walk off the buffer end
When we're iterating a directory, the loop control code reads the
length of the next directory record, failing to account for the fact
that there must be at least 8 bytes (the minimum size of a directory
entry) left in the buffer to read the next directory record.  Fix the
loop conditional so that we don't read off the end of the buffer.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: Sami Liedes <sami.liedes@iki.fi>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-11-04 11:39:51 -05:00
Eric Sandeen a7db275f31 quotaio: annotate & fix up for sparse endian checker
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-11-04 11:26:24 -05:00
Eric Sandeen 8f358e58fe libext2: minor sparse endian checker fixup
The sparse checker treats 0 assignments as special, but
doesn't catch a = b = 0; separate them to make it quieter.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-11-04 11:26:23 -05:00
Eric Sandeen 1224c0d3ea endian-annotate most on-disk structures
This annotates most on-disk structures for endianness;
however it does not annotate some, like the superblock, inodes,
mmp, etc, as these are swapped in-place at this point.  This is
a little inconsistent, but should help catch some endian mistakes,
at least.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-11-04 11:24:56 -05:00
Eric Sandeen 387e03160c libext2fs: fix endian handling of ext3_extent_header in inline_data
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
2014-11-04 11:24:50 -05:00
Eric Sandeen 3e683eef93 define bitwise types and annotate conversion routines
This lays the groundwork for sparse-checking e2fsprogs for
endianness; defines bitwise types, and fixes up the ext2fs_*
swapping routines to do the proper casts.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-11-04 11:24:44 -05:00
Theodore Ts'o 8b779489ea Merge branch 'maint' into next
Conflicts:
	configure
2014-11-04 11:20:09 -05:00
Eric Sandeen 160f131dee libext2fs: fix endian handling of ext3_extent_header
This turned up when trying to resize a filesystem containing
a file with many extents on PPC64.

Fix all locations where ext3_extent_header members aren't
handled in an endian-safe manner.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
2014-11-04 11:12:45 -05:00
Theodore Ts'o 441eb337a8 util: allow subst to build on systems that do not have utimes()
Make subst more portable so it can deal with such oler systems that do
not have utimes().  Note that it is important that subst build
correctly without an autoconf-generated config.h (since that is what
happens on a cross-compile), as well as using whatever features are
available as determined by autoconf when doing a native build.  We
currently assume the presence of utime(), but not utimes() or
futimes().

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-10-19 22:13:09 -04:00
Darrick J. Wong 08c8e319e3 libext2fs/e2fsck: refactor everyone who writes zero blocks to disk
Convert all call sites that write zero blocks to disk to use
ext2fs_zero_blocks2() since it can use Linux's zero out feature to do
the writes more quickly.  Reclaim the zero buffer at freefs time and
make the write-zeroes fallback use a larger buffer.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-10-13 04:31:17 -04:00
Theodore Ts'o 074931ab76 libext2fs: use ~0UL instead of -1UL to avoid static checker warnings
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-09-21 15:36:57 -04:00
Darrick J. Wong b291c11f08 misc: use libmagic when libblkid can't identify something
If we're using check_plausibility() to try to identify something that
obviously isn't an ext* filesystem and libblkid doesn't know what it
is, try libmagic instead.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-09-20 23:42:19 -04:00
Darrick J. Wong c8b20b40eb misc: add plausibility checks to debugfs/tune2fs/dumpe2fs/e2fsck
If any of these utilities detect a bad superblock magic, call
check_plausibility to see if blkid can identify the passed-in argument
as something else (xfs, partition, etc.) in the hopes of catching a
user error.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-09-19 23:44:31 -04:00
Darrick J. Wong b598c517b3 misc: move check_plausibility into a separate file
Move check_plausibility() into a separate file so that various
programs can use it without having to declare useless global variables
that the util.c functions seem to require.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-09-19 13:10:21 -04:00
Andreas Dilger ca209dc625 ext2fs: add readahead method to improve scanning
Add a readahead method for prefetching ranges of disk blocks.  This is
useful for inode table scanning, and other large contiguous ranges of
blocks, and may also prove useful for random block prefetch, since it
will allow reordering of the IO without waiting synchronously for the
reads to complete.

It is currently using the posix_fadvise(POSIX_FADV_WILLNEED)
interface, as this proved most efficient during our testing.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-09-19 12:16:08 -04:00
Theodore Ts'o cc0d983303 Fix build failures due to missing $(SYSLIBS)
Two link lines were missing $(SYSLIBS), which is needed for dietlibc.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-09-19 01:05:14 -04:00
Theodore Ts'o 1bbea9c909 Merge branch 'maint' into next 2014-09-18 21:28:59 -04:00
Darrick J. Wong d9112409a2 misc: zero s_jnl_blocks when adding journal online or removing external journal
Erase s_jnl_blocks when removing an external journal, or adding an
internal journal online.  We can't add the backup for the internal
journal because we have no good way to get the indirect block or ETB
addresses, so the best we can do is hope that the user runs e2fsck,
which will correct that.  We are motivated to erase during external
journal removal to state emphatically that there's no journal.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: thomas_reardon@hotmail.com
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-09-18 21:24:26 -04:00
Theodore Ts'o a2c664ae90 lib/ext2fs: fix Makefile to avoid a build splat when building without VPATH
When building in the source tree, the order of the includes caused the
compiling of debugfs/journal.c while in the lib/ext2fs directory to
find the version in lib/ext2fs instead of the desired version in
e2fsck/jfs_user.h.

We need to eventually get rid of this whole mess and have only one
jfs_user.h and build the journal-related functions once in an internal
library which is used only by e2fsprogs progams.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reported-by: "Darrick J. Wong" <darrick.wong@oracle.com>
2014-09-11 19:15:22 -04:00
Darrick J. Wong 551ab6d8e0 libext2fs: check ea value offset when loading
When reading extended attributes, check e_value_offs to make sure that
it starts in the value area and not the name area.  The attached test
case image will crash the kernel if it is mounted and you append more
than 4096 bytes of data to /a, due to insufficient validation.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-09-11 18:10:23 -04:00
Darrick J. Wong 463eb92131 debugfs: add the ability to write transactions to the journal
Extend debugfs with the ability to create transactions and replay the
journal.  This will eventually be used to test kernel recovery and
metadata_csum recovery.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-09-11 16:52:37 -04:00
Darrick J. Wong 759c46cf45 debugfs: create journal handling routines
Create a journal.c with routines adapted from e2fsck/journal.c to
handle opening and closing the journal, and setting up the
descriptors, and all that.  Unlike e2fsck's versions which try to
identify and fix problems, the routines here have no way to repair
anything.

[ Modified by tytso to fold debugfs/jfs_user.h into e2fsck/jfs_user.h,
  so we don't have to copy recovery.c and revoke.c into debugfs. --tytso ]

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-09-11 16:44:10 -04:00
Darrick J. Wong e690eae513 misc: zero s_jnl_blocks when removing internal journal
When we're removing the internal journal (broken journal, turning it
off, or adding an external journal), zero s_jnl_blocks so that they
can't be picked up by accident later.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Cc: TR Reardon <thomas_reardon@hotmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-09-11 12:40:55 -04:00
Darrick J. Wong fc06f25a10 libext2fs: write_journal_inode should check iterate return value
When creating a journal inode, check the return value from
block_iterate3() because otherwise we fail to capture errors such as
being unable to allocate an extent tree block, which leads to e2fsck
creating broken journals.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-09-11 12:40:54 -04:00
Darrick J. Wong f92c600c09 libext2fs: report bad magic over bad sb checksum
We don't want ext2fs_open2() to report bad sb checksum on something
that's not even an ext* superblock.  This apparently happens pretty
easily if we try to open an XFS filesystem.  Thus, make it so that a
bad magic number code always trumps the sb checksum error code.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-09-11 12:40:54 -04:00
Darrick J. Wong 38d5adf339 e2fsck/debugfs: fix descriptor block size handling errors with journal_csum
It turns out that there are some serious problems with the on-disk
format of journal checksum v2.  The foremost is that the function to
calculate descriptor tag size returns sizes that are too big.  This
causes alignment issues on some architectures and is compounded by the
fact that some parts of jbd2 use the structure size (incorrectly) to
determine the presence of a 64bit journal instead of checking the
feature flags.  These errors regrettably lead to the journal
corruption reported by Mr. Reardon.

Therefore, introduce journal checksum v3, which enlarges the
descriptor block tag format to allow for full 32-bit checksums of
journal blocks, fix the journal tag function to return the correct
sizes, and fix the jbd2 recovery code to use feature flags to
determine 64bitness.

Add a few function helpers so we don't have to open-code quite so
many pieces.

Switching to a 16-byte block size was found to increase journal size
overhead by a maximum of 0.1%, to convert a 32-bit journal with no
checksumming to a 32-bit journal with checksum v3 enabled.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: TR Reardon <thomas_reardon@hotmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-09-11 12:40:54 -04:00
Theodore Ts'o 330cebc0e9 Merge branch 'maint' into next
Conflicts:
	debugfs/debugfs.c
	e2fsck/Makefile.in
	lib/ext2fs/Makefile.in
	tests/test_config
2014-09-11 12:40:43 -04:00
Michael Forney 943e21cda8 compile_et: Allow user to override ET_DIR
Signed-off-by: Michael Forney <forney@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-09-08 19:02:12 -04:00
Michael Forney 53904ae543 Apply LDFLAGS when building tests
Signed-off-by: Michael Forney <forney@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-09-08 19:00:24 -04:00
Michael Forney 60abcd7394 tests: Add to LD_LIBRARY_PATH instead of overriding
Signed-off-by: Michael Forney <forney@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-09-08 18:55:42 -04:00
Darrick J. Wong 97f168b67e e2fsck: resync jbd2 revoke code from Linux 3.16
Synchronize e2fsck's copy of revoke.c with the kernel's copy in
fs/jbd2.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-26 23:43:20 -04:00
Darrick J. Wong 13af4b93fb e2fsck: resync jbd2 recovery code from Linux 3.16
Synchronize e2fsck's copy of recovery.c with the kernel's copy in
fs/jbd2.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-26 23:32:14 -04:00
Darrick J. Wong 2432a41a58 libext2fs: fix problems with LE<->BE conversions on BE platforms
Fix more problems that I found when testing on ppc64:

- Inode swap cut and paste error leads to immutable inodes being
  detected as inlinedata inodes, leading to e2fsck incorrectly barfing
  on i_block[] contents.

- Superblock csum/verify must be aware of the fs->super byte order
  when checking for metadata_csum feature flag.  (Hint: in _openfs(),
  fs->super is in LE order for the first csum verification)

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-24 22:01:36 -04:00
Darrick J. Wong 18b234b121 libext2fs: byteswap inode when performing the sanity scan
On BE platforms, we need to swap the inode bytes after doing the
checksum verification but before looking at i_blocks.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-24 22:00:56 -04:00
Darrick J. Wong c4c9bc590c misc: fix gcc warnings
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-24 12:22:11 -04:00
Darrick J. Wong 8a17df7195 libext2fs: create inlinedata symlinks
Add to ext2fs_symlink the ability to create inline data symlinks.

[ Modified by tytso to add more logging to the test script ]

Suggested-by: Pu Hou <houpu.hp@alibaba-inc.com>
Cc: Pu Hou <houpu.hp@alibaba-inc.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-24 12:21:16 -04:00
Theodore Ts'o 68083287ae Merge branch 'maint' into next
Conflicts:
	e2fsck/unix.c
2014-08-10 19:33:31 -04:00
Darrick J. Wong 657f508cbf libext2fs: fix memory leak when failing to iterate inline_data directory
The xattr_get method returns to us a pointer to a buffer containing
the EA value.  If for some reason we decide to fail out of iterating
the EA part of an inline-data directory, we must free the buffer that
xattr_get passed to us (via inline_data_ea_get).

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-10 18:32:21 -04:00
Darrick J. Wong 996999a1a4 libext2fs: don't fail inline data operations if there's no EA
Fix up the rest of the inline data code not to complain if there's no
EA, since it's possible that there's no EA because we're in the
process of creating an inline data file.  Also, don't return an error
code when removing a nonexistent EA, because there's no reason to.

Furthermore, if we write less than 60 bytes of inline data, remove the
EA to avoid wasting space.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-10 18:31:04 -04:00
Darrick J. Wong cffe0221eb libext2fs: strict inline data overwrite should not return ENOSPC
If we're doing a strict overwrite (same data size) of data in an
inline data file, we should be able to skip the size check.  If the
in-core EA representation is fine but the on-disk EA is slightly
corrupt (this happens when fixing minor errors in an inline dir), the
ext2fs_xattr_inode_max_size() call, which reads the disk EA, can lead
us to think that there's no space when in reality there is no issue
with doing a strict overwrite.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-10 18:27:10 -04:00
Darrick J. Wong cd971869d7 misc: fix various endianness problems with inline_data
The inline data code fails to perform endianness conversions correctly
or at all in a number of places, so fix this so that big-endian
machines function properly.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-10 18:22:54 -04:00
Darrick J. Wong 88334ce084 libext2fs/e2fsck: don't run off the end of the EA block
When we're (a) reading EAs into a buffer; (b) byte-swapping EA
entries; or (c) checking EA data, be careful not to run off the end of
the memory buffer, because this causes invalid memory accesses and
e2fsck crashes.  This can happen if we encounter a specially crafted
FS image.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-10 18:22:07 -04:00
Darrick J. Wong 71e177a2a5 libext2fs: check EA value offset
Perform a little more sanity checking of EA value offsets so that we
don't crash while trying to load things from the filesystem.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-10 18:21:16 -04:00
Theodore Ts'o f66e6ce444 libext2fs: avoid buffer overflow if s_first_meta_bg is too big
If s_first_meta_bg is greater than the of number block group
descriptor blocks, then reading or writing the block group descriptors
will end up overruning the memory buffer allocated for the
descriptors.  Fix this by limiting first_meta_bg to no more than
fs->desc_blocks.  This doesn't correct the bad s_first_meta_bg value,
but it avoids causing the e2fsprogs userspace programs from
potentially crashing.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-10 16:20:59 -04:00
Theodore Ts'o 489ff4a2c7 Merge branch 'maint' into next
Conflicts:
	configure
2014-08-08 17:02:34 -04:00
Theodore Ts'o f00948ad1d libext2fs: have UNIX IO manager use pread64/pwrite64
Commit baa3544609 ("libext2fs: have UNIX IO manager use
pread/pwrite) causes a breakage on 32-bit systems where off_t is
32-bits for file systems larger than 4GB.  Fix this by using
pread64/pwrite64 if possible, and if pread64/pwrite64 is not present,
using pread/pwrite only if the size of off_t is at least as big as
ext2_loff_t.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-08 16:42:05 -04:00
Theodore Ts'o b1988056fb libext2fs: place metadata blocks in the last flex_bg so they are contiguous
Place the allocation bitmaps and inode table blocks so they are
adjacent, even in the last flexbg.

Previously, after running "mke2fs -t ext4 DEV 286720", the layout of
the last few block groups would look like this:

Group 32: (Blocks 262145-270336) [INODE_UNINIT, ITABLE_ZEROED]
  Block bitmap at 262145 (+0), Inode bitmap at 262161 (+16)
  Inode table at 262177-262432 (+32)
Group 33: (Blocks 270337-278528) [INODE_UNINIT, BLOCK_UNINIT, ITABLE_ZEROED]
  Block bitmap at 262146 (bg #32 + 1), Inode bitmap at 262162 (bg #32 + 17)
  Inode table at 262433-262688 (bg #32 + 288)
Group 34: (Blocks 278529-286719) [INODE_UNINIT, ITABLE_ZEROED]
  Block bitmap at 262147 (bg #32 + 2), Inode bitmap at 262163 (bg #32 + 18)
  Inode table at 262689-262944 (bg #32 + 544)

Now, they look like this:

Group 32: (Blocks 262145-270336) [INODE_UNINIT, ITABLE_ZEROED]
  Block bitmap at 262145 (+0), Inode bitmap at 262148 (+3)
  Inode table at 262151-262406 (+6)
Group 33: (Blocks 270337-278528) [INODE_UNINIT, BLOCK_UNINIT, ITABLE_ZEROED]
  Block bitmap at 262146 (bg #32 + 1), Inode bitmap at 262149 (bg #32 + 4)
  Inode table at 262407-262662 (bg #32 + 262)
Group 34: (Blocks 278529-286719) [INODE_UNINIT, ITABLE_ZEROED]
  Block bitmap at 262147 (bg #32 + 2), Inode bitmap at 262150 (bg #32 + 5)
  Inode table at 262663-262918 (bg #32 + 518)

This reduces the free space fragmentation in a freshly created file
system.  It also allows the following mke2fs command to succeed:

mke2fs -t ext4 -b 4096 -O ^resize_inode -G $((2**20)) DEV 2130483

(Note that while this allows people to run mke2fs with insanely large
flexbg sizes, this is not a recommended practice, as the kernel may
refuse to resize such a file system while mounted, since it currently
tries to allocate an in-memory data structure based on the size of the
flexbg, and so a file system with a very large flexbg size will cause
the memory allocation to fail.  This will hopefully be fixed in a
future kernel release, but if the goal is to force all of the metadata
blocks to be at the beginning of the file system, it's better to use
the packed_meta_blocks configuration parameter in mke2fs.conf.)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-04 18:58:23 -04:00
Theodore Ts'o 457e49981e Revert "mke2fs: prevent creation of unmountable ext4 with large flex_bg count"
This reverts commit d988201ef9.

The problem with this commit is that causes common small file system
configurations to fail.  For example:

    mke2fs -O flex_bg -b 4096 -I 1024 -F /tmp/tt 79106
    mke2fs 1.42.11 (09-Jul-2014)
    /tmp/tt: Invalid argument passed to ext2 library while setting
             up superblock

This check in ext2fs_initialize() was added to prevent the metadata
from being allocated beyond the end of the filesystem, but it is
also causing a wide range of failures for small filesystems.

We'll address this in a different way, by using a smarter algorithm
for deciding the layout of metadata blocks for the last flex block
group.

Reported-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-04 18:58:17 -04:00
Darrick J. Wong b9f95911e9 libext2fs: don't cache inodes that fail checksum verification
If an inode fails checksum verification, don't stuff a copy of it in
the inode cache, because this can cause the library to fail to return
the "corrupt inode" error code.

In general, this happens if ext2fs_read_inode_full() is called twice
on an inode with an incorrect checksum.  If fs->flags has
EXT2_FLAG_IGNORE_CSUM_ERRORS set during the first call and *unset*
during the second call, the cache hit during the second call fails to
return EXT2_ET_INODE_CSUM_INVALID as you'd expect.  This happens
during fsck because the first read_inode call happens as part of
check_blocks and the second call happens during inode checksum
revalidation.  A file system with a slightly corrupt non-extent inode
will trigger this.

While we're at it, make the inode read function consistent with the
rest of libext2fs -- copy the metadata object into the caller's buffer
even if it fails checksum verification.  This will help e2fsck avoid a
double re-read later on down the line.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-02 22:49:23 -04:00
Darrick J. Wong 2e9d839156 e2fsck: correctly preserve fs flags when modifying ignore-csum-error flag
When we need to modify the "ignore checksum error" behavior flag to
get us past a library call, it's possible that the library call can
result in other flag bits being changed.  Therefore, it is not correct
to restore unconditionally the previous flags value, since this will
have unintended side effects on the other fs->flags; nor is it correct
to assume that we can unconditionally set (or clear) the "ignore csum
error" flag bit.  Therefore, we must merge the previous value of the
"ignore csum error" flag with the value of flags after the call.

Note that we want to leave checksum verification on as much as
possible because doing so exposes e2fsck bugs where two metadata
blocks are "sharing" the same disk block, and attempting to fix one
before relocating the other causes major filesystem damage.  The
damage is much more obvious when a previously checked piece of
metadata suddenly fails in a subsequent pass.

The modifications to the pass 2, 3, and 3A code are justified as
follows: When e2fsck encounters a block of directory entries and
cannot find the placeholder entry at the end that contains the
checksum, it will try to insert the placeholder.  If that fails, it
will schedule the directory for a pass 3A reconstruction.  Until that
happens, we don't want directory block writing (pass 2), block
iteration (pass 3), or block reading (pass 3A) to fail due to checksum
errors, because failing to find the placeholder is itself a checksum
verification error, which causes e2fsck to abort without fixing
anything.

The e2fsck call to ext2fs_read_bitmaps must never fail due to a
checksum error because e2fsck subsequently (a) verifies the bitmaps
itself; or (b) decides that they don't match what has been observed,
and rewrites them.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-02 22:48:21 -04:00
Darrick J. Wong 68d70624e3 e2fsck: offer to clear inode table blocks that are insane
Add a new behavior flag to the inode scan functions; when specified,
this flag will do some simple sanity checking of entire inode table
blocks.  If all the checksums are ok, we can skip checksum
verification on individual inodes later on.  If more than half of the
inodes look "insane" (bad extent tree root or checksum failure) then
ext2fs_get_next_inode_full() can return a special status code
indicating that what's in the buffer is probably garbage.

When e2fsck' inode scan encounters the 'inode is garbage' return code
it'll offer to zap the inode straightaway instead of trying to recover
anything.  This replaces the previous behavior of asking to zap
anything with a checksum error (strict_csum).

Signed-off-by: Darrick J. Wong <darrick.wong@orale.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-02 22:46:16 -04:00
Darrick J. Wong 49fed79e7c e2fsck: try to salvage extent blocks with bad checksums
Remove the code that would zap an extent block immediately if the
checksum failed (i.e. strict_csums).  Instead, we'll only do that if
the extent block header shows obvious structural problems; if the
header checks out, then we'll iterate the block and see if we can
recover some extents.

Requires a minor modification to ext2fs_extent_get such that the
extent block will be returned in the buffer even if the return code
indicates a checksum error.  This brings its behavior in line with
the rest of libext2fs.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-02 22:32:11 -04:00
Darrick J. Wong 5b9cbd76df libext2fs: check EA block headers when reading in the block
When reading an EA block in from disk, do a quick sanity check of the
block header, and return an error if we think we have garbage.  Teach
e2fsck to ignore the new error code in favor of doing its own
checking, and remove the strict_csums bits while we're at it.

(Also document some assumptions in the new ext_attr code.)

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-02 22:32:11 -04:00
Theodore Ts'o 14207cf60e Merge branch 'maint' into next
Conflicts:
	configure
	misc/Makefile.in
2014-08-02 22:05:03 -04:00
Darrick J. Wong 07c479dd97 libext2fs: when appending to a file, don't split an index block in equal halves
When we're appending an extent to the end of a file and the index
block is full, don't split the index block into two half-full index
blocks because this leaves us with under utilized index blocks, at
least in the fallocate case.  Instead, copy the last extent from the
full block into the new block.  This isn't perfect utilization, but
there's a lot of work involved in teaching extent.c to be able to goto
a nonexistent node in a newly allocated (and empty) extent block.

This patch does not fix the general problem of keeping the extent tree
balanced.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-02 19:18:39 -04:00
Darrick J. Wong baa3544609 libext2fs: have UNIX IO manager use pread/pwrite
If pread/pwrite are present, have the UNIX IO manager use them for
aligned IOs (instead of the current seek -> read/write), thereby
saving us a (minor) amount of system call overhead.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-08-02 19:18:03 -04:00
Theodore Ts'o 643fd7e7aa Merge branch 'maint' into next 2014-07-29 10:53:49 -04:00
Azat Khuzhin 9c2c1e9a3d tune2fs: update journal users while updating fs UUID (with external journal)
When we have fs with external journal device, and updating it's UUID, we
should update UUID in users list for that external journal device.

Before:
$ tune2fs -U clear /tmp/dev
tune2fs 1.42.10 (18-May-2014)
$ dumpe2fs /tmp/dev | fgrep UUID
dumpe2fs 1.42.10 (18-May-2014)
Filesystem UUID:          <none>
Journal UUID:             da1f2ed0-60f6-aaaa-92fd-738701418523
$ dumpe2fs /tmp/journal | fgrep users -A10
dumpe2fs 1.42.10 (18-May-2014)
Journal number of users:  2
Journal users:            0707762d-638e-4bc6-944e-ae8ee7a3359e
                          0ad849df-1041-4f0a-b1c1-2f949d6a1e37

After:
$ sudo tune2fs -U clear /tmp/dev
tune2fs 1.43-WIP (18-May-2014)
$ dumpe2fs /tmp/dev | fgrep UUID
dumpe2fs 1.42.10 (18-May-2014)
Filesystem UUID:          <none>
Journal UUID:             da1f2ed0-60f6-aaaa-92fd-738701418523
$ dumpe2fs /tmp/journal | fgrep users -A10
dumpe2fs 1.42.10 (18-May-2014)
Journal number of users:  2
Journal users:            0707762d-638e-4bc6-944e-ae8ee7a3359e
                          00000000-0000-0000-0000-000000000000

Also add some consts to avoid *magic numbers*:
- UUID_STR_SIZE
- UUID_SIZE
- JFS_USERS_MAX
- JFS_USERS_SIZE

Proposed-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Azat Khuzhin <a3at.mail@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-07-28 20:21:59 -04:00
Azat Khuzhin 7f33024ac2 journal: use consts instead of 1024 and add helper for journal with 1k blocksize
Use EXT2_MIN_BLOCK_SIZE, JFS_MIN_JOURNAL_BLOCKS, SUPERBLOCK_SIZE, and
SUPERBLOCK_OFFSET instead of hardcoded 1024 when it is okay, and also
add a helper ext2fs_journal_sb_start() that will return start of
journal sb with special case for fs with 1k block size.

Signed-off-by: Azat Khuzhin <a3at.mail@gmail.com>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-07-28 20:21:47 -04:00
Ross Burton c84da2eed0 misc: copy extended attributes in populate_fs
When creating a file system using a source directory, also copy any extended
attributes that have been set.

[ Add configure tests for Linux-specific xattr syscalls and add fallback
  when compiling on non-Linux systems. --tytso ]

Signed-off-by: Ross Burton <ross.burton@intel.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-07-27 21:19:24 -04:00
Theodore Ts'o 22302aa320 Merge branch 'maint' into next
Conflicts:
	debugfs/debugfs.c
	e2fsck/pass1.c
2014-07-26 15:57:42 -04:00
Darrick J. Wong 97c607b1a2 libext2fs: provide a function to set inode size
Provide an API to set i_size in an inode and take care of all required
feature flag modifications.  Refactor the code to use this new
function.

[ Moved the function to lib/ext2fs/blk_num.c, which is the rest of
  these sorts of functions live, and renamed it to be
  ext2fs_inode_size_set() instead of ext2fs_inode_set_size() to be
  consistent with the other functions in in blk_num.c -- tytso ]

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-07-26 14:34:56 -04:00
Theodore Ts'o 50972e1f7d Merge branch 'maint' into next
Conflicts:
	debugfs/debugfs.c
	e2fsck/pass5.c
2014-07-26 09:46:50 -04:00
Theodore Ts'o a30a4e93f3 libext2fs: fix free block accounting for 64-bit file systems
We rely on a nasty hack to adjust the free block count where we pass
signed value into ext2fs_free_blocks_count_add(), which takes an
64-bit unsigned value, and relies on overflow and C's signed->unsigned
semantics to do the subtraction.  This works, so long as a 64-bit
signed value is used.

Unfortunately, ext2fs_block_alloc_stats2() and
ext2fs_block_alloc_stats_range(), this is not true, so on a 64-bit
file system, the free blocks accounting can get screwed up.

A simple way to demonstrate the problem is:

mke2fs -F -t ext4 -O 64bit /tmp/foo.img 1M
e2fsck -fy /tmp/foo.img

... which will result in the following e2fsck complaint:

Pass 5: Checking group summary information
Free blocks count wrong (4294968278, counted=982).
Fix? yes

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-07-26 09:25:40 -04:00
Theodore Ts'o 1e33a8b408 Fix 32/64-bit overflow when multiplying by blocks/clusters per group
There are a number of places where we need convert groups to blocks or
clusters by multiply the groups by blocks/clusters per group.
Unfortunately, both quantities are 32-bit, but the result needs to be
64-bit, and very often the cast to 64-bit gets lost.

Fix this by adding new macros, EXT2_GROUPS_TO_BLOCKS() and
EXT2_GROUPS_TO_CLUSTERS().

This should fix a bug where resizing a 64bit file system can result in
calculate_minimum_resize_size() looping forever.

Addresses-Launchpad-Bug: #1321958

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-07-26 07:40:36 -04:00
Theodore Ts'o d4ecec45ab libext2fs: use C99 initializers for the io_manager structure
Using C99 initializers makes the code a bit more readable, and it
avoids some gcc -Wall warnings regarding missing initializers.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-07-26 00:49:14 -04:00
Theodore Ts'o 2dbf34e505 libext2fs: fix rb_resize_bmap to handle the padding bits
The bits between end and real_end are set as a safety measure for the
kernel when it uses the bit scan instructions.  We need to take this
into account when shrinking or growing the block allocation bitmap,
before we can safely use rbtree bitmaps in resize2fs.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-07-26 00:48:29 -04:00
Andreas Dilger 8b90ab2b1c build: fix unused/uninitialized variable warnings
Fix a few warnings about unused and uninitialized variables.

Also fix util/subst.c to include <sys/time.h> to avoid using
undeclared functions gettimeofday() and futimes().

Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-07-25 22:48:01 -04:00
Theodore Ts'o 60203cb171 Merge branch 'maint' into next
Conflicts:
	e2fsck/pass1.c
2014-07-25 08:38:39 -04:00
Darrick J. Wong 203e13cf9c libext2fs: fix bounds check of the bitmap test range in get_free_blocks2
In the loop in ext2fs_get_free_blocks2, we ask the bitmap if there's a
range of free blocks starting at "b" and ending at "b + num - 1".
That quantity is the number of the last block in the range.  Since
ext2fs_blocks_count() returns the number of blocks and not the number
of the last block in the filesystem, the check is incorrect.

Put in a shortcut to exit the loop if finish > start, because in that
case it's obvious that we don't need to reset to the beginning of the
FS to continue the search for blocks.  This is needed to terminate the
loop because the broken test meant that b could get large enough to
equal finish, which would end the while loop.

The attached testcase shows that with the off by one error, it is
possible to throw e2fsck into an infinite loop while it tries to
find space for the inode table even though there's no space for one.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-07-25 07:11:57 -04:00
Theodore Ts'o f9f3050a0a Merge branch 'maint' into next
Conflicts:
	debian/changelog
	e2fsck/pass1.c
	lib/ext2fs/Makefile.in
2014-07-22 14:57:40 -04:00
Theodore Ts'o a34c72ab2d lib/ext2fs: Only build tst_libext2fs for make check
It's only necessary to build tst_libext2fs when running "make check".

Also make sure the links of the tst_* programs are done with
$(ALL_LDFLAGS).

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-07-10 16:26:14 -04:00
Theodore Ts'o 599915f080 Use sys/syscall.h instead of syscall.h
Most systems have a backwards compatibility symlink in
/usr/include/syscall.h to /usr/include/sys/syscall.h, but
sys/syscall.h is the documented location of the header file.  Fix two
locations where we were using <syscall.h> instead of <sys/syscall.h>.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-07-10 15:54:42 -04:00
Theodore Ts'o 7f7d1cb462 Merge branch 'maint' into next
Conflicts:
	debugfs/set_fields.c
	tests/f_mmp/script
	tests/f_mmp_garbage/script
	tests/m_mmp/script
	tests/t_mmp_1on/script
	tests/t_mmp_2off/script
2014-07-06 00:09:27 -04:00
Akira Fujita d988201ef9 mke2fs: prevent creation of unmountable ext4 with large flex_bg count
In mke2fs command, if flex_bg count is too large to filesystem blocks
count, unmountable ext4 which has the out of filesystem block offset
is created (Case1).  Moreover this large flex_bg count causes an
unintentional metadata layout (bmap-imap-itable-bmap-imap-itable .. in
block group) (Case2).

To fix these issues and keep healthy flex_bg layout, disallow creating
ext4 with obviously large flex_bg count to filesystem blocks count.

Steps to reproduce:
(Case1)
1.
    # mke2fs -t ext4 -b 4096 -O ^resize_inode -G $((2**20)) DEV 2130483

2.
    # mount -t ext4 DEV MP
    mount: wrong fs type, bad option, bad superblock on /dev/sdb4,
           missing codepage or helper program, or other error
           In some cases useful info is found in syslog - try
           dmesg | tail  or so

3.
    # dumpe2fs DEV
    ...
    Block count:              2130483
    ...
    Flex block group size:    1048576
    ...
    Group 65: (Blocks 2129920-2130482) [INODE_UNINIT]
      Checksum 0x4cb3, unused inodes 8080
      Block bitmap at 67 (bg #0 + 67), Inode bitmap at 1048643 (bg #32 + 67)
      Inode table at 2129979-2130483 (+59)
                             ^^^^^^^  2130483 is out of FS!
      65535 free blocks, 8080 free inodes, 0 directories, 8080 unused inodes
      Free blocks:
      Free inodes: 525201-533280

(Case2)
1.
    # mke2fs -t ext4 -G 2147483648 DEV 3145728

2.
    # debugfs -R stats DEV
    ...
    Block count:              786432
    ...
    Flex block group size:    2147483648
    ...
     Group  0: block bitmap at 193, inode bitmap at 194, inode table at 195
    ...
     Group  1: block bitmap at 707, inode bitmap at 708, inode table at 709
    ...
     Group  2: block bitmap at 1221, inode bitmap at 1222, inode table at 1223
    ...

Signed-off-by: Akira Fujita <a-fujita@rs.jp.nec.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
2014-07-05 22:43:28 -04:00
Andreas Dilger f7a51a126c mke2fs: handle flex_bg collision with backup descriptors
If a large flex_bg factor is specified and the block allocator was
laying out block or inode bitmaps or inode tables, and collides with
previously allocated metadata (for example the backup superblock or
group descriptors) it would reset the allocator back to the beginning
of the flex_bg instead of continuing past the obstruction.

For example, with "-G 131072" the inode table will hit the backup
descriptors in groups 1, 3, 5, 7, 9 and start interleaving with the
block and inode bitmaps.  That results in poorly allocated bitmaps
and inode tables that are interleaved and not contiguous as was
intended for flex_bg:

 Group 0: (Blocks 0-32767)
   Primary superblock at 0, Group descriptors at 1-2048
   Block bitmap 2049 (+2049), Inode bitmap at 133121 (bg #4+2049)
   Inode table 264193-264200 (bg #8+2049)
   :
   :
 Group 3838: (Blocks 125763584-125796351) [INODE_UNINIT, BLOCK_UNINIT]
   Block bitmap 5887 (bg #0+5887), Inode bitmap 136959 (bg #4+5887)
   Inode table 294897-294904 (bg #8 + 32753)
 Group 3839: (Blocks 125796352-125829119) [INODE_UNINIT, BLOCK_UNINIT]
   Block bitmap 5888 (bg #0+5888), Inode bitmap 136960 (bg #4+5888)
   Inode table 5889-5896 (bg #0 + 5889)
 Group 3840: (Blocks 125829120-125861887) [INODE_UNINIT, BLOCK_UNINIT]
   Block bitmap 5897 (bg #0+5897), Inode bitmap 136961 (bg #4+5889)
   Inode table 5898-5905 (bg #0 + 5898)
   :
   :

Instead, skip the intervening blocks if there aren't too many of them.
That mostly keeps the flex_bg allocations from colliding, though still
not perfect because there is still some overlap with the backups.
This patch addresses the majority of the problem, allowing about 124k
groups to be layed out perfectly, instead of less than 4k groups with
the previous code.

Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-07-05 22:11:08 -04:00
Lukas Czerner 47fee2ef6a e2fsprogs: introduce ext2fs_close_free() helper
Currently there are many uses of ext2fs_close() which might be wrong.
First of all ext2fs_close() does not set the ext2_filsys pointer to NULL
so the caller is responsible for clearing it, however there are some
cases there we do not do it.

Second of all very small number of users of ext2fs_close() actually
check the return value. If there is a problem in ext2fs_close() it will
not even free the ext2_filsys structure, but majority of users expect it
to do so.

To fix both problems this commit introduces a new helper
ext2fs_close_free() which will not only check for the return value and
free the ext2_filsys structure if the call to ext2fs_close2() failed,
but it will also set the ext2_filsys pointer to NULL.

Replace every use of ext2fs_close() in e2fsprogs tools with
ext2fs_close_free() - there is no real reason to keep using
ext2fs_close().

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
2014-07-05 21:06:48 -04:00
Andreas Dilger 5023510dd7 blkid,ext2fs: avoid name clash with __u{8,16,32,64}
Try to avoid name clashes with definitions of __u8, __u16, __u32,
and __u64 in userspace, in case other headers also define these
types.  Define HAVE___{S,U}{8,16,32,64} preprocessor macros to
show that these types are already defined.

This would avoid the need to check for _BLKID_TYPES_H in ext2_types.h
and _EXT2_TYPES_H in blkid_types.h, but since older versions of these
headers did not use HAVE___U8 et.al. keep these checks around for now.

Report an error if there are no 64-bit types available.  The code
will not compile if these are not available.

Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-07-04 23:59:45 -04:00
Andreas Dilger 51050544b5 blkid: remove unnecessary header and comment
The LIST_HEAD macro is not directly used in getsize.c, so
<sys/queue.h> is not needed at all, and could cause confusion at
some later point if the Linux-style list macros are ever used.

Build was verified on MacOS which defined HAVE_SYS_DISK_H true.
I manually inspected the sources for recent *BSD headers to check
if this was needed there or not.  MacOS and FreeBSD <sys/disk.h>
do not use lists at all.  NetBSD and OpenBSD <sys/disk.h> and all
of the <sys/mount.h> headers include <sys/queue.h> internally.

I used http://fxr.watson.org/fxr/source/sys/mount.h?v={OSTYPE}
as a reference, checking both old and new *BSD versions.

Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-07-04 23:59:45 -04:00
Eric Sandeen 2f8d025196 e2fsprogs: remove checker infrastructure
Per http://www.gnu.org/software/checker/ the gcc "-checker" option
is long deprecated.  Nuke it from e2fsprogs.

Most people would never hit this, but people who love to turn knobs,
such as the reporter of kernel.org bz#74171, might run into it and be
sad.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-07-04 23:59:30 -04:00
Theodore Ts'o 9361395106 aclocal.m4: update to newer versions of autoconf macros
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-07-04 23:49:23 -04:00
Andreas Dilger 3120a50e1a misc: quiet minor build warnings, ignore generated files
Quiet a couple of build warnings in tst_libext2fs.c
Add missing unistd.h header for misc/util.c.

Ignore generated files for lib/ext2fs/tst_libext2fs and intl/ files.

Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-07-04 16:39:44 -04:00
Zheng Liu ab74c09071 debugfs: fix two warning messages when compiling with LLVM
This commit fixes two warning messages when compiling with LLVM.

Reported-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Lukas Czerner <lczerner@redhat.com>
2014-05-27 13:20:06 -04:00
Andreas Dilger bbccc6f3c6 misc: fix compile warnings on master branch
Fix compile warnings found on the master branch when using LLVM.

- Add missing format string when using the libintl _() macro
- include <limits.h> header to get PATH_MAX definition
- fix format vs. variable mismatches
- add header block for create_inode.c file
- remove use of bzero(), use ext2fs_get_memzero() instead

Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-05-27 13:14:18 -04:00
Darrick J. Wong ec3a42b192 Misc coverity fixes
Fix various small resource leaks and error code handling issues that
Coverity pointed out.

Fixes-Coverity-Bugs: 1215250, 1193379, 119194[2-4], 1049160
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-05-27 13:02:19 -04:00
Theodore Ts'o 6bb8845953 Merge branch 'maint' into next
Conflicts:
	debugfs/debugfs.c
2014-05-27 12:38:23 -04:00
Theodore Ts'o 1f15ba0034 lib/ext2fs: clean and build ext2_types.h as necessary
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-05-27 12:24:33 -04:00
Theodore Ts'o ceff962c67 tst_libext2fs: add new debug program which used for libext2fs unit tests
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-05-27 12:14:27 -04:00
Theodore Ts'o ba08cb996a Merge branch 'maint' into next
Conflicts:
	debugfs/Makefile.in
	debugfs/debug_cmds.ct
	debugfs/debugfs.c
	debugfs/debugfs.h
	e2fsck/Makefile.in
	misc/Makefile.in
	misc/mke2fs.c
2014-05-13 11:01:07 -04:00
Theodore Ts'o cbc1280dc7 quota: support storing the quota file handles in the quota context
This makes memory management easier because when the quota context is
released, all of the quota file handles get released automatically.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Aditya Kali <adityakali@google.com>
2014-05-13 10:13:12 -04:00
Theodore Ts'o d37178bb87 quota: integrate mkquota.h into quotaio.h
There are interfaces that are used by mke2fs.c and tune2fs.c which are
in quotaio.h, and some future changes will be much simpler if we can
combine the two header files together.  Also the guard #ifdef for
mkquota.h was incorrect, which caused problems when both header files
needed to be included.

Also remove quota.pc and installation rules for libquota, since this
library is never going to be something that we can export externally
anyway.  Eventually we'll want to clean up the interfaces and move the
external publishable interfaces to the libext2fs library, and then
rename what's left from libquota.a to libsupport.a for internal use
only.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Aditya Kali <adityakali@google.com>
2014-05-13 10:13:04 -04:00
Theodore Ts'o 4af2b156ff quota: fix memory leak in quota_compare_and_update()
The quota_handle wasn't getting closed in quota_compare_and_update().
Fix this, and also make sure that quota_file_close() doesn't
unnecessarily modify the quota inode if it's not necessary.  Otherwise
e2fsck will claim that the file system is modified when it didn't need
to be.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Aditya Kali <adityakali@google.com>
2014-05-13 10:12:57 -04:00
Theodore Ts'o 68ba77caa7 quota: fix e2fsck to notice missing quota entries
Previously if there was a missing quota entry --- i.e., if there were
files owned by group "eng", but there was no quota record for group
"eng", e2fsck would not notice the missing entry.  This means that the
usage informtion would not be properly repaired.  This is unfortunate.
Fix this by marking each quota record in quota_dict that has a
corresponding record on disk, and then check to see if there are any
records in quota_dict that have not been marked as having been seen.
In that case, we know we need to update the relevant quota inode.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Aditya Kali <adityakali@google.com>
2014-05-13 10:12:50 -04:00
Theodore Ts'o ac7ea7b081 quota: fix e2fsck so we update the quota file correctly
In scan_dquota_callback() we were copying dqb_off from the on-disk
dquot structure to the dqot structure that we have constructed in
memory.  This is a bad idea, because if we detect that the quota inode
is corrupted and needs to be replaced, when we later write out the
inode, the fact that we have a non-zero dqb_off value means that quota
routines will not bother updating the quota tree index, since
presumably the quota tree index already has an entry for this dqot.

The problem is that e2fsck, the only user of these functions, has
zapped the quota inode so it can be written from scratch.  So this
means the index for the quota records are not written out, so the
kernel can not find any of the records in the quota inodes.  Oops.

The fix is simple; there is no reason to copy the dqb_off field into
the quota_dict copy of struct dquot.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Aditya Kali <adityakali@google.com>
2014-05-13 10:12:42 -04:00
Theodore Ts'o a0811c1a0e quota: add debugging code to print the dquot structure
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Aditya Kali <adityakali@google.com>
2014-05-13 10:12:15 -04:00
Darrick J. Wong 1bad6f4624 misc: coverity fixes
Fix various small resource leaks and error code handling issues that
Coverity pointed out.

Fixes-Coverity-Bugs: 11919{39-45}, 1174118, 1049160, 1049144
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-05-11 18:40:20 -04:00
Darrick J. Wong a46300fe70 misc: create better-packaged static analysis reports
Fix some minor bugs relating to passing CFLAGS to cppcheck, and
package the cppcheck output into nicer looking reports.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-05-11 18:32:07 -04:00
Theodore Ts'o c9bc7484cc Merge branch 'maint' into next
Conflicts:
	e2fsck/pass1.c
2014-05-11 18:30:11 -04:00
Niu Yawei 4cf0b0fe44 libquota: fix dict_uint_cmp()
dict_uint_cmp() returns an usigned int value in int type, which
could mess the dict key comparison when the difference of two
keys is greater than INT_MAX.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-05-09 22:11:25 -04:00
David Michael 4e2d9f7f4e Add a make variable for the pkgconfig install dir
Adding the pkgconfigdir variable allows specifying an installation
location for pkg-config files independent of libdir.

Signed-off-by: David Michael <fedora.dm0@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-05-07 10:51:50 -04:00
Theodore Ts'o 2de2628b2d mke2fs: check for a partition table and warn if present
This supercedes the "whole disk" check, since it does a better job and
there are times when it is quite legitimate to want to use the whole
disk.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-05-07 10:51:50 -04:00
Theodore Ts'o d51072af0a libext2fs: fix alloc_allocate_group_table() if the flexbg_offset wraps
If the previous block group's inode table ends at the very end of file
system, wrap around to the beginning of the flex_bg.

This fixes a bug was tickled by:

mke2fs.conf:
	frontload = {
		features = extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,^resize_inode,sparse_super2
		hash_alg = half_md4
		num_backup_sb = 0
		packed_meta_blocks = 1
		inode_ratio = 4194304
		flex_bg_size = 262144
	}

mke2fs -T frontload /tmp/foo.img 2T
resize2fs -M /tmp/foo.img
resize2fs -M /tmp/foo.img

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-05-04 23:15:49 -04:00
Mark Fasheh bd1413d499 e2fsprogs/filefrag: print shared extent flag
Filefrag doesn't catch and print the shared extent flag. Add this for
users of filefrag on file systems with shared extents (such as btrfs).

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-05-04 23:15:48 -04:00
Theodore Ts'o 71715a5a9a Merge branch 'maint' into next 2014-04-20 08:31:24 -04:00
Andreas Dilger d8f401b135 fix miscellaneous build warnings
Fix various unused variable and use-uninitialized warnings.

Add generated files into .gitignore.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-04-14 12:22:42 -04:00
Eryu Guan 251edc3d3b libext2fs: fix build errors on bigendian hosts
Fix the following build errors on bigendian hosts.

- ctx is a pointer, use '->' not '.'
- add missing argument to ext2fs_dirent_swab_in2

make[2]: Entering directory `/root/e2fsprogs/lib/ext2fs'
        CC inline_data.c
inline_data.c: In function ‘ext2fs_inline_data_dir_iterate’:
inline_data.c:221:5: error: request for member ‘errcode’ in something not a structure or union
  ctx.errcode = ext2fs_dirent_swab_in2(fs, ctx->buf, ctx->buflen, 0);
     ^
inline_data.c:222:9: error: request for member ‘errcode’ in something not a structure or union
  if (ctx.errcode) {
         ^
inline_data.c: In function ‘ext2fs_inline_data_dir_expand’:
inline_data.c:364:2: error: too few arguments to function ‘ext2fs_dirent_swab_in2’
  retval = ext2fs_dirent_swab_in2(fs, buf, size);
  ^
In file included from inline_data.c:19:0:
ext2fs.h:1569:18: note: declared here
 extern errcode_t ext2fs_dirent_swab_in2(ext2_filsys fs, char *buf, size_t size,
                  ^
make[2]: *** [inline_data.o] Error 1

Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Zheng Liu <wenqing.lz@taobao.com>
2014-04-08 14:33:28 -04:00
Darrick J. Wong acbca26e25 libext2fs: fix parents when modifying extents
In ext2fs_extent_set_bmap() and ext2fs_punch_extent(), fix the parents
when altering either end of an extent so that the parent nodes reflect
the added mapping.

There's a slight complication to using fix_parents: if there are two
mappings to an lblk in the tree, the value of handle->path->curr can
point to either extent afterwards), which is documented in a comment.

Some additional color commentary from Darrick:

  In the _set_bmap() case, I noticed that the "remapping last block in
  extent" case would produce symptoms if we are trying to remap a
  block from "extent" to "next_extent", and the two extents are
  pointed to by different index nodes.  _extent_replace(...,
  next_extent) updates e_lblk in the leaf extent, but because there's
  no _extent_fix_parents() call, the index nodes never get updated.

  In the _punch_extent() case, we conclude that we need to split an
  extent into two pieces since we're punching out the middle.  If the
  extent is the last extent in the block, the second extent will be
  inserted into a new leaf node block.  Without _fix_parents(), the
  index node doesn't seem to get updated.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-15 11:36:36 -04:00
Darrick J. Wong a14f0a647b libext2fs: fix memory leak when drastically shrinking extent tree depth
In ext2fs_extent_free(), h(andle)->max_depth is used as a loop
conditional variable to free all the h->path[].buf pointers.  However,
ext2fs_extent_delete() sets max_depth = 0 if we've removed everything
from the extent tree, which causes a subsequent _free() to leak some
buf pointers.  max_depth can be re-incremented when splitting extent
nodes, but there's no guarantee that it'll reach the old value before
the free.

Therefore, remember the size of h->paths[] separately, and use that
when freeing the extent handle.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-14 09:56:36 -04:00
Darrick J. Wong 8a6cc1aebd misc: fix header complaints and resource leaks in e2fsprogs
Fix a few minor bugs that cppcheck complained about.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-14 09:53:19 -04:00
Darrick J. Wong b052dc1c8d libext2fs: fix 64bit overflow in ext2fs_block_alloc_stats_range
In ext2fs_block_alloc_stats_range(), the quantity "-inuse * n" is
calculated as a signed 32-bit quantity.  Unfortunately, gcc (4.6.3 on
Ubuntu 12.04) doesn't sign-extend this quantity to fill the blk64_t
parameter that ext2fs_free_blocks_count_add() wants, so the end result
is that the superblock gets a ridiculously huge free block count.

Changing the declaration of 'n' to blk64_t seems to fix this.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-14 09:35:02 -04:00
Darrick J. Wong 24997f1cd2 misc: cppcheck cleanups
Fix a number of things that cppcheck complains about.  Most of these
are minor resource leaks and forgotten declarations.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-14 09:34:11 -04:00
Darrick J. Wong f1605243a4 all: Introduce cppcheck static checking for make C=1
Introduce more static checking via cppcheck.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-14 09:32:50 -04:00
Darrick J. Wong 8f22fa055b libext2fs: repair side effects when iterating dirents in inline dirs
In ext2fs_inline_data_dir_iterate(), we must be very careful to undo
any modifications we make to the dir_context pointer passed in by the
caller, because it's entirely possible that the caller will still want
to do something with the ctx or something inside.

Specifically, ext2fs_dblist_dir_iterate() wants to be able to free
ctx->buf, and it reuses the ctx for multiple dblist entries.  That
means that assigning ctx->buf will cause weird crashes at the end of
dir_iterate().

Since we're being careful with ctx, we might as well handle adding the
INLINE_DATA flag to ctx->flags for ext2fs_process_dir_block, since the
dblist caller forgets to unset the flag before reusing the ctx.

This fixes some crashes and valgrind complaints in resize2fs, and is
necessary for the next patch, which fixes resize2fs not to corrupt
inline_data filesystems.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-14 09:30:20 -04:00
Darrick J. Wong 179e3bd708 libext2fs: during inlinedata expand, don't corrupt inode
When expanding an inline data inode, it's possible that the reduction
in the size of the EA structures causes the freeing of the EA block,
which changes the inode.  If this happens, the local version of the
inode that ext2fs_inline_data_expand was modifying will be out of sync
with what's on the disk.  This local copy gets written out to disk
after a block allocation, at which point it's possible that the inode
EA block and logical block zero point to the same physical block,
which is bad news.

Therefore, write the local copy to disk before removing the inline
data EA, and reread it afterwards.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-14 09:28:14 -04:00
Darrick J. Wong 97bd89cd39 libext2fs: fix iblocks correctly when expanding an inline_data file
i_blocks covers the number of blocks allocated to an inode for data,
extents, and ACL blocks.  Since it's possible for a file to have a
separate ACL block and inline data, we must be careful when expanding
an inline data file to adjust, not set, the value of i_blocks.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-14 09:23:23 -04:00
Darrick J. Wong cb803fc2ca libext2fs: don't fail when doing a strict rewrite of inline data
ext2fs_inline_data_set() tries to ensure that there is sufficient free
space in the inode to store the inline data.  Unfortunately, it gets
the check wrong -- ext2fs_xattr_inode_max_size() returns the amount of
unused bytes in the EA area, and _data_set() doesn't factor in the
size of the existing inline data.  Therefore, a strict rewrite of an
N-byte inlinedata with another N-byte inlinedata fails.

Fix the code to do the size check correctly.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-14 09:23:08 -04:00
Theodore Ts'o f9574ad0ba libext2fs: fix build failure on big endian systems
Fix a typo that we didn't notice because all the world's an x86.  :-)

Reported-by: jon ernst <jonernst07@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-13 13:40:37 -04:00
Darrick J. Wong 2ddcee1771 debugfs, libext2fs: minor fixups to xattr support
Add magic number checking to the extended attribute editing handle;
move inline data to the head of the attribute list when writing so
that inline data ends up in the inode area; and always zero the
attribute space before writing to ensure that we can delete the last
xattr.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-03-11 23:49:16 -04:00
Theodore Ts'o cecc79ac3f Update Makefile dependencies
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-06 11:26:41 -05:00
Zheng Liu 3125348838 libext2fs: add a unit test for inline data
In this unit test, we will test the interface of inline data and make
sure it is fine.

Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-04 08:46:16 -05:00
Zheng Liu be31a8de5a libext2fs: export inode cache creation function
Currently we have already exported inode cache flush and free functions
for users.  This commit exports inode cache creation function.  Later
we will use this function to initialize inode cache and do some unit
tests for inline data.

Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-04 08:46:15 -05:00
Zheng Liu 042e0719e2 e2fsck: check inline_data in pass1
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-04 08:46:15 -05:00
Zheng Liu 448cc0e43c libext2fs: add inline_data feature into EXT2_LIB_FEATURE_INCOMPAT_SUPP
Let e2fsprogs support inline_data feature.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-03-04 08:46:14 -05:00
Zheng Liu 54e880b870 libext2fs: handle inline data in read/write function
Currently ext2fs_file_read/write are used to copy data from/to a file.
But they manipulate data by blocksize.  For supporting inline data, we
handle it in two new fucntions called ext2fs_file_read/write_inline_data.

In read path the implementation is straightforward.  But in write path
things get more complicated because if the size of data is greater than
the maximum size of inline data we will expand this file.  So now we
will check this in ext2fs_inline_data_set.  If this inode doesn't have
enough space, it will return EXT2_ET_INLINE_DATA_NO_SPACE error.  Then
the caller will check this error and tries to expand the file.

The following commands in debugfs can handle inline_data feature after
applying this patch:
	- dump
	- cat
	- rdump
	- write

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-03-04 08:46:14 -05:00
Zheng Liu 97581d4481 debugfs: handle inline data feature in punch command
Now punch command only can remove all inline data because now
punch command is based on block unit and the size of inline data is
never beyond a block size.

Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-04 08:46:14 -05:00
Zheng Liu f5f6c020ca debugfs: handle inline_data feature in bmap command
No physical block mapping if an inode has inline data.

Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-04 08:46:14 -05:00
Zheng Liu 82e77d07ee debugfs: make mkdir command support inline data
This commit tries to make mkdir command in debugfs support inline data.

Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-04 08:46:14 -05:00
Zheng Liu 46bd6bdfc8 debugfs: make expand command support inline data
This commit defines a ext2fs_inline_data_expand() to expand an inode with
inline data.  In this commit this function only can expand a directory.
But later it will expand a file.

Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-04 08:46:14 -05:00
Zheng Liu 133e94627b debugfs: make stat command support inline data
If there is an inode with inline data, we just print the size of inline
data in stat command.

Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-04 08:46:13 -05:00
Zheng Liu 716a3c3536 libext2fs: handle inline_data in block iterator function
After applied this commit (a7f4c635), we have banned to traverse blocks
for an inode which has inline data because no block belongs to it.  But
before calling this function, we need to check inline data flag.  This
commit add a sanity check ext2fs_inode_has_valid_blocks2() to fix them
except that ext2fs_expand_dir because it will be fixed by another patch.

Meanwhile in this commit it fixes a bug that when we kill a file we
could leak an inode.

Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-04 08:46:13 -05:00
Zheng Liu 416c1de94d libext2fs: handle inline data in dir iterator function
Inline_data is handled in dir iterator because a lot of commands use
this function to traverse directory entries in debugfs.  We need to
handle inline_data individually because inline_data is saved in two
places.  One is in i_block, and another is in ibody extended attribute.

After applied this commit, the following commands in debugfs can
support the inline_data feature:
	- cd
	- chroot
	- link*
	- ls
	- ncheck
	- pwd
	- unlink

* TODO: Inline_data doesn't expand to ibody extended attribute because
  link command doesn't handle DIR_NO_SPACE error until now.  But if we
  have already expanded inline data to ibody ea area, link command can
  occupy this space.

Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-04 08:46:08 -05:00
Zheng Liu 11f9374660 libext2fs: define new dirent_swab interfaces for inline data
Later we will use ext2fs_dirent_swab_in/out to handle big-endian problem
for inline data.  Now interfaces assume that it handles a block, but it
is not true after adding inline data.  So this commit defines a new
interface for inline data.

Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-03-03 00:24:33 -05:00
Darrick J. Wong 456b93c08f libext2fs: free key/value pairs before reading
Before loading extended attributes, free any key/value pairs that
might already be associated with the file.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-02-23 23:08:43 -05:00
Darrick J. Wong 9224d817b9 libext2fs: extend xattr api to query number of attrs
Add another API to query the number of extended attributes.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-02-23 23:08:39 -05:00
Darrick J. Wong 685156954f libext2fs: various tweaks to the xattr editor APIs
A few tweaks to the extended attribute editing APIs:

 * Use size_t, not unsigned int, in the new extended attribute editing
   API.

 * Don't expose the _expand() call since there should be no external
   users.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-02-23 23:08:34 -05:00
Darrick J. Wong 2077c208a6 libext2fs: support modifying arbitrary extended attributes
Add functions to allow clients to get, set, and remove extended
attributes from any file.  It also supports modifying EAs living in
i_file_acl.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-02-23 23:08:23 -05:00
Theodore Ts'o 3ba59bf649 Merge branch 'maint' into next 2014-02-23 23:07:38 -05:00
Theodore Ts'o fedfb27fe5 Add coverage testing using gcov
To check the coverage of e2fsprogs's regression test, do the
following:

	configure --enable-gcov
	make -j8 ; make -j8 check ; make coverage.txt

The coverage information will be the coverage.txt and *.gcov files in
the build directories.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-02-23 00:17:09 -05:00
Theodore Ts'o dc084ab7b1 Merge branch 'maint' into next 2014-02-20 21:39:29 -05:00
Lukas Czerner 585bca68f8 Set pointer to NULL after ext2fs_free
ext2fs_free() does not set the ext2_filsys pointer to null so the
caller is responsible to setting it himself if it is needed.

This patch fixes some places where caller did not set ext2_filsys
pointer to NULL after ext2fs_free() which might result in use after
free.  Fix it.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-02-20 20:54:34 -05:00
jon ernst 4f3ded2c15 libext2fs: delete unused "handle" variable
After commit 62f17f3603, variable
"handle" has no use.  So delete it.

Signed-off-by: Jon Enrst <jonernst07@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-02-20 20:49:43 -05:00
Andreas Dilger a4613d1345 ext2fs: declare struct_io_manager at end of file
Declare struct_io_manager at the end of unix_io.c, undo_io.c, and
test_io.c files so that there isn't a need to forward declare every
member of this structure.  That avoids a lot of redundant code
at the start of every one of these files.

Move the test_flush() function above test_abort() to avoid the need
for a forward declaration.

Fix a few instances of space before tab in these files.

Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-02-18 18:31:43 -05:00
Andreas Dilger 3b97799c74 tests: skip unsupported tests on MacOS systems
The "mkswap" program is not available on MacOS, so just use the
existing swap0.img.bz2 and swap1.img.bz2 files directly.

Because MacOS HFS+ doesn't support sparse files (welcome to the 80's)
the m_bigjournal test takes forever to zero out the whole 42GB test
filesystem.  Skip this test for Darwin kernels for now.

Unfortunately, neither "df -T" nor "stat -f -c %T" is available on
MacOS to directly determine the filesystem type, and I'm too lazy
to parse the output of "mount" and match it to the path of the test
directory in shell, so it just checks the kernel type and assumes
the filesystem type is HFS and skips the test.

Since this test runs on Linux the majority of the time, the loss of
test coverage is minimal.  If MacOS should ever get a real filesystem,
this can be revisited.

Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-02-18 18:30:00 -05:00
Andreas Dilger 4d46e6c737 build: fix LLVM compiler warnings
Fix a number of non-literal string format warnings from LLVM due
to the use of _() that were not fixed in commit 45ff69ffeb.

Fix mismatched int vs. __u64 format warnings in blkmap64_rb.c.
There were also some comparisons of __u64 start or count <= 0.
Change them to be comparisons == 0, or start + count overflow.

Fix operator precedence warning for (value & (value - 1) != 0)
introduced in 11d1116a7c.  It seems "&" is lower precedence
than "!=", so the above didn't fail for power-of-two values,
but only odd values.  Fortunately, either s_desc_size nor
s_inode_size is valid if odd.

Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-02-18 12:17:53 -05:00
Theodore Ts'o f3fa5689d7 libe2p: allow libe2p.h to be used in C++ programs
In C++, "private" is a reserved keyword, so don't use it in the header
file as a function parameter name.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-02-07 17:25:28 -05:00
Theodore Ts'o d5fc19fc96 Merge branch 'maint' into next 2014-02-06 15:39:28 -05:00
Darrick J. Wong 156a151f31 libext2fs: try to roll back when splitting an extent fails
If a client asks us to remap a block in the middle of an extent, we
potentially have to allocate a fair number of blocks to handle extent
tree splits.  A failure in either of the ext2fs_extent_insert calls
leaves us with an extent tree that no longer maps the logical block in
question and everything that came after it!  Therefore, try to roll
back the extent tree changes before returning an error code.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-02-06 15:34:00 -05:00
Darrick J. Wong 6225a154d2 libext2fs: don't hang on to unmapped block if extent tree update fails
If we're doing a BMAP_ALLOC allocation and the extent tree update
fails, there's no point in hanging on to the newly allocated block.
So, free it to make fsck happy.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-02-06 15:32:20 -05:00
Darrick J. Wong 3c890ee96e libext2fs: during punch, fix parent extents after modifying extent
When modifying/removing an extent during punch, don't forget to update
the extent's parents.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-02-06 15:31:03 -05:00
Darrick J. Wong 80dec4cb31 libext2fs: iterate past lower extents during punch
When we're iterating extents during a punch operation, the loop exits
if the punch region is entirely to the right of the extent we're
looking at.  This can happen if the punch region starts in the middle
of a hole and covers mapped extents.  When this happens, we want to
skip to the next extent, because it might be punchable.

Also, if we've totally passed the punch range, stop.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-02-06 15:29:33 -05:00
Theodore Ts'o f0996c12d5 Merge branch 'maint' into next
Conflicts:
	lib/ext2fs/alloc.c
	misc/mke2fs.c
2014-01-30 19:05:16 -05:00
Theodore Ts'o c714d02482 Fix up the Makefiles dependencies in lib/ext2fs and lib/quota
Also use angle brackets for the #include of dirpaths.h to avoid the
need to manually massage the Makefile.in for the util directory.  This
is needed because we have to create a fake dirpaths.h file in the util
directory.  The fake dirpaths.h file is rquired to break the circular
dependency caused by util/subst creating dirpaths.h, while
util/subst.c is including config.h, which includes dirpaths.h.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-01-30 18:48:23 -05:00
Theodore Ts'o 9b36ce0e19 libss: fix potential buffer overrun in list_rqs
Addresses-Coverity-Bug: #709516

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-01-30 18:25:15 -05:00
Theodore Ts'o 0dedd011ed quota: fix uninitiaized memory reference in mke2fs with quota enabled
Initialize the on-disk structure before we fill it in, to avoid the
following valgrind warning:

   Conditional jump or move depends on uninitialised value(s)
      at 0x4323A8: qtree_entry_unused (quotaio_tree.c:40)
      by 0x431218: v2r1_mem2diskdqblk (quotaio_v2.c:85)
      by 0x432409: qtree_write_dquot (quotaio_tree.c:336)
      by 0x431136: v2_commit_dquot (quotaio_v2.c:264)
      by 0x42FB63: quota_write_inode (mkquota.c:126)
      by 0x408BE6: create_quota_inodes (mke2fs.c:2466)
      by 0x409A2D: main (mke2fs.c:2850)

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-01-30 18:25:15 -05:00
Theodore Ts'o 65c6c3e06f Add support for new compat feature "sparse_super2"
In practice, it is **extremely** rare for users to try to use more
than the first backup superblock located at the beginning of block
group #1.  (i.e., at block number 32768 for file systems with a 4k
block size).  This new compat feature restricts the backup superblock
to block group #1 and the last block group in the file system.

Aside from reducing the overhead of the file system by a small number
of blocks, by eliminating the rest of the backup superblocks, it
allows us to have a much more flexible metadata layout.  For example,
we can force all of the allocation bitmaps and inode table blocks to
the beginning of the disk, which allows most of the disk to be
exclusively used for contiguous data blocks.

This simplifies taking advantage of certain HDD specific features,
such as Shingled Magnetic Recording (aka Shingled Drives), and the
TCG's OPAL Storage Specification where having a simple mapping between
LBA block ranges and the data blocks used by the file system can make
life much simpler.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-01-30 13:58:18 -05:00
Theodore Ts'o 0fe6e3ae7b libext2fs: add new function ext2fs_add_journal_inode2()
This new function has an parameter which allows the caller to specify
the location of the journal.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-01-30 13:58:17 -05:00
Theodore Ts'o 3550f722e9 libext2fs: factor out get_midpoint_journal_block() in mkjournal.c
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-01-30 13:58:17 -05:00
Theodore Ts'o 424de6e8e3 libext2fs: optimize ext2fs_new_block2()
If there are hundreds of thousands of blocks which are in use before
the first free block, it is much, MUCH faster to use
ext2fs_find_first_zero_block_bitmap2() instead of searching the
allocation bitmap bit by bit.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-01-20 22:16:16 -05:00
Theodore Ts'o fccdbac394 libext2fs: optimize ext2fs_allocate_group_table()
By using ext2fs_mark_block_bitmap_range2 and/or
ext2fs_block_alloc_stats_range(), we can significantly speed up the
time needed by mke2fs to allocate the inode table.

For example, the CPU time needed to run the command "mke2fs -t ext4
/tmp/foo.img 32T" (where tmpfs was mounted on /tmp) was decreased from
21.7 CPU seconds down to under 1.7 seconds.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-01-20 22:16:16 -05:00
Theodore Ts'o 5c41451e16 libext2fs: add ext2fs_block_alloc_stats_range()
This function is more efficient than using ext2fs_block_alloc_stats2()
for each block in a range.  The efficiencies come from being able to
set a block range in the block bitmap at once, and from being update
the block group descriptors once per block group.  Especially now that
we are checksuming the block group descriptors, and we are using red
black trees for the allocation bitmaps, these changes can make a huge
difference in the CPU time used by mke2fs when creating very large
file systems.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-01-20 22:13:14 -05:00
Theodore Ts'o dd9aa132bf libext2fs: further clean up and rename check_block_uninit
Commit 8e44eb64bb (libext2fs: mark group data blocks when loading
block bitmap) simplified check_block_uninit since we are now
initializing the bitmap when it is loaded from disk.  It left some
variables which were being set but never used, however.  In addition,
since we only need check_block_uninit() to clear the block bitmap's
uninit flag, rename it to clear_block_uninit(), and only call it once
we have found a free block in ext2fs_new_blocks2().

This cleans up the code some and optimizes things if we need to search
multiple block groups trying to find a free block.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: Darrick J. Wong <darrick.wong@oracle.com>
2014-01-20 01:00:05 -05:00
Theodore Ts'o 3203cd9332 libext2fs: optimize find_first_{zero,set}() for red-black tree based bitmaps
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-01-20 01:00:05 -05:00
Theodore Ts'o 14717832dd libext2fs: optimize find_first_set() for bitarray-based bitmaps
Basically just a trivial adaption of the find_first_zero() function
for bitarray-based bitmaps.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-01-20 00:59:49 -05:00
Theodore Ts'o d954fa407b libext2fs: build tst_bitmaps with rep invariants checking enabled
When building tst_bitmaps, enable #define DEBUG_RB, so we are
always testing the sanity of the in-memory representation of the
bitmap when using red-black trees as part of a "make check" run.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-01-19 19:19:26 -05:00
Theodore Ts'o 3602197449 libext2fs: clean up generic handling of ext2fs_find_first_{set,zero}_*()
Move the error checking into the the generic bitmap code, and add
support for bitmaps with cluster_bits set.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-01-19 19:19:17 -05:00
Theodore Ts'o a2acec75c0 libext2fs: fix off-by-one bug in ext2fs_extent_insert()
When inserting the first extent into an empty inode, the
ext2fs_extent_insert() leaves path->left set to 1 instead of 0.  Since
path->curr is pointing at the last (only) extent in the file,
path->left should be 0.

This is mostly harmless, and gets corrected fairly quickly if the
calling applicaton jumps to a different part of the extent tree ---
for example, by calling ext2fs_extent_goto(), or calling
ext2fs_extent_get with the flags argument set to EXT2_EXTENT_ROOT.
Which is why we hadn't noticed this problem until now.

However, if you insert four extents using ext2fs_extent_insert, the
fourth insert will end up copying too many bytes in the i_block[]
array, since path->left is one larger than it should be.  This results
in the inode fields i_generation, i_file_acl, and i_size_high getting
zeroed out.

This problem can be replicated as follows:

% cp /dev/null /tmp/foo.img
% mke2fs -F -t ext4 /tmp/foo.img 100
% debugfs -w /tmp/foo.img
debugfs: write /dev/null foo
debugfs: set_inode_field foo i_size_hi 1
debugfs: stat foo
 <----- note that the inode's size is 4294967296
debugfs: extent_open foo
debugfs (extent ino 12): insert --after 0 1 100
debugfs (extent ino 12): insert --after 1 1 101
debugfs (extent ino 12): insert --after 2 1 102
debugfs (extent ino 12): insert --after 3 1 103
debugfs (extent ino 12): extent_close
debugfs: stat foo
 <----- note that the inode's size is now 0
debugfs: quit

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-01-19 19:18:48 -05:00
Theodore Ts'o dff0b6a333 libext2fs: add ext2fs_find_first_set_{block,inode}_bitmap2()
Add functions which try to find the first set block or inode in a
bitmap.  This is useful when trying to allocate a range of blocks
efficiently.

Like the find_first_zero family of functions, provide a generic O(N)
search function which will be used if there is no optimized version
provided by the red-black tree or bitarray functions.

Also, expand the test cases for ext2fs_find_first_zero_*() functions.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-01-13 14:02:06 -05:00
Darrick J. Wong d7c64cdc96 libext2fs: no need to clear BLOCK_UNINIT during ext2fs_reserve_super_and_bgd
Since the beginning of the uninit_bg feature, the kernel[1] and
e2fsck[2] have always been careful to detect the presence of the
BLOCK_UNINIT flag, and compute a block bitmap with any group metadata
blocks marked in that bitmap.  With that in mind, I think it's safe to
say that this is a design feature of uninit_bg.

Now that we've trained libext2fs to have this same behavior whenever
it's loading a block bitmap, we no longer need to unset BLOCK_UNINIT
for a group that contains only its own group metadata -- kernel,
e2fsck, and e2fsprogs will handle this correctly.

[1] kernel git 717d50e4971b81b96c0199c91cdf0039a8cb181a
    "Ext4: Uninitialized Block Groups"
[2] e2fsprogs git f5fa20078b
    "Add support for EXT2_FEATURE_COMPAT_LAZY_BG"

Reported-by: Akira Fujita <a-fujita@rs.jp.nec.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-01-11 14:15:51 -05:00
Darrick J. Wong 8e44eb64bb libext2fs: mark group data blocks when loading block bitmap
The kernel[1] and e2fsck[2] both react to a BLOCK_UNINIT group by
calculating the block bitmap that's needed to show all the group
blocks for that group (if any) and using that.  However, when reading
bitmaps from disk, libext2fs simply imports a block of zeroes into the
bitmap, without bothering to check for group blocks.  This erroneous
behavior results in the filesystem having a block bitmap that does not
accurately reflect disk contents, and worse yet makes it seem as
though superblocks, group descriptors, bitmaps, and inode tables are
"free" space on disk.

So, fix the block bitmap loading routines to calculate the correct
block bitmap for all groups and load it into the main fs block bitmap.

This also fixes bogus debugfs output such as:

Group 1: (Blocks 8193-16384) [INODE_UNINIT, BLOCK_UNINIT]
  Checksum 0x1310, unused inodes 512
  Backup superblock at 8193, Group descriptors at 8194-8217
  Reserved GDT blocks at 8218-8473
  Block bitmap at 283 (bg #0 + 282), Inode bitmap at 299 (bg #0 + 298)
  Inode table at 442-569 (bg #0 + 441)
  7911 free blocks, 512 free inodes, 0 directories, 512 unused inodes
  Free blocks: 8193-16384
  Free inodes: 513-1024

Notice how the "free blocks" range includes the backup sb & GDT area
and doesn't match the free block count.

Worse yet, debugfs' testb command will report those group descriptor
blocks as not being in use unless the user also instructs debugfs to
find a free block first.  That is a rather surprising result:

debugfs:  testb 8194
Block 8194 not in use
debugfs:  ffb 1 16380
Free blocks found: 16380
debugfs:  testb 8194
Block 8194 marked in use

Also, remove the part of check_block_uninit() that "fixes" the bitmap
since we're doing that at bitmap load time now.

[1] kernel git 717d50e4971b81b96c0199c91cdf0039a8cb181a
    "Ext4: Uninitialized Block Groups"
[2] e2fsprogs git f5fa20078b
    "Add support for EXT2_FEATURE_COMPAT_LAZY_BG"

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-01-11 14:04:48 -05:00
Darrick J. Wong d36b957b34 libext2fs: don't always read backup group descriptors on a 1k-block meta_bg fs
On a filesystem with 1K blocks and meta_bg enabled, opening a
filesystem with automatic superblock detection tries to compensate for
the fact that the superblock lives in block 1.  However, the method by
which this is done is later misinterpreted to mean "read the backup
group descriptors", which is not what we want in this case.

Therefore, in ext2fs_open3() separate the 'group zero' adjustment into
its own variable so that we don't get fed backup group descriptors
when we try to load meta_bg group descriptors.

Furthermore, enhance ext2fs_descriptor_block_loc2() to perform its own
group zero correction.  The other caller of this function neglects to
do any group-zero correction of their own, so this fixes them too.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-01-11 13:58:15 -05:00
Darrick J. Wong 6ff02832a0 libext2fs: detect correct superblock adjustments when loading backup groups
If ext2fs_descriptor_block_loc2() is called with a meta_bg filesystem
and group_block is not the normal value, the function will return the
location of the backup group descriptor block in the next block group.
Unfortunately, it fails to account for the possibility that the backup
group contains a backup superblock but the regular superblock does
not.  This is the case with block groups 48-49 on a meta_bg fs with 1k
blocks; in this case, libext2fs will fail to open the filesystem.

Therefore, teach the function to adjust for superblocks in the backup
group, if necessary.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2014-01-11 13:55:20 -05:00
Theodore Ts'o 72958b6670 Merge branch 'maint' into next 2014-01-10 23:26:00 -05:00
Theodore Ts'o 0d719bad86 libblkid: add error checking for rename() while saving the blkid cache
Addresses-Coverity-Id: #1049141

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2014-01-10 22:09:45 -05:00