Commit Graph

1336 Commits (e35ff9b92ac8e1038f9ab972e6f3a73e0091d3ee)

Author SHA1 Message Date
Richard W.M. Jones 9d9a53e651 libext2fs: add flag to ext2fs_flush() and ext2fs_close() to avoid fsync
This adds new APIs: ext2fs_flush2 and ext2fs_close2 which take an
extra 'int flags' parameter.

This allows us to pass in an EXT2_FLAG_FLUSH_NO_SYNC flag which avoids
fsync'ing the filesystem when closing it.  For the case we have in
mind where we are just constructing a throwaway ext2 filesystem in a
file in order to boot a VM, this saves over 5 seconds during the boot
process and avoids many unnecessary disk writes.

Existing code using ext2fs_flush and ext2fs_close remains unaffected
by this change.

Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-09-24 12:53:16 -04:00
Theodore Ts'o 8f9d708f6a libquota: only build a static library
Since the libquota library has namespace contamination issues, don't
build a shared library and link against it statically.  Don't include
it as part of the Debian packages.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-09-19 13:39:38 -04:00
Theodore Ts'o d1154eb460 Shorten compile commands run by the build system
The DEFS line in MCONFIG had gotten so long that it exceeded 4k, and
this was starting to cause some tools heartburn.  It also made "make
V=1" almost useless, since trying to following the individual commands
run by make was lost in the noise of all of the defines.

So fix this by putting the configure-generated defines in lib/config.h
and the directory pathnames to lib/dirpaths.h.

In addition, clean up some vestigal defines in configure.in and in the
Makefiles to further shorten the cc command lines.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-09-18 17:34:37 -04:00
Eric Sandeen 5ea3c82592 libext2: Fix leaks in write_bitmaps on error returns
block_buf and/or inode_buf may not be properly freed on an error
return.

Create a new errout: target to free them as needed in error conditions.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-09-16 19:54:01 -04:00
Eric Sandeen 624e8ebe30 e2fsprogs: Fix some error cleanup path bugs
In inode_open(), if the allocation of &io fails, we go to cleanup
and dereference io to test io->name, which is a bug.

Similarly in undo_open()  if allocation of &data fails, we
go to cleanup and dereference data to test data->real.

In the test_open() case we explicitly set retval to the only
possible error return from ext2fs_get_mem(), so remove that
for tidiness.

The other changes just make make earlier returns go through
the error goto for consistency.

In many cases we returned directly from the first error, but
"goto cleanup" etc for every subsequent error.  In some
cases this leads to "impossible" tests such as:

	if (ptr)
		ext2fs_free_mem(&ptr)

on paths where ptr cannot be null because we would have
returned directly earlier, and Coverity flags this.

This isn't really indicative of an error in most cases, but
I think it can be clearer to always exit through the error goto
if it's used later in the function.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-09-16 18:43:05 -04:00
Eric Sandeen 06e41ddea9 libe2p: reach unreachable code
The EOPNOTSUPP case is unreachable, being outside a set of:
 #if
	...
	return;
 #else
	...
	return;
 #endif

Fix this up so that if neither HAVE_CHFLAGS nor
HAVE_EXT2_IOCTLS applies, we set EOPNOTSUPP.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-09-16 18:43:05 -04:00
Eric Sandeen 9e30fb23ef e2fsprogs: annotate intentional fallthroughs in case statements
Using the /* fallthrough */ comment lets Coverity (and humans)
know that we really do want to fall through in these case statements.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-09-16 18:43:05 -04:00
Eric Sandeen 665563b825 libext2fs: Potential null ptr deref in undo_err_handler_init
In the !undo_io_backing_manager case, undo_err_handler_init
will be passed a null data->real, which will be dereferenced.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-09-16 18:43:04 -04:00
Eric Sandeen d0f196d867 libext2: move buf variable completely under ifdef
If !WORDS_BIGENDIAN, it is pointless to test whether buf
is NULL, because it is initialized to NULL and never changed.
This makes Coverity complain, so we can just move all handling
of "buf" under the #ifdef.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-09-16 18:43:04 -04:00
Eric Sandeen bc28abc537 libext2: Fix EXT2_LIB_SOFTSUPP masking
EXT2_LIB_SOFTSUPP_INCOMPAT_* are supposed to be bitmasks
of features which can be opened even though they are
under development.  The intent is that these are masked
out of the features list, so that they will be ignored
on open.

However, the code does a logical not vs. a bitwise not:

	features &= !EXT2_LIB_SOFTSUPP_INCOMPAT;

which will not have the desired effect...

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-09-16 18:43:04 -04:00
Theodore Ts'o ae96c678e1 libext2s: fix swapfs.c so it builds on big endian systems
Also cleaned up ext2_fs.h, and improved the byte swapping code so the
extra fields in the large inode are properly byte swapped.

Addresses-Debian-Bug: #641838

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-09-16 17:51:03 -04:00
Theodore Ts'o cd0b97ed93 libquota: indicate in the ELF library image that it requires libext2fs
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-09-16 11:12:59 -04:00
Theodore Ts'o 16c581d0e8 debugfs: add 64-bit support to the set_field commands
The set_fields commands (set_super_value, set_inode_field,
set_block_group) now handle fields which store in split fields on
ext4's on-disk format.  For example, the superblock fields
s_blocks_count and s_blocks_count_hi.

The user can either set the low or high part of the field via
"blocks_count_lo" or "blocks_count_hi", or both parts can be set via
"blocks_count".

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-09-16 10:32:02 -04:00
Theodore Ts'o 89efc88e65 libext2fs: add metadata checksum and snapshot feature flags
Reserve EXT4_FEATURE_RO_COMPAT_METADATA_CSUM and
EXT2_FEATURE_COMPAT_EXCLUDE_BITMAP.  Also reserve fields in the
superblock and the inode for the checksums.  In the block group
descriptor, reserve the exclude bitmap field for the snapshot feature,
and checksums for the inode and block allocation bitmaps.

With this commit, the metadata checksum and exclude bitmap features
should have reserved all of the fields they need in ext4's on-disk
format.

This commit also fixes an a missing byte swap for s_overhead_blocks.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: Darrick J. Wong <djwong@us.ibm.com>
Cc: Amir Goldstein <amir73il@gmail.com>
2011-09-16 10:24:09 -04:00
Eric Sandeen f314bb0821 libext2fs: remove redundant last-group check in ext2fs_check_desc()
ext2fs_group_last_block2() already properly calculates
the last block in the last group, so there is no need
to special-case this after the call.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-09-16 09:32:18 -04:00
Eric Sandeen 98f4547198 e2fsprogs: add ext2fs_group_blocks_count helper
Code to count the number of blocks in the last partial
group is cut and pasted around the e2fsprogs codebase
a few times.

Making this a helper function should improve matters.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-09-16 09:28:43 -04:00
Theodore Ts'o 65b7a463da libext2fs: fix size check in tst_inode_size
Also add run tst_inode_size automaically from "make check"

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-09-16 09:27:06 -04:00
Theodore Ts'o 75405ffde6 Merge branch 'maint' into next 2011-09-16 00:00:04 -04:00
Lukas Czerner d2bfdc7ff1 e2fsprogs: Use punch hole as "discard" on regular files
If e2fsprogs tools (mke2fs, e2fsck) is run on regular file instead of
on block device, we can use punch hole instead of regular discard
command which would not work on regular file anyway. This gives us
several advantages. First of all when e2fsck is run with '-E discard'
parameter it will punch out all ununsed space from the image, hence
trimming down the file system image. And secondly, when creating an
file system on regular file (with '-E discard' which is default), we
can use punch hole to clear the file content, hence we can skip inode
table initialization, because reads from sparse area returns zeros. This
will result in faster file system creation (without the need to specify
lazy_itable_init) and smaller images.

This commit also fixes some tests that would fail due to mke2fs showing
discard progress, hence the output would differ.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-09-15 23:49:20 -04:00
Lukas Czerner c859cb1de0 e2fsprogs: create open() and stat() helpers
In many places we are using #ifdef HAVE_OPEN64 to determine if we can
use open64() but that's ugly. This commit creates two new helpers
ext2fs_open_file() for open() and ext2fs_stat() for stat(). Also we need
new typedef ext2fs_struct_stat for struct stat.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-09-15 23:46:57 -04:00
Darrick J. Wong db8bbf27c9 libext2fs: Add crc32c implementation for metadata checksumming
Add a slicing-by-8 CRC32c implementation for metadata checksumming.
Adapted from Bob Pearson's kernel patch.

Also added a self-test mechanism so we can verify that the crc32c
implementation is working correctly.

Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-09-15 23:46:51 -04:00
Amir Goldstein 15749d7da9 libext2fs: fix the range validation in bitmap_range2 funcs
The condition ((start+num) & ~0xffffffffULL) in bitmap_range2
and generic_bmap_range funcs in get_bitmap64.c was wrong and
inconsistent with the condition (start+num-1 > bmap->real_end)
in generic_bitmap_range funcs in get_bitmap.c.

I got the following error from tune2fs on a 16TB fs:
Illegal block number passed to ext2fs_unmark_block_bitmap #4294967295
for block bitmap for 16TB.img
tune2fs: Invalid argument while reading bitmaps

Fix to condition to ((start+num-1) & ~0xffffffffULL), because
the bit (start+num) is not going to be changed by the funcs.

Signed-off-by: Amir Goldstein <amir73il@users.sf.net>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-09-15 22:23:24 -04:00
Theodore Ts'o 3fbfad558e libext2fs: fix binary and source compatibility with the dump program
The dump program relies on fs->frag_size and the
EXT2_FRAGS_PER_BLOCK() macro.  Kind of silly for it to do so, but it's
part of the kludgy way the dump program (which was originally written
for the BSD FFS was ported over to support ext2/3.)  Given how it
makes assumptions about the ext2/3/4 file system being similar to the
BSD FFS, it's a bit of a miracle it works for ext4 --- or at least
appears to work...

Addresses-Debian-Bug: #636418

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-09-15 15:44:56 -04:00
Theodore Ts'o 63165136ea libe2p: fix bug so that MNTOPT_ options can be successfully parsed
Thanks to Israel G. Lugo for pointing this out.

Addresses-Debian-Bug: #641667

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-09-15 14:25:45 -04:00
Theodore Ts'o 449c87a49c libquota: fix "make install" so it works in with a VPATH build directory
Also fix up the name of the header file which we are installing so it
is correct.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-09-14 13:47:35 -04:00
Theodore Ts'o 897fbaac7d libext2fs: add new test: tst_inode_size
This test makes sure the size of the ext2_inode is what we expect

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-09-14 13:47:35 -04:00
Yongqiang Yang 9f6ba888f0 resize2fs: add support for new in-kernel online resize ioctl
This is needed to support online resizing for > 32-bit file systems

Signed-off-by: Yongqiang Yang <xiaoqiangnk@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-09-14 13:26:57 -04:00
Aditya Kali 624e4a6466 e2fsck: add support for checking the built-in quota files
This patch adds support for doing quota accounting during full
e2fsck scan if the 'quota' feature was set on the superblock.
If user-visible quota inodes are in use, they will be hidden
and converted to the reserved quota inodes.

Signed-off-by: Aditya Kali <adityakali@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-08-31 16:31:13 -04:00
Aditya Kali f239fefc14 e2fsprogs: add quota library to e2fsprogs
This patch adds the quota library (ported form Jan Kara's quota-tools) in
e2fsprogs in order to make quotas as a first class supported feature in Ext4.
This patch also provides interface in lib/quota/mkquota.h that will be used by
mke2fs, tune2fs, e2fsck, etc. to initialize and update quota files.
This first version of the quota library does not support reading existing quota
files. This support will be added in the near future.
Thanks to Jan Kara for his work on quota-tools. Most of the files in this patch
are taken as-is from quota tools and were simply modified to work with
libext2fs in e2fsprogs.

Signed-off-by: Aditya Kali <adityakali@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-08-31 15:21:40 -04:00
Theodore Ts'o a4aff9ca5b libext2fs: fix binary search for the icount and badblocks stores
Remove the interpolation because there is a bug in icount which can
cause a core dump if calculated range gets turned into a NaN and then
do an out-of-bounds array access.  We could fix this with some more
tests, but the complexity is such that nuking all of the interpolation
code will be faster than fixing the interpolation.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-08-31 14:27:21 -04:00
Eric Sandeen 9c79612f21 libext2fs: copy cluster_bits in ext2fs_copy_generic_bmap
The f_lotsbad regression test was failing on some systems
with:

 Restarting e2fsck from the beginning...
 Pass 1: Checking inodes, blocks, and sizes
+Illegal block number passed to ext2fs_test_block_bitmap #0 for in-use block map
 Pass 2: Checking directory structure
 Entry 'termcap' in / (2) has deleted/unused inode 12.  Clear? yes

Running with valgrind (./test_script --valgrind f_lotsbad) we
see:

+==31409== Conditional jump or move depends on uninitialised value(s)
+==31409==    at 0x42927A: ext2fs_test_generic_bmap (gen_bitmap64.c:378)

among others.

Looking at gen_bitmap64.c:
376:        arg >>= bitmap->cluster_bits;
377:
378:        if ((arg < bitmap->start) || (arg > bitmap->end)) {

A little more debugging showed that it was actually
bitmap->cluster_bits which was uninitialized, because it never
gets copied over in ext2fs_copy_generic_bmap()

Patch below resolves the issue.

Reported-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-08-10 21:49:38 -04:00
Theodore Ts'o d8fae3cf42 libext2fs: fix block iterator when the callback function modifies an extent
If the callback interator modifies a block in the middle of an extent
during a call to the block iterator, causing the extent to be split,
ext2_block_iterate3() will end up calling the callback function twice
for some number of blocks.  Fix this.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-07-09 22:25:52 -04:00
Andreas Dilger 0bd0e59320 misc: use EXT2_I_SIZE() consistently to get size
Use the EXT2_I_SIZE() macro consistently to access the inode size.
The i_size/i_size_high combination is open coded in several places.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-07-08 20:52:59 -04:00
Theodore Ts'o 81c6377d93 libext2fs: don't hang in ext2fs_new_block2() on a full bigalloc file system
Prevent ext2fs_new_block2() from looping forever when a bigalloc file
system is full.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-07-07 22:29:00 -04:00
Theodore Ts'o 8e8a190fa2 libext2fs: teach ext2fs_bmap2() about bigalloc
This allows debugfs's write command to work correctly on bigalloc file
systems.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-07-07 22:27:42 -04:00
Theodore Ts'o 551e2e46ee libext2fs: move ext2fs_bmap2()'s the extent handling to a separate function
Separate the extent handling to a separate function to make BMAP_ALLOC
processing more efficient.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-07-07 14:21:12 -04:00
Theodore Ts'o b98d3e8fad libext2fs: teach ext2fs_block_alloc_stats2() about bigalloc
Change ext2fs_block_alloc_stats2() so that when a cluster is
allocated, the free blocks counter in the superblock is appropriately
decremented by the cluster size.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-07-07 14:21:12 -04:00
Theodore Ts'o c4d2d43e2d libext2fs: replace missing flexbg initialization in flexbg_offset
Commit 25567a7b0f accidentally removed the initialization for flexbg
and flexbg_size, which affected ext2fs_allocate_group_table() and
ext2fs_allocate_tables().   Replace them.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-07-07 14:20:42 -04:00
Theodore Ts'o d32c915abf libext2fs: Fix gcc -Wall warnings
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-07-07 13:50:22 -04:00
Theodore Ts'o 062cc472e6 libext2fs: fix 64-bit support in ext2fs_bmap2()
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-07-05 20:35:46 -04:00
Theodore Ts'o 41e102a4d1 libext2fs: fix 64-bit support in ext2fs_{read,write}_inode_full()
This fixes a problem where reading or writing inodes located after the
4GB boundary would fail.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-07-05 20:02:27 -04:00
Theodore Ts'o 25567a7b0f e2fsck, libext2fs: support for bigalloc file systems with a blocksize of 1024
Filesystems with a blocksize of 1024 have the superblock starting at
block #1.  However, the first data block in the superblock is 0 to
simplify the cluster calculations.  So we must compensate for this in
a number of places, mostly in the ext2fs library, but also in e2fsck.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-07-05 13:42:07 -04:00
Theodore Ts'o 1ca87790b9 libext2fs: fix makefile dependency problem
lib/ext2fs/Makefile.in had a buggy entry for blkmap64_ba.c in $(SRCS),
which caused this source file to not have a valid Makefile dependency
entry, so blkmap64_ba.o would not get rebuilt when it needed to be.

Also updated the Makefile dependency for the misc directory while
we're at it.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-06-19 21:58:20 -04:00
Theodore Ts'o b2e6c86d61 mke2fs, e2fsck: fix i_blocks handling for bigalloc file systems
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-06-16 10:12:43 -04:00
Theodore Ts'o fe75afbf33 Fix superblock field s_blocks_count for bigalloc file systems
Treat the s_blocks_count field in the superblock as a free block count
(instead of the number of free clusters) for bigalloc file systems.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-06-16 01:38:43 -04:00
Andreas Dilger 96367ad3bc misc: quiet "unused variable" compiler warnings
Some of the newly-merged patches added "unused variable" compiler
warnings.  Delete unused variables.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-06-15 22:17:38 -04:00
Theodore Ts'o 5f2e1a81d2 libext2fs: fix regression in ext2fs_new_block2() for uninit_bg file systems
Commit b0ecb787ef introduced a bug in check_block_uninit(), which is
used by ext2fs_new_block2().  This bug resulted in the block bitmap
for the block group in question not having space reserved for the file
system metadata blocks.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-06-15 22:11:59 -04:00
Theodore Ts'o 8f36e6925f Merge branch 'maint' into next 2011-06-11 12:55:26 -04:00
Andreas Dilger 388b66f727 ext2fs: fix error handling in ext2fs_add_dir_block
In ext2fs_add_dir_block() the dblist allocation size was changed to
grow as the number of items in the dblist increases.  However, the
error handling in case of allocation failure wasn't changed to match.

Fix the error case to revert to the old allocation size on failure.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-06-11 12:51:21 -04:00
Andreas Dilger 6c54689fad mke2fs: skip zeroing journal blocks
Add the ability to skip zeroing journal blocks on disk.  This can
significantly speed up mke2fs with large journals.  At worst the
uninitialized journal is only a very short-term risk (if at all),
because the journal will be overwritten on any new filesystem as
soon as any significant amount of data is written to disk, and
the new journal TID would need to match the offset/TID of an old
commit block still left on disk.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-06-11 12:19:12 -04:00
Andreas Dilger 931b58e1cb ext2fs: Handle internal journal over 2^32 bytes
The write_journal_inode() code is only setting the low 32-bit i_size
for the journal size, even though it is possible to specify a journal
up to 10M blocks in size.  Trying to create a journal larger than 2GB
will succeed, but an immediate e2fsck would fail.  Store i_size_high
for the journal inode when creating it, and load it upon access.

Use s_jnl_blocks[15] to store the journal i_size_high backup.  This
field is currently unused, as EXT2_N_BLOCKS is 15, so it is using
s_jnl_blocks[0..14], and i_size is in s_jnl_blocks[16].

Rename the "size" argument "num_blocks" for the journal creation functions
to clarify this parameter is in units of filesystem blocks and not bytes.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-06-11 12:17:29 -04:00
Andreas Dilger 534a4c3db5 ext2fs: don't use O_DIRECT if not available
O_DIRECT is not defined on OSX.  Since direct IO is only a new
optimization and not needed for correct functionality, disable
it if O_DIRECT is unavailable.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-06-11 11:50:01 -04:00
Andreas Dilger a1a769987b misc: fix compile warnings on OSX
The BLKFLSBUF and FDFLUSH ioctls are Linux specific, and do not
really have anything to do with __GNUC__ (which is also used on
OS/X and Solaris).  Only print these warnings on Linux systems.

statfs64() is deprecated on OSX and generates a deliberate warning.
Fix some other warnings that show up on OSX builds.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-06-11 11:50:01 -04:00
Andreas Dilger 65d05cec3d ext2fs: fix ext2fs_warn_bitmap32() return warning
This was reported as "control reaches end of non-void function",
but comparing to other similar functions it should be a void
function.  Since it is only declared in the "private" ext2fsP.h
header, it should be OK to change the function prototype.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-06-11 11:33:02 -04:00
Andreas Dilger 1ac2aa23ef ext2fs: fix undeclared posix_memalign() warning
Older distros do not define posix_memalign() by default in the
headers.  If ext2fs.h is included early in the headers, it is
possible to "#define _XOPEN_SOURCE 600" so that the stdlib.h
header will define it, but if ext2fs.h is included after stdlib.h
there is no posix_memalign() declaration.

Add a posix_memalign() declaration if stdlib.h didn't do it.  This
is a bit of a hack for GNU headers, but it works on Linux and OS/X
without problems.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-06-11 11:04:42 -04:00
Andreas Dilger cf5301d7f2 misc: clean up compiler warnings
Fix several types of compiler warnings (unused variables/labels),
uninitialized variables, etc that are hit with gcc -Wall.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-06-11 10:58:25 -04:00
Theodore Ts'o 44fe08f1fa e2fsck: add basic bigalloc support to check (but not yet repair) file systems
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-06-10 18:58:16 -04:00
Theodore Ts'o c6ed60cdeb mke2fs: support creating bigalloc file systems
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-06-10 18:57:22 -04:00
Theodore Ts'o 4a2a9b70c8 libext2fs: allocate clusters to files in expand_dir.c and mkjournal.c
Teach ext2fs_expand_dir() and ext2fs_add_journal_inode() about
allocating blocks when clustered allocation is enabled.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-06-10 18:54:53 -04:00
Theodore Ts'o b0ecb787ef libext2fs: teach block allocation functions about bigalloc/clusters
Optimize ext2fs_new_block2() and ext2fs_get_free_blocks2() when
bigalloc is enabled.  Also fix the uninitialized block bitmap code so
that it correctly deals clustered allocation.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-06-10 18:51:58 -04:00
Theodore Ts'o 94968e749b libext2fs: teach bitmap functions about bigalloc/cluster
This patch makes the following changes:

* ext2fs_allocate_block_bitmap() now allocates a bitmap with cluster
  granularity for bigalloc file systems.  For mke2fs and e2fsck, a
  newly added function, ext2fs_allocate_subcluster_bitmap() allocates
  a bitmap with block granularity (even for bigalloc file systems).
  The newly added function ext2fs_get_bitmap_granularity() will return
  the number of bits (log2) of the granularity used by the bitmap.

* The ext2fs_{mark,unmark,test}_block_bitmap2() functions will shift
  their passed-in argument by log2(cluster_ganularity) bits right.
  This means that the arguments for the single-argument bitmap
  functions will be interpreted with block granluarity, since this
  minimizes code changes in the rest of the code base.

* The ext2fs_{get,set}_block_bitmap_range() functions will interpret
  their arguments in cluster granularity.  This is a bit inconsistent,
  but the caller of those functions will need to be taught about the
  subtleties of clusters for bigalloc file systems.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-06-10 17:55:09 -04:00
Theodore Ts'o 1afb468b9a libext2fs: create extent-based directories if the extents feature is enabled
This allows mke2fs to create the root and lost+found directories using
extents.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-06-10 13:58:18 -04:00
Theodore Ts'o 6a6337c3df Merge branch 'maint' into next
Conflicts:
	lib/ext2fs/bitmaps.c
	lib/ext2fs/rw_bitmaps.c
	misc/dumpe2fs.c
2011-06-04 20:24:36 -04:00
Theodore Ts'o b92fbed778 libext2fs: Teach block bitmap read, write, and allocation funcs about clusters
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-06-04 20:05:22 -04:00
Theodore Ts'o 1da5ef7079 libext2fs: change fs->clustersize to fs->cluster_ratio_bits
The log2 of the ratio of cluster size to block size is far more useful
than just storing the cluster size.  So make this change, and then
define basic utility macros: EXT2FS_CLUSTER_RATIO(),
EXT2FS_CLUSTER_MASK(), EXT2FS_B2C(), EXT2FS_C2B(), and
EXT2FS_NUM_B2C().

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-06-04 17:43:10 -04:00
Theodore Ts'o ae9e37cd11 libext2fs: change EXT2_MAX_BLOCKS_PER_GROUP() to be cluster size aware
Change the EXT2_MAX_BLOCKS_PER_GROUP so that it takes the cluster size
into account.  This way we can open bigalloc file systems without
ext2fs_open() thinking that they are corrupt.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-06-04 16:40:26 -04:00
Theodore Ts'o 6220b6715f libext2fs: require cluster size == block_size when opening a !bigalloc fs
In ext2fs_open() check to make sure the cluster size superblock field
is the same as the block size field when the bigalloc feature is not
set.  This is necessary since we will start introducing calculations
based on the cluster size field.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-06-04 16:36:19 -04:00
Theodore Ts'o 1e082c6209 libcom_err: Fix install rule if installing with hard links
If $(LINK_INSTALL_FLAGS) is -f instead of -sf, the Makefile's install
rule would not work correctly while installing com_err.h

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-06-04 11:24:18 -04:00
Lukas Czerner bf0449b1a6 e2image: Add support for qcow2 format
This commit adds support for exporting filesystem into QCOW2 image
format. Like sparse format this saves space, by writing only necessary
(metadata blocks) into image. Unlike sparse image, QCOW2 image is NOT
sparse, hence does not change its size by copying with not-sparse-aware
tools.

New options '-Q' has been added to tell the e2image to use QCOW2 as an
output image format. QCOW2 supports encryption and compression, however
e2image so far does no support such features, however you can still
scramble filenames with '-s' option.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-05-18 11:51:06 -04:00
Lukas Czerner ffec46fea4 e2fsprogs: Add memory allocation and zero-out helpers
Add functions ext2fs_get_memzero() which will malloc() the memory
using ext2fs_get_mem(), but it will zero the allocated memory afterwards
with memset().

Add function ext2fs_get_arrayzero() which will use calloc() for
allocating and zero-out the array.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-05-18 11:41:02 -04:00
Tao Ma 598d20cc3c mke2fs: make s_inodes_per_group >= 8 in ext2fs_initialize
current mkfs.ext4 fails if we tried with the following parameters:

	mkfs.ext4  -m 0 -N 16 -O ^has_journal,^resize_inode,^uninit_bg,extent,meta_bg -b 1024 /dev/sdb3

It will cause segfault, but it is caused by another issue.  See my
patch "mke2fs: Avoid segmentation fault in
ext2fs_alloc_generic_bmap". And with that patch, the mkfs.ext4 will
fail with the error: /dev/sdb3: Memory allocation failed while setting
up superblock

The reason is that in ext2fs_initialize, we align s_inodes_per_group
to 8, but fails to consider the case that s_inodes_per_group < 8.
So make at least 8 inodes for s_inodes_per_group.

Signed-off-by: Tao Ma <boyu.mt@taobao.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-05-14 23:26:13 -04:00
Lukas Czerner 5fff975431 e2fsprogs: fix memory leak in ext2fs_free_generic_bmap()
In ext2fs_free_generic_bmap() when we are freeing 64-bit bitmap, we do
call free_bmap() to free backend specific bitmap structures, however we
should also free ext2fs_generic_bitmap structure as well.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-05-08 18:21:59 -04:00
Lukas Czerner ba7cb5d9d7 e2fsprogs: fix freeing bitmap in allocation error path
In ext2fs_alloc_generic_bmap() error path, when new bitmap allocation
fails ext2fs_generic_bitmap should be freed, however in current state it
first frees ext2fs_generic_bitmap and then
ext2fs_generic_bitmap->description which is wrong. This commit fix the
free ordering.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-05-08 18:06:30 -04:00
Theodore Ts'o 5af9eeaa7d Merge branch 'maint' into next
Conflicts:
	lib/e2p/ls.c
2011-03-18 16:44:37 -04:00
Theodore Ts'o 4df1618250 add new superblock field: s_overhead_blocks
It turns out that it's very hard to calculate overheads in the face of
clustered allocation (bigalloc).  This is because multiple metadata
blocks from different block groups can end up in the same allocation
cluster.  Calculating the exact overhead requires O(all block bitmaps)
in memory, or O(number of block groups**2) in time.  So we will
calculate this at mkfs time and stash it in the superblock.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-03-18 14:47:15 -04:00
Theodore Ts'o aa07cb79b0 mke2fs: If the device supports discard, don't print an error message
Check to see if the device supports discard before starting the
progress bar, and then printing an error about inappropriate ioctl for
device (when creating a file system image to a file, for example).

Also, add a function signature in the ext2_io.h header file for
io_channel_discard() and fix an extra, uneeded argument in mke2fs's
call to that function.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-02-27 20:09:54 -05:00
Theodore Ts'o 829d999488 Merge branch 'maint' into next
Conflicts:
	lib/ext2fs/initialize.c
2011-02-27 19:47:44 -05:00
Theodore Ts'o 412376efff Add basic BIGALLOC support for cluster-based allocation
This adds the superblock fields needed so that dumpe2fs works and the
code points and renames the superblock fields from describing
fragments to clusters.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2011-02-25 21:43:54 -05:00
Aditya Kali 0edcc27021 e2fsprogs: reserving code points for new ext4 quota feature
This patch adds support for detecting the new 'quota' feature in ext4.
The patch reserves code points for usr and group quota inodes and also
for the feature flag EXT4_FEATURE_RO_COMPAT_QUOTA.

Signed-off-by: Aditya Kali <adityakali@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-02-25 18:31:52 -05:00
Theodore Ts'o 214580a339 Merge branch 'maint' into next
Conflicts:
	misc/mke2fs.conf.5.in
2011-02-20 21:54:43 -05:00
Eric Sandeen 6a426c97ec e2fsprogs: enable user namespace xattrs by default
User namespace xattrs are generally useful, and I think extN
is the only filesystem requiring a special mount option to
enable them, when xattrs are otherwise available.  So this
change sets that mount option into the defaults, via a
mke2fs.conf option.

Note that if xattrs are config'd off, this will lead to a
mostly-harmless:

   EXT4-fs (sdc1): (no)user_xattr options not supported

message at mount time...

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-02-20 21:36:26 -05:00
Eric Sandeen 3daf592646 e2fsprogs: turn off enforced fsck intervals by default
The forced fsck often comes at unexpected and inopportune moments,
and even enterprise customers are often caught by surprise when
this happens.  Because a filesystem with an error condition will
be marked as requiring fsck anyway, I submit that the time-based
and mount-based checks are not particularly useful, and that
administrators can schedule fscks on their own time, or tune2fs
the enforced intervals if they so choose.  This patch disables the
intervals by default, and I've added a new mkfs.conf option to
turn on the old behavior of random, unexpected, time-consuming
fscks at boot time.  ;)

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-02-20 20:45:12 -05:00
Eric Sandeen b6d274f7c6 e2fsprogs: create com_err.h link in includedir
After debian bug #192277, debian/rules started making a symlink
to com_err.h in /usr/include.  Now I have Fedora bug #550889
for the same issue, and perhaps it's time to make this link
by default, rather than fixing it up in packaging steps?

[ Changed by tytso to remove the explicit -s option; this will default
  to creating a hard link by default, which slightly faster.  If
  people want to use symlinks for all links during the install
  process, they can use configure option --enable-symlink-install.
  The reason for this change is that some file systems, like AFS,
  don't support symlinks, and AFS users complain when they can't build
  or install into AFS.  So I don't want to use symlinks
  unconditionally without a way of switching things back and forth,
  and it's easier if we just make all links made during the install
  process to be hard links or sym links. ]

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2011-02-20 18:10:21 -05:00
Theodore Ts'o da2a5a4bae Merge branch 'maint' into next
Conflicts:
	resize/resize2fs.c
2010-12-22 19:00:21 -05:00
Theodore Ts'o a4fdf09414 libext2fs: Don't use the extended rec_len encoding for standard file systems
If the file system has a blocksize less than 64k, then don't use the
extended rec_len encoding, to be consistent with what the kernel will
do.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2010-12-22 13:53:02 -05:00
Namhyung Kim 91dc3f0db0 libext2fs: fix possible memory leak in write_journal_inode()
ext2fs_zero_block2() allocates static buffer if needed so it
should be freed at last (call it again with 0 args).

Signed-off-by: Namhyung Kim <namhyung@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2010-12-22 10:43:01 -05:00
Theodore Ts'o 58caabc37d ext2fs: fix memory leak in an error case in inode_open()
Reported-by: Namhyung Kim <namhyung@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2010-12-21 18:05:07 -05:00
Theodore Ts'o 2150278fa2 libext2fs: fix potential free() of garbage in ext2fs_update_bb_inode()
There was a potential of freeing an uninitialized pointer in
rec.block_buf, which was pointed out by Namhyung Kim <namhyung@gmail.com>

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2010-12-20 10:58:58 -05:00
Namhyung Kim f57e64a851 libext2fs: fix dubious code in ext2fs_unmark_generic_bitmap()
Looks like a copy & paste problem to me.

Signed-off-by: Namhyung Kim <namhyung@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2010-12-20 10:54:27 -05:00
Namhyung Kim 877d1dc0ab libext2fs: remove unnecessary casts to ext2fs_generic_bitmap
Signed-off-by: Namhyung Kim <namhyung@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2010-12-20 10:50:33 -05:00
Namhyung Kim 3ece37ea0a libext2fs: fix dubious code in ext2fs_numeric_progress_init()
Looks like a copy & paste problem to me.

Signed-off-by: Namhyung Kim <namhyung@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2010-12-20 10:07:10 -05:00
Namhyung Kim beb4295ca4 libext2fs: fix potential build failure with OMIT_COM_ERR
This fixes following build failure when OMIT_COM_ERR is defined:

 lib/ext2fs/gen_bitmap.c: In function ‘ext2fs_clear_generic_bitmap’:
 lib/ext2fs/gen_bitmap.c:437: error: invalid storage class for function ‘ext2fs_test_clear_generic_bitmap_range’
 lib/ext2fs/gen_bitmap.c:559: error: expected declaration or statement at end of input
 lib/ext2fs/gen_bitmap.c: In function ‘ext2fs_get_generic_bitmap_end’:
 lib/ext2fs/gen_bitmap.c:559: error: expected declaration or statement at end of input
 lib/ext2fs/gen_bitmap.c: In function ‘ext2fs_get_generic_bitmap_start’:
 lib/ext2fs/gen_bitmap.c:559: error: expected declaration or statement at end of input
 lib/ext2fs/gen_bitmap.c: In function ‘ext2fs_unmark_generic_bitmap’:
 lib/ext2fs/gen_bitmap.c:559: error: expected declaration or statement at end of input
 lib/ext2fs/gen_bitmap.c: In function ‘ext2fs_mark_generic_bitmap’:
 lib/ext2fs/gen_bitmap.c:559: error: expected declaration or statement at end of input
 lib/ext2fs/gen_bitmap.c: In function ‘ext2fs_test_generic_bitmap’:
 lib/ext2fs/gen_bitmap.c:559: error: expected declaration or statement at end of input
make[2]: *** [gen_bitmap.o] Error 1
make[2]: Leaving directory e2fsprogs/lib/ext2fs'
make[1]: *** [all-libs-recursive] Error 1
make[1]: Leaving directory e2fsprogs'
make: *** [all] Error 2

Signed-off-by: Namhyung Kim <namhyung@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2010-12-20 10:07:10 -05:00
Theodore Ts'o 562f264243 Merge branch 'maint' into next
Conflicts:
	lib/ext2fs/block.c
	lib/ext2fs/csum.c
	resize/main.c
2010-12-20 10:06:58 -05:00
Theodore Ts'o 9098c986b6 libext2fs: Fix compile bug on big-endian architectures
Addresses-Sourceforge-Bug: #3138115

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2010-12-16 22:11:43 -05:00
Theodore Ts'o 94ded6c16b libext2fs: fix ref_offset callback with sparse files in ext2fs_block_iterate2()
Addresses-Sourceforge-Bug: 3081087

Reported-by: vmo@users.sourceforge.net
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
2010-12-16 19:34:24 -05:00
Eric Sandeen c816ecb204 e2fsprogs: fix type-punning warnings
Flags used during RHEL/Fedora builds lead to a couple type-punning
warnings:

  recovery.c: In function 'do_one_pass':
  recovery.c:539: warning: dereferencing type-punned pointer will break strict-aliasing rules
  ./csum.c: In function 'print_csum':
  ./csum.c:170: warning: dereferencing type-punned pointer will break strict-aliasing rules

The two changes below fix this up.

Note that the csum test binary output changes slightly, but this does
not break any tests.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2010-12-16 17:15:30 -05:00
Lukas Czerner efa1a355a1 e2fsck: Discard free data and inode blocks.
In Pass 5 when we are checking block and inode bitmaps we have great
opportunity to discard free space and unused inodes on the device,
because bitmaps has just been verified as valid. This commit takes
advantage of this opportunity and discards both, all free space and
unused inodes.

I have added new set of options, 'nodiscard' and 'discard'. When the
underlying devices does not support discard, or discard ends with an
error, or when any kind of error occurs on the filesystem, no further
discard attempt will be made and the e2fsck will behave as it would
with nodiscard option provided.

As an addition, when there is any not-yet-zeroed inode table and
discard zeroes data, then inode table is marked as zeroed.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2010-11-22 21:01:41 -05:00
Lukas Czerner d866599ab4 e2fsprogs: Add CHANNEL_FLAGS_DISCARD_ZEROES flag for io_manager
When the device have discard support and simultaneously discard zeroes
data (and it is properly advertised), then we can take advantage of such
behavior in several e2fsprogs tools.

Add new flag CHANNEL_FLAGS_DISCARD_ZEROES for struct_io_channel so
each io_manager can take advantage of this. The flag is properly set
according to BLKDISCARDZEROES ioctl in unix_open.

Also remove old mke2fs_discard_zeroes_data() function and substitute it
with helper which test this flag.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2010-11-22 20:41:46 -05:00
Lukas Czerner e90a59ed43 e2fsprogs: Add discard function into struct_io_manager
In order to provide generic "discard" function for all e2fsprogs tools
add a discard function prototype into struct_io_manager. Specific
function for specific io managers can be crated that way.

This commit also creates unix_discard function which uses BLKDISCARD
ioctl to discard data blocks on the block device and bind it into
unit_io_manager structure to be available for all e2fsprogs tools.
Note that BLKDISCARD is still Linux specific ioctl, however other
unix systems may provide similar functionality. So far the
unix_discard() remains linux specific hence is embedded in #ifdef
__linux__ macro.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2010-11-22 20:41:00 -05:00
Justin Maggard 4fca2b9c07 Fix EXT4_FEATURE_RO_COMPAT_HUGE_FILE check
Creating a 4TB file on a filesystem with the 64bit flag set results in
e2fsck consistently complaining about i_blocks being wrong, with
confusing messages like this:

Inode 29818882, i_blocks is 8388608816, should be 8388608816.  Fix? no

That appears to be caused by ext2fs_inode_i_blocks() checking for the
EXT4_FEATURE_RO_COMPAT_HUGE_FILE in the wrong place.  Fix it.

Signed-off-by: Justin Maggard <jmaggard10@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2010-11-22 17:32:28 -05:00