Convert all call sites that write zero blocks to disk to use
ext2fs_zero_blocks2() since it can use Linux's zero out feature to do
the writes more quickly. Reclaim the zero buffer at freefs time and
make the write-zeroes fallback use a larger buffer.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
In the loop in ext2fs_get_free_blocks2, we ask the bitmap if there's a
range of free blocks starting at "b" and ending at "b + num - 1".
That quantity is the number of the last block in the range. Since
ext2fs_blocks_count() returns the number of blocks and not the number
of the last block in the filesystem, the check is incorrect.
Put in a shortcut to exit the loop if finish > start, because in that
case it's obvious that we don't need to reset to the beginning of the
FS to continue the search for blocks. This is needed to terminate the
loop because the broken test meant that b could get large enough to
equal finish, which would end the while loop.
The attached testcase shows that with the off by one error, it is
possible to throw e2fsck into an infinite loop while it tries to
find space for the inode table even though there's no space for one.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Fix various unused variable and use-uninitialized warnings.
Add generated files into .gitignore.
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If there are hundreds of thousands of blocks which are in use before
the first free block, it is much, MUCH faster to use
ext2fs_find_first_zero_block_bitmap2() instead of searching the
allocation bitmap bit by bit.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Commit 8e44eb64bb (libext2fs: mark group data blocks when loading
block bitmap) simplified check_block_uninit since we are now
initializing the bitmap when it is loaded from disk. It left some
variables which were being set but never used, however. In addition,
since we only need check_block_uninit() to clear the block bitmap's
uninit flag, rename it to clear_block_uninit(), and only call it once
we have found a free block in ext2fs_new_blocks2().
This cleans up the code some and optimizes things if we need to search
multiple block groups trying to find a free block.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: Darrick J. Wong <darrick.wong@oracle.com>
The kernel[1] and e2fsck[2] both react to a BLOCK_UNINIT group by
calculating the block bitmap that's needed to show all the group
blocks for that group (if any) and using that. However, when reading
bitmaps from disk, libext2fs simply imports a block of zeroes into the
bitmap, without bothering to check for group blocks. This erroneous
behavior results in the filesystem having a block bitmap that does not
accurately reflect disk contents, and worse yet makes it seem as
though superblocks, group descriptors, bitmaps, and inode tables are
"free" space on disk.
So, fix the block bitmap loading routines to calculate the correct
block bitmap for all groups and load it into the main fs block bitmap.
This also fixes bogus debugfs output such as:
Group 1: (Blocks 8193-16384) [INODE_UNINIT, BLOCK_UNINIT]
Checksum 0x1310, unused inodes 512
Backup superblock at 8193, Group descriptors at 8194-8217
Reserved GDT blocks at 8218-8473
Block bitmap at 283 (bg #0 + 282), Inode bitmap at 299 (bg #0 + 298)
Inode table at 442-569 (bg #0 + 441)
7911 free blocks, 512 free inodes, 0 directories, 512 unused inodes
Free blocks: 8193-16384
Free inodes: 513-1024
Notice how the "free blocks" range includes the backup sb & GDT area
and doesn't match the free block count.
Worse yet, debugfs' testb command will report those group descriptor
blocks as not being in use unless the user also instructs debugfs to
find a free block first. That is a rather surprising result:
debugfs: testb 8194
Block 8194 not in use
debugfs: ffb 1 16380
Free blocks found: 16380
debugfs: testb 8194
Block 8194 marked in use
Also, remove the part of check_block_uninit() that "fixes" the bitmap
since we're doing that at bitmap load time now.
[1] kernel git 717d50e4971b81b96c0199c91cdf0039a8cb181a
"Ext4: Uninitialized Block Groups"
[2] e2fsprogs git f5fa20078b
"Add support for EXT2_FEATURE_COMPAT_LAZY_BG"
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
There are a number of places where we multiply a dgrp_t with
s_blocks_per_group expecting that we will get a blk64_t. This
requires a cast, or using the convenience function
ext2fs_group_first_block2().
This audit was suggested by Eric Sandeen.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Change the block group algorithm to use the same algorithm as the rest
of the metadata_csum. This mostly involves providing a helper
function to tell if group descriptors should have checksums set or
verified, and modifying the gdt checksum code to use the correct
algorithm.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Update the block group descriptor checksum and mark the superblock and
allocation bitmaps as dirty in check_inode_uninit() and
check_block_uninit().
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This function searches a bitmap for the first zero bit within a range.
It checks if there is a bitmap backend specific implementation
available (if the relevant field in bitmap_ops is non-NULL). If not,
it uses a generic and slow method by repeatedly calling test_bmap() in
a loop. Also change ext2fs_new_inode() to use this new function.
This change in itself does not result in a large speedup, rather it
refactors the code in preparation for the introduction of a faster
find_first_zero() for bitarray based bitmaps.
Signed-off-by: Sami Liedes <sami.liedes@iki.fi>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Filesystem shrinking in particular is a heavy user of this loop in
ext2fs_new_inode(). This change makes resize2fs use 24% less CPU time
for shrinking a 100G filesystem.
Signed-off-by: Sami Liedes <sami.liedes@iki.fi>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The DEFS line in MCONFIG had gotten so long that it exceeded 4k, and
this was starting to cause some tools heartburn. It also made "make
V=1" almost useless, since trying to following the individual commands
run by make was lost in the noise of all of the defines.
So fix this by putting the configure-generated defines in lib/config.h
and the directory pathnames to lib/dirpaths.h.
In addition, clean up some vestigal defines in configure.in and in the
Makefiles to further shorten the cc command lines.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Commit b0ecb787ef introduced a bug in check_block_uninit(), which is
used by ext2fs_new_block2(). This bug resulted in the block bitmap
for the block group in question not having space reserved for the file
system metadata blocks.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Optimize ext2fs_new_block2() and ext2fs_get_free_blocks2() when
bigalloc is enabled. Also fix the uninitialized block bitmap code so
that it correctly deals clustered allocation.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The top-level COPYING file states that the e2p and ext2fs libraries
are available under the LGPLv2. The files were incorrectly labelled.
Alex Thomas/Luster has been consulted wrt to the ext3_extents.h file;
the rest of the files were primarily authored by Theodore Ts'o.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
After cleaning up ext2fs_bg_flag_set() and ext2fs_bg_flag_clear(),
we're left with ext2fs_bg_flag_test(). Convert it to
ext2fs_bg_flags_test().
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The ext2fs_bg_flag* functions were confusing.
Currently we have this:
void ext2fs_bg_flags_set(ext2_filsys fs, dgrp_t group, __u16 bg_flags);
void ext2fs_bg_flags_clear(ext2_filsys fs, dgrp_t group,__u16 bg_flags);
(_set (unused) sets exactly bg_flags; _clear clears all and ignores bg_flags)
and these, which can twiddle individual bits in bg_flags:
void ext2fs_bg_flag_set(ext2_filsys fs, dgrp_t group, __u16 bg_flag);
void ext2fs_bg_flag_clear(ext2_filsys fs, dgrp_t group, __u16 bg_flag);
A better interface, after the patch below, is just:
ext2fs_bg_flags_zap(fs, group) /* zeros bg_flags */
ext2fs_bg_flags_set(fs, group, flags) /* adds flags to bg_flags */
ext2fs_bg_flags_clear(fs, group, flags) /* clears flags in bg_flags */
and remove the original ext2fs_bg_flags_set / ext2fs_bg_flags_clear.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The new ext2fs_super_and_bgd_loc2() function has some changes aside
from just blk64_t support. Lets make sure that the interfaces are
sane by adding libext2fs support early to get the new API tested here.
Signed-off-by: Jose R. Santos <jrs@us.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add new ext2fs_new_block2(), ext2fs_get_free_blocks2() and
ext2fs_alloc_block2() that take and return blk64_t.
Signed-off-by: Jose R. Santos <jrs@us.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add a sanity check to makesure that even if the superblock field
s_first_inode is insane, that we won't return an invalid inode number.
(The function will return the error EXT2_ET_INODE_ALLOC_FAIL in that
case.)
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Some of these could affect filesystems between 2^31 and 2^32-1 blocks.
Thanks to Valerie Aurora Henson for pointing out the problems in
lib/ext2fs/alloc_tables.c, which led me to do a "make gcc-wall" scan
over the source tree.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If the allocation functions need to allocate out of a block group
where the inode and/or block bitmaps have not yet been initialized,
initialize them so ext2fs_new_block() and ext2fs_new_inode() work
correctly.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add callback functions for ext2fs_alloc_block() and
ext2fs_block_alloc_stats(). This is needed so e2fsck can be informed
when the extent_set_bmap() function needs to allocate or deallocate
blocks.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If a block buffer was not supplied and ext2fs_alloc_block() returned
with no errors, it would leak a temporary block buffer.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add support for a new flag, DIRENT_FLAG_INCLUDE_REMOVED,
which will return deleted directory entries.
ext2fs_dir_iterate2 takes a new callback function which
is identical with the one used by
ext2fs_dblist_dir_iterate(). If the directory entry is
deleted, the callback function will be called with the
entry paraemter set to DIRENT_DELETED_FILE.
Makefile.in, alloc_stats.c (ext2fs_inode_alloc_stats,
ext2fs_block_alloc_stats): New functions which update
block/inode allocation statistics in the bitmaps, block
group descriptors, and superblock.
mkjournal.c (mkjournal_proc), mkdir.c (ext2fs_mkdir),
expanddir.c (expand_dir_proc), bb_inode.c
(clear_bad_block_proc, set_bad_block_proc,
ext2fs_update_bb_inode), alloc.c (ext2fs_alloc_block):
Update to use new block/inode allocation statistics.
alloc.c, bb_inode.c, bitmaps.c, bitops.h, block.c, bmap.c, bmove.c,
brel.h, cmp_bitmaps.c, dblist.c, dblist_dir.c, dir_iterate.c,
expanddir.c, ext2fs.h, ext2fsP.h, fileio.c, finddev.c, get_pathname.c,
icount.c, inode.c, irel.h, irel_ma.c, ismounted.c, link.c, lookup.c,
mkdir.c, mkjournal.c, namei.c, newdir.c, read_bb_file.c, test_io.c,
tst_iscan.c, unix_io.c, unlink.c: Change use of ino_t to ext2_ino_t,
to protect applications that attempt to compile
-D_FILE_OFFSET_BITS=64, since this inexplicably changes ino_t(!?). So
we use ext2_ino_t to avoid an unexpected ABI change.
inode.c (ext2fs_open_inode_scan): Initialize the group variables
so that we don't need to call get_next_blockgroup() the first
time around. Saves a bit of time, and prevents us from
needing to assign -1 to current_group (which is an unsigned
value).
icount.c (insert_icount_el): Cast the estimated number of inodes
from a float to an ino_t.
alloc.c, alloc_tables.c, badlbocks.c, bb_compat.c, bb_inode.c,
bitmaps.c, bitops.c, block.c, bmap.c, bmove.c, brel_ma.c,
check_desc.c, closefs.c, cmp_bitmaps.c, dblist.c,
dblist_dir.c, dir_iterate.c, dirblock.c, dupfs.c, expanddir.c,
ext2fs.h, fileio.c, freefs.c, get_pathname.c, getsize.c,
icount.c, initialize.c, inline.c, inode.c, irel_ma.c,
ismounted.c, link.c, lookup.c, mkdir.c, namei.c, native.c,
newdir.c, openfs.c, read_bb.c, read_bb_file.c, rs_bitmap.c,
rw_bitmaps.c, swapfs.c, test_io.c, tst_badblocks.c,
tst_getsize.c, tst_iscan.c, unix_io.c, unlink.c, valid_blk.c,
version.c: If EXT2_FLAT_INCLUDES is defined, then assume all
of the ext2-specific header files are in a flat directory.
block.c, bmove.c, dirblock.c, fileio.c: Explicitly cast
all assignments from void * to be compatible with C++.
closefs.c (ext2fs_flush): Add a call to io_channel_flush() to
make sure the contents of the disk are flushed to disk.
dblist.c (ext2fs_add_dir_block): Change new to be new_entry to
avoid C++ namespace clash.
bitmaps.c (ext2fs_copy_bitmap): Change new to be new_map to
avoid C++ namespace clash.
ext2fs.h, bb_inode.c, block.c, bmove.c, brel.h, brel_ma.c,
irel.h, irel_ma.c, dblist.c, dblist_dir.c, dir_iterate.c,
ext2fsP.h, expanddir.c, get_pathname.c, inode.c, link.c,
unlink.c: Change private to be priv_data (to avoid C++
namespace clash)
dblist.c (ext2fs_get_num_dirs): Make ext2fs_get_num_dirs more paranoid
about validating the directory counts from the block group
information.
all files: Don't include stdlib.h anymore; include it in ext2_fs.h,
since that file requires stdlib.h
ChangeLog, Makefile.in, dirinfo.c:
dirinfo.c (e2fsck_add_dir_info): Use ext2fs_get_num_dirs instead of
e2fsck_get_num_dirs, which has been removed.
Makefile.in (PROGS): Remove @EXTRA_PROGS@, since we don't want to
compile and install flushb.
ChangeLog, configure.in:
Remove @EXTRA_PROGS@, since we aren't using it in 2fsck/Makefile.in anymore
ChangeLog, Makefile.in:
Install debugfs in /sbin, instead of /usr/sbin.
libext2fs.texinfo:
Update version string to be 1.12
Makefile.in:
Fix bug in find script which made the exclusion list, where a '-' was
missing from an -name option.
Added definition of ext2fs_get_mem, ext2fs_free_mem, and
ext2fs_resize_mem in ext2fs.h, and changed all library routines to use
these wrapper functions.
ext2fs.h: Added function declarations and constants for bmap.c and
fileio.c.
ext2_err.et.in: Added new error messages EXT2_FILE_RO and
EXT2_ET_MAGIC_EXT2_FILE
Makefile.in: Added files bmap.c and fileio.c, and temporarily
commented out brel_ma.c and irel_ma.c
bmap.c: New file which maps a file's logical block number to its
physical block number.
fileio.c: New file which implements simple file reading and writing
primitives.
alloc.c (ext2fs_alloc_block): New function which allocates a block,
zeros it, and updates the filesystem accounting records appropriately.
alloc.c (ext2fs_alloc_block): New function which allocates a
block and updates the filesystem accounting records
appropriately.
ext2_err.et.in: Added new error codes: EXT2_NO_MEMORY,
EXT2_INVALID_ARGUMENT, EXT2_BLOCK_ALLOC_FAIL, EXT2_INODE_ALLOC_FAIL,
EXT2_NOT_DIRECTORY
Change various library files to use these functions instead of EINVAL,
ENOENT, etc.
ChangeLog, pass1.c, pass3.c:
pass3.c (get_lost_and_found): Check error return of
EXT2_FILE_NOT_FOUND instead of ENOTDIR
pass1.c (pass1_check_directory): Return EXT2_NO_DIRECTORY instead of
ENOTDIR
expect.icount:
Change expected error string to be "Invalid argument passed to ext2 library"
instead of just "Invalid argument"
block.c (ext2fs_block_iterate2): Use retval which is a errcode_t type.
bitmaps.c (make_bitmap): Use size_t instead of int where appropriate.
bb_inode.c (set_bad_block_proc): Add #pragma argsused for Turbo C.
alloc.c (ext2fs_new_inode): Use ino_t instead of int for the group number.
get_pathname.c: Use ino_t instead of int where appropriate.
ext2fs.h: Make the magic structure element be errcode_t instead of int.
alloc.c alloc_tables.c badblocks.c bb_compat.c bb_inode.c
bitmaps.c block.c bmove.c brel_ma.c check_desc.c closefs.c
cmp_bitmaps.c dblist.c dblist_dir.c dir_iterate.c dirblock.c
dupfs.c expanddir.c freefs.c get_pathname.c icount.c
initialize.c inline.c inode.c irel_ma.c link.c llseek.c
lookup.c mkdir.c namei.c newdir.c read_bb.c read_bb_file.c
rs_bitmap.c rw_bitmaps.c swapfs.c test_io.c tst_badblocks.c
tst_iscan.c unix_io.c unlink.c valid_blk.c version.c: Add an
#ifdef for HAVE_UNISTD_H
Darrick J. Wong