If we're doing a strict overwrite (same data size) of data in an
inline data file, we should be able to skip the size check. If the
in-core EA representation is fine but the on-disk EA is slightly
corrupt (this happens when fixing minor errors in an inline dir), the
ext2fs_xattr_inode_max_size() call, which reads the disk EA, can lead
us to think that there's no space when in reality there is no issue
with doing a strict overwrite.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The inline data code fails to perform endianness conversions correctly
or at all in a number of places, so fix this so that big-endian
machines function properly.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
When we're (a) reading EAs into a buffer; (b) byte-swapping EA
entries; or (c) checking EA data, be careful not to run off the end of
the memory buffer, because this causes invalid memory accesses and
e2fsck crashes. This can happen if we encounter a specially crafted
FS image.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Perform a little more sanity checking of EA value offsets so that we
don't crash while trying to load things from the filesystem.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If s_first_meta_bg is greater than the of number block group
descriptor blocks, then reading or writing the block group descriptors
will end up overruning the memory buffer allocated for the
descriptors. Fix this by limiting first_meta_bg to no more than
fs->desc_blocks. This doesn't correct the bad s_first_meta_bg value,
but it avoids causing the e2fsprogs userspace programs from
potentially crashing.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Commit baa3544609 ("libext2fs: have UNIX IO manager use
pread/pwrite) causes a breakage on 32-bit systems where off_t is
32-bits for file systems larger than 4GB. Fix this by using
pread64/pwrite64 if possible, and if pread64/pwrite64 is not present,
using pread/pwrite only if the size of off_t is at least as big as
ext2_loff_t.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Place the allocation bitmaps and inode table blocks so they are
adjacent, even in the last flexbg.
Previously, after running "mke2fs -t ext4 DEV 286720", the layout of
the last few block groups would look like this:
Group 32: (Blocks 262145-270336) [INODE_UNINIT, ITABLE_ZEROED]
Block bitmap at 262145 (+0), Inode bitmap at 262161 (+16)
Inode table at 262177-262432 (+32)
Group 33: (Blocks 270337-278528) [INODE_UNINIT, BLOCK_UNINIT, ITABLE_ZEROED]
Block bitmap at 262146 (bg #32 + 1), Inode bitmap at 262162 (bg #32 + 17)
Inode table at 262433-262688 (bg #32 + 288)
Group 34: (Blocks 278529-286719) [INODE_UNINIT, ITABLE_ZEROED]
Block bitmap at 262147 (bg #32 + 2), Inode bitmap at 262163 (bg #32 + 18)
Inode table at 262689-262944 (bg #32 + 544)
Now, they look like this:
Group 32: (Blocks 262145-270336) [INODE_UNINIT, ITABLE_ZEROED]
Block bitmap at 262145 (+0), Inode bitmap at 262148 (+3)
Inode table at 262151-262406 (+6)
Group 33: (Blocks 270337-278528) [INODE_UNINIT, BLOCK_UNINIT, ITABLE_ZEROED]
Block bitmap at 262146 (bg #32 + 1), Inode bitmap at 262149 (bg #32 + 4)
Inode table at 262407-262662 (bg #32 + 262)
Group 34: (Blocks 278529-286719) [INODE_UNINIT, ITABLE_ZEROED]
Block bitmap at 262147 (bg #32 + 2), Inode bitmap at 262150 (bg #32 + 5)
Inode table at 262663-262918 (bg #32 + 518)
This reduces the free space fragmentation in a freshly created file
system. It also allows the following mke2fs command to succeed:
mke2fs -t ext4 -b 4096 -O ^resize_inode -G $((2**20)) DEV 2130483
(Note that while this allows people to run mke2fs with insanely large
flexbg sizes, this is not a recommended practice, as the kernel may
refuse to resize such a file system while mounted, since it currently
tries to allocate an in-memory data structure based on the size of the
flexbg, and so a file system with a very large flexbg size will cause
the memory allocation to fail. This will hopefully be fixed in a
future kernel release, but if the goal is to force all of the metadata
blocks to be at the beginning of the file system, it's better to use
the packed_meta_blocks configuration parameter in mke2fs.conf.)
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
This reverts commit d988201ef9.
The problem with this commit is that causes common small file system
configurations to fail. For example:
mke2fs -O flex_bg -b 4096 -I 1024 -F /tmp/tt 79106
mke2fs 1.42.11 (09-Jul-2014)
/tmp/tt: Invalid argument passed to ext2 library while setting
up superblock
This check in ext2fs_initialize() was added to prevent the metadata
from being allocated beyond the end of the filesystem, but it is
also causing a wide range of failures for small filesystems.
We'll address this in a different way, by using a smarter algorithm
for deciding the layout of metadata blocks for the last flex block
group.
Reported-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If an inode fails checksum verification, don't stuff a copy of it in
the inode cache, because this can cause the library to fail to return
the "corrupt inode" error code.
In general, this happens if ext2fs_read_inode_full() is called twice
on an inode with an incorrect checksum. If fs->flags has
EXT2_FLAG_IGNORE_CSUM_ERRORS set during the first call and *unset*
during the second call, the cache hit during the second call fails to
return EXT2_ET_INODE_CSUM_INVALID as you'd expect. This happens
during fsck because the first read_inode call happens as part of
check_blocks and the second call happens during inode checksum
revalidation. A file system with a slightly corrupt non-extent inode
will trigger this.
While we're at it, make the inode read function consistent with the
rest of libext2fs -- copy the metadata object into the caller's buffer
even if it fails checksum verification. This will help e2fsck avoid a
double re-read later on down the line.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
When we need to modify the "ignore checksum error" behavior flag to
get us past a library call, it's possible that the library call can
result in other flag bits being changed. Therefore, it is not correct
to restore unconditionally the previous flags value, since this will
have unintended side effects on the other fs->flags; nor is it correct
to assume that we can unconditionally set (or clear) the "ignore csum
error" flag bit. Therefore, we must merge the previous value of the
"ignore csum error" flag with the value of flags after the call.
Note that we want to leave checksum verification on as much as
possible because doing so exposes e2fsck bugs where two metadata
blocks are "sharing" the same disk block, and attempting to fix one
before relocating the other causes major filesystem damage. The
damage is much more obvious when a previously checked piece of
metadata suddenly fails in a subsequent pass.
The modifications to the pass 2, 3, and 3A code are justified as
follows: When e2fsck encounters a block of directory entries and
cannot find the placeholder entry at the end that contains the
checksum, it will try to insert the placeholder. If that fails, it
will schedule the directory for a pass 3A reconstruction. Until that
happens, we don't want directory block writing (pass 2), block
iteration (pass 3), or block reading (pass 3A) to fail due to checksum
errors, because failing to find the placeholder is itself a checksum
verification error, which causes e2fsck to abort without fixing
anything.
The e2fsck call to ext2fs_read_bitmaps must never fail due to a
checksum error because e2fsck subsequently (a) verifies the bitmaps
itself; or (b) decides that they don't match what has been observed,
and rewrites them.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Add a new behavior flag to the inode scan functions; when specified,
this flag will do some simple sanity checking of entire inode table
blocks. If all the checksums are ok, we can skip checksum
verification on individual inodes later on. If more than half of the
inodes look "insane" (bad extent tree root or checksum failure) then
ext2fs_get_next_inode_full() can return a special status code
indicating that what's in the buffer is probably garbage.
When e2fsck' inode scan encounters the 'inode is garbage' return code
it'll offer to zap the inode straightaway instead of trying to recover
anything. This replaces the previous behavior of asking to zap
anything with a checksum error (strict_csum).
Signed-off-by: Darrick J. Wong <darrick.wong@orale.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Remove the code that would zap an extent block immediately if the
checksum failed (i.e. strict_csums). Instead, we'll only do that if
the extent block header shows obvious structural problems; if the
header checks out, then we'll iterate the block and see if we can
recover some extents.
Requires a minor modification to ext2fs_extent_get such that the
extent block will be returned in the buffer even if the return code
indicates a checksum error. This brings its behavior in line with
the rest of libext2fs.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
When reading an EA block in from disk, do a quick sanity check of the
block header, and return an error if we think we have garbage. Teach
e2fsck to ignore the new error code in favor of doing its own
checking, and remove the strict_csums bits while we're at it.
(Also document some assumptions in the new ext_attr code.)
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
When we're appending an extent to the end of a file and the index
block is full, don't split the index block into two half-full index
blocks because this leaves us with under utilized index blocks, at
least in the fallocate case. Instead, copy the last extent from the
full block into the new block. This isn't perfect utilization, but
there's a lot of work involved in teaching extent.c to be able to goto
a nonexistent node in a newly allocated (and empty) extent block.
This patch does not fix the general problem of keeping the extent tree
balanced.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If pread/pwrite are present, have the UNIX IO manager use them for
aligned IOs (instead of the current seek -> read/write), thereby
saving us a (minor) amount of system call overhead.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Use EXT2_MIN_BLOCK_SIZE, JFS_MIN_JOURNAL_BLOCKS, SUPERBLOCK_SIZE, and
SUPERBLOCK_OFFSET instead of hardcoded 1024 when it is okay, and also
add a helper ext2fs_journal_sb_start() that will return start of
journal sb with special case for fs with 1k block size.
Signed-off-by: Azat Khuzhin <a3at.mail@gmail.com>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Provide an API to set i_size in an inode and take care of all required
feature flag modifications. Refactor the code to use this new
function.
[ Moved the function to lib/ext2fs/blk_num.c, which is the rest of
these sorts of functions live, and renamed it to be
ext2fs_inode_size_set() instead of ext2fs_inode_set_size() to be
consistent with the other functions in in blk_num.c -- tytso ]
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
We rely on a nasty hack to adjust the free block count where we pass
signed value into ext2fs_free_blocks_count_add(), which takes an
64-bit unsigned value, and relies on overflow and C's signed->unsigned
semantics to do the subtraction. This works, so long as a 64-bit
signed value is used.
Unfortunately, ext2fs_block_alloc_stats2() and
ext2fs_block_alloc_stats_range(), this is not true, so on a 64-bit
file system, the free blocks accounting can get screwed up.
A simple way to demonstrate the problem is:
mke2fs -F -t ext4 -O 64bit /tmp/foo.img 1M
e2fsck -fy /tmp/foo.img
... which will result in the following e2fsck complaint:
Pass 5: Checking group summary information
Free blocks count wrong (4294968278, counted=982).
Fix? yes
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
There are a number of places where we need convert groups to blocks or
clusters by multiply the groups by blocks/clusters per group.
Unfortunately, both quantities are 32-bit, but the result needs to be
64-bit, and very often the cast to 64-bit gets lost.
Fix this by adding new macros, EXT2_GROUPS_TO_BLOCKS() and
EXT2_GROUPS_TO_CLUSTERS().
This should fix a bug where resizing a 64bit file system can result in
calculate_minimum_resize_size() looping forever.
Addresses-Launchpad-Bug: #1321958
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Using C99 initializers makes the code a bit more readable, and it
avoids some gcc -Wall warnings regarding missing initializers.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The bits between end and real_end are set as a safety measure for the
kernel when it uses the bit scan instructions. We need to take this
into account when shrinking or growing the block allocation bitmap,
before we can safely use rbtree bitmaps in resize2fs.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Fix a few warnings about unused and uninitialized variables.
Also fix util/subst.c to include <sys/time.h> to avoid using
undeclared functions gettimeofday() and futimes().
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
In the loop in ext2fs_get_free_blocks2, we ask the bitmap if there's a
range of free blocks starting at "b" and ending at "b + num - 1".
That quantity is the number of the last block in the range. Since
ext2fs_blocks_count() returns the number of blocks and not the number
of the last block in the filesystem, the check is incorrect.
Put in a shortcut to exit the loop if finish > start, because in that
case it's obvious that we don't need to reset to the beginning of the
FS to continue the search for blocks. This is needed to terminate the
loop because the broken test meant that b could get large enough to
equal finish, which would end the while loop.
The attached testcase shows that with the off by one error, it is
possible to throw e2fsck into an infinite loop while it tries to
find space for the inode table even though there's no space for one.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
It's only necessary to build tst_libext2fs when running "make check".
Also make sure the links of the tst_* programs are done with
$(ALL_LDFLAGS).
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Most systems have a backwards compatibility symlink in
/usr/include/syscall.h to /usr/include/sys/syscall.h, but
sys/syscall.h is the documented location of the header file. Fix two
locations where we were using <syscall.h> instead of <sys/syscall.h>.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
In mke2fs command, if flex_bg count is too large to filesystem blocks
count, unmountable ext4 which has the out of filesystem block offset
is created (Case1). Moreover this large flex_bg count causes an
unintentional metadata layout (bmap-imap-itable-bmap-imap-itable .. in
block group) (Case2).
To fix these issues and keep healthy flex_bg layout, disallow creating
ext4 with obviously large flex_bg count to filesystem blocks count.
Steps to reproduce:
(Case1)
1.
# mke2fs -t ext4 -b 4096 -O ^resize_inode -G $((2**20)) DEV 2130483
2.
# mount -t ext4 DEV MP
mount: wrong fs type, bad option, bad superblock on /dev/sdb4,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so
3.
# dumpe2fs DEV
...
Block count: 2130483
...
Flex block group size: 1048576
...
Group 65: (Blocks 2129920-2130482) [INODE_UNINIT]
Checksum 0x4cb3, unused inodes 8080
Block bitmap at 67 (bg #0 + 67), Inode bitmap at 1048643 (bg #32 + 67)
Inode table at 2129979-2130483 (+59)
^^^^^^^ 2130483 is out of FS!
65535 free blocks, 8080 free inodes, 0 directories, 8080 unused inodes
Free blocks:
Free inodes: 525201-533280
(Case2)
1.
# mke2fs -t ext4 -G 2147483648 DEV 3145728
2.
# debugfs -R stats DEV
...
Block count: 786432
...
Flex block group size: 2147483648
...
Group 0: block bitmap at 193, inode bitmap at 194, inode table at 195
...
Group 1: block bitmap at 707, inode bitmap at 708, inode table at 709
...
Group 2: block bitmap at 1221, inode bitmap at 1222, inode table at 1223
...
Signed-off-by: Akira Fujita <a-fujita@rs.jp.nec.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
If a large flex_bg factor is specified and the block allocator was
laying out block or inode bitmaps or inode tables, and collides with
previously allocated metadata (for example the backup superblock or
group descriptors) it would reset the allocator back to the beginning
of the flex_bg instead of continuing past the obstruction.
For example, with "-G 131072" the inode table will hit the backup
descriptors in groups 1, 3, 5, 7, 9 and start interleaving with the
block and inode bitmaps. That results in poorly allocated bitmaps
and inode tables that are interleaved and not contiguous as was
intended for flex_bg:
Group 0: (Blocks 0-32767)
Primary superblock at 0, Group descriptors at 1-2048
Block bitmap 2049 (+2049), Inode bitmap at 133121 (bg #4+2049)
Inode table 264193-264200 (bg #8+2049)
:
:
Group 3838: (Blocks 125763584-125796351) [INODE_UNINIT, BLOCK_UNINIT]
Block bitmap 5887 (bg #0+5887), Inode bitmap 136959 (bg #4+5887)
Inode table 294897-294904 (bg #8 + 32753)
Group 3839: (Blocks 125796352-125829119) [INODE_UNINIT, BLOCK_UNINIT]
Block bitmap 5888 (bg #0+5888), Inode bitmap 136960 (bg #4+5888)
Inode table 5889-5896 (bg #0 + 5889)
Group 3840: (Blocks 125829120-125861887) [INODE_UNINIT, BLOCK_UNINIT]
Block bitmap 5897 (bg #0+5897), Inode bitmap 136961 (bg #4+5889)
Inode table 5898-5905 (bg #0 + 5898)
:
:
Instead, skip the intervening blocks if there aren't too many of them.
That mostly keeps the flex_bg allocations from colliding, though still
not perfect because there is still some overlap with the backups.
This patch addresses the majority of the problem, allowing about 124k
groups to be layed out perfectly, instead of less than 4k groups with
the previous code.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Currently there are many uses of ext2fs_close() which might be wrong.
First of all ext2fs_close() does not set the ext2_filsys pointer to NULL
so the caller is responsible for clearing it, however there are some
cases there we do not do it.
Second of all very small number of users of ext2fs_close() actually
check the return value. If there is a problem in ext2fs_close() it will
not even free the ext2_filsys structure, but majority of users expect it
to do so.
To fix both problems this commit introduces a new helper
ext2fs_close_free() which will not only check for the return value and
free the ext2_filsys structure if the call to ext2fs_close2() failed,
but it will also set the ext2_filsys pointer to NULL.
Replace every use of ext2fs_close() in e2fsprogs tools with
ext2fs_close_free() - there is no real reason to keep using
ext2fs_close().
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Try to avoid name clashes with definitions of __u8, __u16, __u32,
and __u64 in userspace, in case other headers also define these
types. Define HAVE___{S,U}{8,16,32,64} preprocessor macros to
show that these types are already defined.
This would avoid the need to check for _BLKID_TYPES_H in ext2_types.h
and _EXT2_TYPES_H in blkid_types.h, but since older versions of these
headers did not use HAVE___U8 et.al. keep these checks around for now.
Report an error if there are no 64-bit types available. The code
will not compile if these are not available.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The LIST_HEAD macro is not directly used in getsize.c, so
<sys/queue.h> is not needed at all, and could cause confusion at
some later point if the Linux-style list macros are ever used.
Build was verified on MacOS which defined HAVE_SYS_DISK_H true.
I manually inspected the sources for recent *BSD headers to check
if this was needed there or not. MacOS and FreeBSD <sys/disk.h>
do not use lists at all. NetBSD and OpenBSD <sys/disk.h> and all
of the <sys/mount.h> headers include <sys/queue.h> internally.
I used http://fxr.watson.org/fxr/source/sys/mount.h?v={OSTYPE}
as a reference, checking both old and new *BSD versions.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Per http://www.gnu.org/software/checker/ the gcc "-checker" option
is long deprecated. Nuke it from e2fsprogs.
Most people would never hit this, but people who love to turn knobs,
such as the reporter of kernel.org bz#74171, might run into it and be
sad.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Quiet a couple of build warnings in tst_libext2fs.c
Add missing unistd.h header for misc/util.c.
Ignore generated files for lib/ext2fs/tst_libext2fs and intl/ files.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
This commit fixes two warning messages when compiling with LLVM.
Reported-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Lukas Czerner <lczerner@redhat.com>
Fix compile warnings found on the master branch when using LLVM.
- Add missing format string when using the libintl _() macro
- include <limits.h> header to get PATH_MAX definition
- fix format vs. variable mismatches
- add header block for create_inode.c file
- remove use of bzero(), use ext2fs_get_memzero() instead
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Fix various small resource leaks and error code handling issues that
Coverity pointed out.
Fixes-Coverity-Bugs: 1215250, 1193379, 119194[2-4], 1049160
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Fix various small resource leaks and error code handling issues that
Coverity pointed out.
Fixes-Coverity-Bugs: 11919{39-45}, 1174118, 1049160, 1049144
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Fix some minor bugs relating to passing CFLAGS to cppcheck, and
package the cppcheck output into nicer looking reports.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Adding the pkgconfigdir variable allows specifying an installation
location for pkg-config files independent of libdir.
Signed-off-by: David Michael <fedora.dm0@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If the previous block group's inode table ends at the very end of file
system, wrap around to the beginning of the flex_bg.
This fixes a bug was tickled by:
mke2fs.conf:
frontload = {
features = extent,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize,^resize_inode,sparse_super2
hash_alg = half_md4
num_backup_sb = 0
packed_meta_blocks = 1
inode_ratio = 4194304
flex_bg_size = 262144
}
mke2fs -T frontload /tmp/foo.img 2T
resize2fs -M /tmp/foo.img
resize2fs -M /tmp/foo.img
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Filefrag doesn't catch and print the shared extent flag. Add this for
users of filefrag on file systems with shared extents (such as btrfs).
Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Fix various unused variable and use-uninitialized warnings.
Add generated files into .gitignore.
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Fix the following build errors on bigendian hosts.
- ctx is a pointer, use '->' not '.'
- add missing argument to ext2fs_dirent_swab_in2
make[2]: Entering directory `/root/e2fsprogs/lib/ext2fs'
CC inline_data.c
inline_data.c: In function ‘ext2fs_inline_data_dir_iterate’:
inline_data.c:221:5: error: request for member ‘errcode’ in something not a structure or union
ctx.errcode = ext2fs_dirent_swab_in2(fs, ctx->buf, ctx->buflen, 0);
^
inline_data.c:222:9: error: request for member ‘errcode’ in something not a structure or union
if (ctx.errcode) {
^
inline_data.c: In function ‘ext2fs_inline_data_dir_expand’:
inline_data.c:364:2: error: too few arguments to function ‘ext2fs_dirent_swab_in2’
retval = ext2fs_dirent_swab_in2(fs, buf, size);
^
In file included from inline_data.c:19:0:
ext2fs.h:1569:18: note: declared here
extern errcode_t ext2fs_dirent_swab_in2(ext2_filsys fs, char *buf, size_t size,
^
make[2]: *** [inline_data.o] Error 1
Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Zheng Liu <wenqing.lz@taobao.com>
In ext2fs_extent_set_bmap() and ext2fs_punch_extent(), fix the parents
when altering either end of an extent so that the parent nodes reflect
the added mapping.
There's a slight complication to using fix_parents: if there are two
mappings to an lblk in the tree, the value of handle->path->curr can
point to either extent afterwards), which is documented in a comment.
Some additional color commentary from Darrick:
In the _set_bmap() case, I noticed that the "remapping last block in
extent" case would produce symptoms if we are trying to remap a
block from "extent" to "next_extent", and the two extents are
pointed to by different index nodes. _extent_replace(...,
next_extent) updates e_lblk in the leaf extent, but because there's
no _extent_fix_parents() call, the index nodes never get updated.
In the _punch_extent() case, we conclude that we need to split an
extent into two pieces since we're punching out the middle. If the
extent is the last extent in the block, the second extent will be
inserted into a new leaf node block. Without _fix_parents(), the
index node doesn't seem to get updated.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
In ext2fs_extent_free(), h(andle)->max_depth is used as a loop
conditional variable to free all the h->path[].buf pointers. However,
ext2fs_extent_delete() sets max_depth = 0 if we've removed everything
from the extent tree, which causes a subsequent _free() to leak some
buf pointers. max_depth can be re-incremented when splitting extent
nodes, but there's no guarantee that it'll reach the old value before
the free.
Therefore, remember the size of h->paths[] separately, and use that
when freeing the extent handle.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Fix a few minor bugs that cppcheck complained about.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
In ext2fs_block_alloc_stats_range(), the quantity "-inuse * n" is
calculated as a signed 32-bit quantity. Unfortunately, gcc (4.6.3 on
Ubuntu 12.04) doesn't sign-extend this quantity to fill the blk64_t
parameter that ext2fs_free_blocks_count_add() wants, so the end result
is that the superblock gets a ridiculously huge free block count.
Changing the declaration of 'n' to blk64_t seems to fix this.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Fix a number of things that cppcheck complains about. Most of these
are minor resource leaks and forgotten declarations.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
In ext2fs_inline_data_dir_iterate(), we must be very careful to undo
any modifications we make to the dir_context pointer passed in by the
caller, because it's entirely possible that the caller will still want
to do something with the ctx or something inside.
Specifically, ext2fs_dblist_dir_iterate() wants to be able to free
ctx->buf, and it reuses the ctx for multiple dblist entries. That
means that assigning ctx->buf will cause weird crashes at the end of
dir_iterate().
Since we're being careful with ctx, we might as well handle adding the
INLINE_DATA flag to ctx->flags for ext2fs_process_dir_block, since the
dblist caller forgets to unset the flag before reusing the ctx.
This fixes some crashes and valgrind complaints in resize2fs, and is
necessary for the next patch, which fixes resize2fs not to corrupt
inline_data filesystems.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
When expanding an inline data inode, it's possible that the reduction
in the size of the EA structures causes the freeing of the EA block,
which changes the inode. If this happens, the local version of the
inode that ext2fs_inline_data_expand was modifying will be out of sync
with what's on the disk. This local copy gets written out to disk
after a block allocation, at which point it's possible that the inode
EA block and logical block zero point to the same physical block,
which is bad news.
Therefore, write the local copy to disk before removing the inline
data EA, and reread it afterwards.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
i_blocks covers the number of blocks allocated to an inode for data,
extents, and ACL blocks. Since it's possible for a file to have a
separate ACL block and inline data, we must be careful when expanding
an inline data file to adjust, not set, the value of i_blocks.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
ext2fs_inline_data_set() tries to ensure that there is sufficient free
space in the inode to store the inline data. Unfortunately, it gets
the check wrong -- ext2fs_xattr_inode_max_size() returns the amount of
unused bytes in the EA area, and _data_set() doesn't factor in the
size of the existing inline data. Therefore, a strict rewrite of an
N-byte inlinedata with another N-byte inlinedata fails.
Fix the code to do the size check correctly.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Fix a typo that we didn't notice because all the world's an x86. :-)
Reported-by: jon ernst <jonernst07@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add magic number checking to the extended attribute editing handle;
move inline data to the head of the attribute list when writing so
that inline data ends up in the inode area; and always zero the
attribute space before writing to ensure that we can delete the last
xattr.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
In this unit test, we will test the interface of inline data and make
sure it is fine.
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Currently we have already exported inode cache flush and free functions
for users. This commit exports inode cache creation function. Later
we will use this function to initialize inode cache and do some unit
tests for inline data.
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Currently ext2fs_file_read/write are used to copy data from/to a file.
But they manipulate data by blocksize. For supporting inline data, we
handle it in two new fucntions called ext2fs_file_read/write_inline_data.
In read path the implementation is straightforward. But in write path
things get more complicated because if the size of data is greater than
the maximum size of inline data we will expand this file. So now we
will check this in ext2fs_inline_data_set. If this inode doesn't have
enough space, it will return EXT2_ET_INLINE_DATA_NO_SPACE error. Then
the caller will check this error and tries to expand the file.
The following commands in debugfs can handle inline_data feature after
applying this patch:
- dump
- cat
- rdump
- write
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Now punch command only can remove all inline data because now
punch command is based on block unit and the size of inline data is
never beyond a block size.
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This commit tries to make mkdir command in debugfs support inline data.
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This commit defines a ext2fs_inline_data_expand() to expand an inode with
inline data. In this commit this function only can expand a directory.
But later it will expand a file.
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If there is an inode with inline data, we just print the size of inline
data in stat command.
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
After applied this commit (a7f4c635), we have banned to traverse blocks
for an inode which has inline data because no block belongs to it. But
before calling this function, we need to check inline data flag. This
commit add a sanity check ext2fs_inode_has_valid_blocks2() to fix them
except that ext2fs_expand_dir because it will be fixed by another patch.
Meanwhile in this commit it fixes a bug that when we kill a file we
could leak an inode.
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Inline_data is handled in dir iterator because a lot of commands use
this function to traverse directory entries in debugfs. We need to
handle inline_data individually because inline_data is saved in two
places. One is in i_block, and another is in ibody extended attribute.
After applied this commit, the following commands in debugfs can
support the inline_data feature:
- cd
- chroot
- link*
- ls
- ncheck
- pwd
- unlink
* TODO: Inline_data doesn't expand to ibody extended attribute because
link command doesn't handle DIR_NO_SPACE error until now. But if we
have already expanded inline data to ibody ea area, link command can
occupy this space.
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Later we will use ext2fs_dirent_swab_in/out to handle big-endian problem
for inline data. Now interfaces assume that it handles a block, but it
is not true after adding inline data. So this commit defines a new
interface for inline data.
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Before loading extended attributes, free any key/value pairs that
might already be associated with the file.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add another API to query the number of extended attributes.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
A few tweaks to the extended attribute editing APIs:
* Use size_t, not unsigned int, in the new extended attribute editing
API.
* Don't expose the _expand() call since there should be no external
users.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add functions to allow clients to get, set, and remove extended
attributes from any file. It also supports modifying EAs living in
i_file_acl.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
To check the coverage of e2fsprogs's regression test, do the
following:
configure --enable-gcov
make -j8 ; make -j8 check ; make coverage.txt
The coverage information will be the coverage.txt and *.gcov files in
the build directories.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
ext2fs_free() does not set the ext2_filsys pointer to null so the
caller is responsible to setting it himself if it is needed.
This patch fixes some places where caller did not set ext2_filsys
pointer to NULL after ext2fs_free() which might result in use after
free. Fix it.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
After commit 62f17f3603, variable
"handle" has no use. So delete it.
Signed-off-by: Jon Enrst <jonernst07@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Declare struct_io_manager at the end of unix_io.c, undo_io.c, and
test_io.c files so that there isn't a need to forward declare every
member of this structure. That avoids a lot of redundant code
at the start of every one of these files.
Move the test_flush() function above test_abort() to avoid the need
for a forward declaration.
Fix a few instances of space before tab in these files.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Fix a number of non-literal string format warnings from LLVM due
to the use of _() that were not fixed in commit 45ff69ffeb.
Fix mismatched int vs. __u64 format warnings in blkmap64_rb.c.
There were also some comparisons of __u64 start or count <= 0.
Change them to be comparisons == 0, or start + count overflow.
Fix operator precedence warning for (value & (value - 1) != 0)
introduced in 11d1116a7c. It seems "&" is lower precedence
than "!=", so the above didn't fail for power-of-two values,
but only odd values. Fortunately, either s_desc_size nor
s_inode_size is valid if odd.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If a client asks us to remap a block in the middle of an extent, we
potentially have to allocate a fair number of blocks to handle extent
tree splits. A failure in either of the ext2fs_extent_insert calls
leaves us with an extent tree that no longer maps the logical block in
question and everything that came after it! Therefore, try to roll
back the extent tree changes before returning an error code.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If we're doing a BMAP_ALLOC allocation and the extent tree update
fails, there's no point in hanging on to the newly allocated block.
So, free it to make fsck happy.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
When modifying/removing an extent during punch, don't forget to update
the extent's parents.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
When we're iterating extents during a punch operation, the loop exits
if the punch region is entirely to the right of the extent we're
looking at. This can happen if the punch region starts in the middle
of a hole and covers mapped extents. When this happens, we want to
skip to the next extent, because it might be punchable.
Also, if we've totally passed the punch range, stop.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Also use angle brackets for the #include of dirpaths.h to avoid the
need to manually massage the Makefile.in for the util directory. This
is needed because we have to create a fake dirpaths.h file in the util
directory. The fake dirpaths.h file is rquired to break the circular
dependency caused by util/subst creating dirpaths.h, while
util/subst.c is including config.h, which includes dirpaths.h.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
In practice, it is **extremely** rare for users to try to use more
than the first backup superblock located at the beginning of block
group #1. (i.e., at block number 32768 for file systems with a 4k
block size). This new compat feature restricts the backup superblock
to block group #1 and the last block group in the file system.
Aside from reducing the overhead of the file system by a small number
of blocks, by eliminating the rest of the backup superblocks, it
allows us to have a much more flexible metadata layout. For example,
we can force all of the allocation bitmaps and inode table blocks to
the beginning of the disk, which allows most of the disk to be
exclusively used for contiguous data blocks.
This simplifies taking advantage of certain HDD specific features,
such as Shingled Magnetic Recording (aka Shingled Drives), and the
TCG's OPAL Storage Specification where having a simple mapping between
LBA block ranges and the data blocks used by the file system can make
life much simpler.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If there are hundreds of thousands of blocks which are in use before
the first free block, it is much, MUCH faster to use
ext2fs_find_first_zero_block_bitmap2() instead of searching the
allocation bitmap bit by bit.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
By using ext2fs_mark_block_bitmap_range2 and/or
ext2fs_block_alloc_stats_range(), we can significantly speed up the
time needed by mke2fs to allocate the inode table.
For example, the CPU time needed to run the command "mke2fs -t ext4
/tmp/foo.img 32T" (where tmpfs was mounted on /tmp) was decreased from
21.7 CPU seconds down to under 1.7 seconds.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This function is more efficient than using ext2fs_block_alloc_stats2()
for each block in a range. The efficiencies come from being able to
set a block range in the block bitmap at once, and from being update
the block group descriptors once per block group. Especially now that
we are checksuming the block group descriptors, and we are using red
black trees for the allocation bitmaps, these changes can make a huge
difference in the CPU time used by mke2fs when creating very large
file systems.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Commit 8e44eb64bb (libext2fs: mark group data blocks when loading
block bitmap) simplified check_block_uninit since we are now
initializing the bitmap when it is loaded from disk. It left some
variables which were being set but never used, however. In addition,
since we only need check_block_uninit() to clear the block bitmap's
uninit flag, rename it to clear_block_uninit(), and only call it once
we have found a free block in ext2fs_new_blocks2().
This cleans up the code some and optimizes things if we need to search
multiple block groups trying to find a free block.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: Darrick J. Wong <darrick.wong@oracle.com>
When building tst_bitmaps, enable #define DEBUG_RB, so we are
always testing the sanity of the in-memory representation of the
bitmap when using red-black trees as part of a "make check" run.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Move the error checking into the the generic bitmap code, and add
support for bitmaps with cluster_bits set.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
When inserting the first extent into an empty inode, the
ext2fs_extent_insert() leaves path->left set to 1 instead of 0. Since
path->curr is pointing at the last (only) extent in the file,
path->left should be 0.
This is mostly harmless, and gets corrected fairly quickly if the
calling applicaton jumps to a different part of the extent tree ---
for example, by calling ext2fs_extent_goto(), or calling
ext2fs_extent_get with the flags argument set to EXT2_EXTENT_ROOT.
Which is why we hadn't noticed this problem until now.
However, if you insert four extents using ext2fs_extent_insert, the
fourth insert will end up copying too many bytes in the i_block[]
array, since path->left is one larger than it should be. This results
in the inode fields i_generation, i_file_acl, and i_size_high getting
zeroed out.
This problem can be replicated as follows:
% cp /dev/null /tmp/foo.img
% mke2fs -F -t ext4 /tmp/foo.img 100
% debugfs -w /tmp/foo.img
debugfs: write /dev/null foo
debugfs: set_inode_field foo i_size_hi 1
debugfs: stat foo
<----- note that the inode's size is 4294967296
debugfs: extent_open foo
debugfs (extent ino 12): insert --after 0 1 100
debugfs (extent ino 12): insert --after 1 1 101
debugfs (extent ino 12): insert --after 2 1 102
debugfs (extent ino 12): insert --after 3 1 103
debugfs (extent ino 12): extent_close
debugfs: stat foo
<----- note that the inode's size is now 0
debugfs: quit
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add functions which try to find the first set block or inode in a
bitmap. This is useful when trying to allocate a range of blocks
efficiently.
Like the find_first_zero family of functions, provide a generic O(N)
search function which will be used if there is no optimized version
provided by the red-black tree or bitarray functions.
Also, expand the test cases for ext2fs_find_first_zero_*() functions.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Since the beginning of the uninit_bg feature, the kernel[1] and
e2fsck[2] have always been careful to detect the presence of the
BLOCK_UNINIT flag, and compute a block bitmap with any group metadata
blocks marked in that bitmap. With that in mind, I think it's safe to
say that this is a design feature of uninit_bg.
Now that we've trained libext2fs to have this same behavior whenever
it's loading a block bitmap, we no longer need to unset BLOCK_UNINIT
for a group that contains only its own group metadata -- kernel,
e2fsck, and e2fsprogs will handle this correctly.
[1] kernel git 717d50e4971b81b96c0199c91cdf0039a8cb181a
"Ext4: Uninitialized Block Groups"
[2] e2fsprogs git f5fa20078b
"Add support for EXT2_FEATURE_COMPAT_LAZY_BG"
Reported-by: Akira Fujita <a-fujita@rs.jp.nec.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The kernel[1] and e2fsck[2] both react to a BLOCK_UNINIT group by
calculating the block bitmap that's needed to show all the group
blocks for that group (if any) and using that. However, when reading
bitmaps from disk, libext2fs simply imports a block of zeroes into the
bitmap, without bothering to check for group blocks. This erroneous
behavior results in the filesystem having a block bitmap that does not
accurately reflect disk contents, and worse yet makes it seem as
though superblocks, group descriptors, bitmaps, and inode tables are
"free" space on disk.
So, fix the block bitmap loading routines to calculate the correct
block bitmap for all groups and load it into the main fs block bitmap.
This also fixes bogus debugfs output such as:
Group 1: (Blocks 8193-16384) [INODE_UNINIT, BLOCK_UNINIT]
Checksum 0x1310, unused inodes 512
Backup superblock at 8193, Group descriptors at 8194-8217
Reserved GDT blocks at 8218-8473
Block bitmap at 283 (bg #0 + 282), Inode bitmap at 299 (bg #0 + 298)
Inode table at 442-569 (bg #0 + 441)
7911 free blocks, 512 free inodes, 0 directories, 512 unused inodes
Free blocks: 8193-16384
Free inodes: 513-1024
Notice how the "free blocks" range includes the backup sb & GDT area
and doesn't match the free block count.
Worse yet, debugfs' testb command will report those group descriptor
blocks as not being in use unless the user also instructs debugfs to
find a free block first. That is a rather surprising result:
debugfs: testb 8194
Block 8194 not in use
debugfs: ffb 1 16380
Free blocks found: 16380
debugfs: testb 8194
Block 8194 marked in use
Also, remove the part of check_block_uninit() that "fixes" the bitmap
since we're doing that at bitmap load time now.
[1] kernel git 717d50e4971b81b96c0199c91cdf0039a8cb181a
"Ext4: Uninitialized Block Groups"
[2] e2fsprogs git f5fa20078b
"Add support for EXT2_FEATURE_COMPAT_LAZY_BG"
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
On a filesystem with 1K blocks and meta_bg enabled, opening a
filesystem with automatic superblock detection tries to compensate for
the fact that the superblock lives in block 1. However, the method by
which this is done is later misinterpreted to mean "read the backup
group descriptors", which is not what we want in this case.
Therefore, in ext2fs_open3() separate the 'group zero' adjustment into
its own variable so that we don't get fed backup group descriptors
when we try to load meta_bg group descriptors.
Furthermore, enhance ext2fs_descriptor_block_loc2() to perform its own
group zero correction. The other caller of this function neglects to
do any group-zero correction of their own, so this fixes them too.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If ext2fs_descriptor_block_loc2() is called with a meta_bg filesystem
and group_block is not the normal value, the function will return the
location of the backup group descriptor block in the next block group.
Unfortunately, it fails to account for the possibility that the backup
group contains a backup superblock but the regular superblock does
not. This is the case with block groups 48-49 on a meta_bg fs with 1k
blocks; in this case, libext2fs will fail to open the filesystem.
Therefore, teach the function to adjust for superblocks in the backup
group, if necessary.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The races would be hard to exploit, but let's close them off.
Addresses-Coverity-Id: #709504
Addresses-Coverity-Id: #709505
Addresses-Coverity-Id: #709506
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Some different types such as u_int16_t and __uint32_t have snuck into
e2fsprogs. These types are not guaranteed by any standard, and they
are not provided by dietlibc. Convert them to __u16, __u32,
etc. since these are guaranteed to be provided by e2fsprogs' build.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Newer versions of autoconf pull in AC_PROG_GCC as part of
AC_CANONICAL_HOST. So we need check for WITH_DIET_LIBC earlier in
configure.in.
Also, e2fsprogs now needs functions which are found in diet libc's
compat library. So add support for autoconf's LIBS function, and
automatically set libs to include -lcompat.
Finally, disable compiling e4defrag by deault if --with-diet-libc is
specified because the program has too many glibc dependencies.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The e2image progam was originally intended to create image files.
However, some people have started using e2image to copy a file system
from one block device to another, since it is more efficient than
using dd because it only copies the blocks which are in use. If we
are doing this, however, we must not skip writing blocks which are all
zero in the source device, since they may not be zero in the
destination device.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: Phillip Susi <psusi@ubuntu.com>
Currently the ext4 block group descriptor is 64 bytes. In case we
need to support larger block group descriptors in the future, teach
ext2fs_group_desc_csum() to checksum parts of the block group
descriptors that libext2fs doesn't yet understand.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The s_desc_size in the superblock specifies the group descriptor
size in bytes, but in various places the EXT4_FEATURE_INCOMPAT_64BIT
flag implies that the descriptor size is EXT2_MIN_DESC_SIZE_64BIT
(64 bytes) instead of checking the actual size. In other places,
the s_desc_size field is used without checking for INCOMPAT_64BIT.
In the case of ext2fs_group_desc() the s_desc_size was being ignored,
and assumed to be sizeof(struct ext4_group_desc), which would result
in garbage for any but the first group descriptor. Similarly, in
ext2fs_group_desc_csum() and print_csum() they assumed that the
maximum group descriptor size was sizeof(struct ext4_group_desc).
Fix these functions to use the actual superblock s_desc_size if
INCOMPAT_64BIT.
Conversely, in ext2fs_swap_group_desc2() s_desc_size was used
without checking for INCOMPAT_64BIT being set.
The e2fsprogs behaviour is different than that of the kernel,
which always checks INCOMPAT_64BIT, and only uses s_desc_size to
determine the offset of group descriptors and what range of bytes
to checksum.
Allow specifying the s_desc_size field at mke2fs time with the
"-E desc_size=NNN" option. Allow a power-of-two s_desc_size
value up to s_blocksize if INCOMPAT_64BIT is specified. This
is not expected to be used by regular users at this time, so it
is not currently documented in the mke2fs usage or man page.
Add m_desc_size_128, f_desc_size_128, and f_desc_bad test cases to
verify mke2fs and e2fsck handling of larger group descriptor sizes.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add a LOG2_CHECK mode for check_super_value() so that it is easy
to verify values that are supposed to be power-of-two values
(s_desc_size and s_inode_size so far). In ext2fs_check_desc()
also check for a power-of-two s_desc_size.
Print out s_desc_size in debugfs "stats" and dumpe2fs output, if
it is non-zero.
It turns out that the s_desc_size validation in check_super_block()
is not currently used by e2fsck, because the group descriptors are
verified earlier by ext2fs_check_desc(), and even without an
explicit check of s_desc_size the group descriptors fail to align
correctly on disk. It makes sense to keep the check_super_block()
regardless, in case the code changes at some point in the future.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Mostly by adding static and removing excess extern qualifiers. Also
convert a few remaining non-ANSI function declarations to ANSI.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
When /etc/mtab is a symlink of /proc/mounts, mke2fs without -FF option
can create a filesystem on the image file that is mounted.
According to mke2fs man page, we should specify -FF option in this case.
This patch protects filesystem from unintended mke2fs caused by human error.
How to reproduce:
# mke2fs -t ext4 -Fq fs.img
# mount -o loop fs.img /mnt/mp1
# mke2fs -t ext4 -Fq fs.img && echo "mke2fs success"
mke2fs success
Signed-off-by: Kazuya Mio <k-mio@sx.jp.nec.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
When we're appending a block to a directory file or the journal file,
and the new block is part of a cluster that has already been allocated
to the file (implied cluster allocation), don't update the bitmap or
the summary counts because that was performed when the cluster was
allocated.
Reviewed-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
When bigalloc is enabled, using ext2fs_block_alloc_stats2() to free
any block in a cluster has the effect of freeing the entire cluster.
This is problematic if a caller instructs us to punch, say, blocks
12-15 of a 16-block cluster, because blocks 0-11 now point to a "free"
cluster.
The naive way to solve this problem is to see if any of the other
blocks in this logical cluster map to a physical cluster. If so, then
we know that the cluster is still in use and it mustn't be freed.
Otherwise, we are punching the last mapped block in this cluster, so
we can free the cluster.
The implementation given only does the rigorous checks for the partial
clusters at the beginning and end of the punching range.
Reviewed-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The LIST_HEAD() macro conflicts with the <sys/queue.h> declaration
of the same name. Delete the unused LIST_HEAD() macro from the
libext2fs and libblkid headers to avoid compiler warnings. It can
be replaced by INIT_LIST_HEAD() or LIST_HEAD_INIT() if needed.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Include ext2fsP.h in fileio.c for ext2fs_file_block_offset_too_big()
declaration. Fix up the declaration to mark it extern in the header.
Include <strings.h> header for strcasecmp() in tune2fs.c if available,
as described in the strcasecmp(3) man page, instead of doing this
indirectly by declaring _BSD_SOURCE and getting it from <string.h>.
If CONFIG_QUOTA is undefined, parse_quota_opts() is unused in
tune2fs.c so #ifdef it out.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Don't print a verbose configure error in parse-types.h if
<asm/types.h> missing and __[SU]*_TYPEDEF is unset. This is
always the case for non-Linux builds.
The printf formatting strings all use "%llu" for printing 64-bit
values and this it produces a large number of warnings if __u64
is defined as "unsigned long". If __U64_TYPEDEF isn't set use
"unsigned long long" for __u64 in ext2-types.h and blkid-types.h
by default instead of using "unsigned long".
Fix a few places where "%d" or "%u" or "%Lu" were used to print a
64-bit value, by converting them to use "%lld" or "%llu" instead.
Fix a few places where "%lu" was used to print .tv_usec, by casting
the variable to "(long)" since .tv_usec is "int" on some systems.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
In the ^extent case, passing ~0ULL as the 'end' parameter to
ext2fs_punch() causes the (end - start + 1) calculation to overflow to
zero. Since the old-style mapped block files cannot have more than
2^32 blocks, just clamp it to ~0U.
This fixes a regression in t_quota_2off with the patch "libext2fs: use
ext2fs_punch() to truncate quota file" applied.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Tweak the wording to be a little less ambiguous, since 'block' can be
a noun or a verb.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
When we set the file size, find the block containing EOF, and zero
everything in that block past EOF so that we can't return stale data
if we ever use fallocate or truncate to lengthen the file.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If we're asked to punch a file with no data blocks mapped to it and a
non-zero length, we don't need to do any work in ext2fs_punch_extent()
and can return success. Unfortunately, the extent_get() function
returns "no current node" because it (correctly) failed to find any
extents, which is bubbled up to callers. Since no extents being found
is not an error in this corner case, fix up ext2fs_punch_extent() to
return 0 to callers.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
When deleting an entire extent, we cannot always slip to the previous
leaf extent because there might not /be/ a previous extent.
Attempting to correct for that error by asking for the 'current' leaf
extent also doesn't work, because the failed attempt to change to the
previous extent leaves us with no current extent.
Fix this problem by recording the lblk of the next extent before
deleting the current extent and _goto()ing to the next extent after
the deletion.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If we're using ext2fs_file_write() to write to a hole in a file,
ensure that we can actually allocate the block before updating i_size.
In other words, don't update i_size and don't return success if we hit
an error while allocating space.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong