The read-modify-write code for the unaligned fallback code wasn't
working for multi-block writes. This was unmasked by FreeBSD 11-rc2,
since its malloc() is returning unaligned memory regions for large
memory regions.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
We need to prevent unaligned accesses, so treat any extra_isize which
is not a multiple of four as an bug.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
On platforms that don't permit unaligned pointer dereferences,
corrupted file systems will as used by the regression test suite can
cause e2fsck and debugfs to crash. Avoid those crashes caused by
corrupted file systems. With this commit the full set of regression
test suites will pass on the sparc64 platform.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The mk_cmds and compile_et scripts include the build directory, which
breaks the build reproducibility goal of Debian.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Recent research has shown that for a metadata-heavy workload, a 128 MB
is journal be a bottleneck on HDD's, and that the optimal journal size
is proportional to number of unique metadata blocks that can be
modified (and written into the journal) in a 30 second window. One
gigabyte should be sufficient for most workloads, which will be used
for file systems larger than 128 gigabytes.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
This new manager is similar to the unix_io_manager except it takes a
file descriptor as first argument instead of a filename.
Some programs may want libext2fs to directly use a fd instead of
letting it opening the file.
The use case for such a io_manager would be to let programs use
a fd even if the filename is unknown:
- the fd comes from a temporary file (O_TMPFILE);
- the fd comes from a unix socket...
Refactoring unix_open() also fix a bug when the IO_DIRECT flag was
specified: ext2fs_get_dio_alignment() was called before the file was
actually opened, resulting in an alignment of 0.
Signed-off-by: Adrien Schildknecht <adriens@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
When a ext2_filsys is freed, ext2fs_zero_blocks2() frees the buffer but
does not reset its size.
If this function is later called with a new ext2_filsys, the code
assumes that the buffer is still valid and return a NULL pointer.
Valgrind output:
==188948== Syscall param pwrite64(buf) points to unaddressable byte(s)
==188948== at 0x4E46D03: __pwrite_nocancel (syscall-template.S:81)
==188948== by 0x1254EC: raw_write_blk (unix_io.c:240)
==188948== by 0x124AAB: unix_write_blk64 (unix_io.c:850)
==188948== by 0x122893: ext2fs_zero_blocks2 (mkjournal.c:204)
Signed-off-by: Adrien Schildknecht <adriens@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Two new e2undo issues exist in the latest release on big endian
machines.
From sparse check:
undo_io.c:157:26: warning: invalid assignment: |=
undo_io.c:157:26: left side has type restricted __le32
undo_io.c:157:26: right side has type int
undo_io.c:161:26: warning: invalid assignment: &=
undo_io.c:161:26: left side has type restricted __le32
undo_io.c:161:26: right side has type int
e2undo.c:211:16: warning: cast to restricted __le64
e2undo.c:211:16: warning: cast from restricted blk64_t
e2undo.c:212:16: warning: cast to restricted __le64
e2undo.c:212:16: warning: cast from restricted blk64_t
Addresses-RedHat-Bugzilla: 1344636
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If there is a feature check, we can just depend on the feature check.
If it is something that can't be checked via a feature flag, then
instead of checking for EXT2_OS_LINUX, we should instead check for
*NOT* EXT2_OS_HURD. since HURD is the special case.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The debugging environment variables EXT2FS_PRETEND_* were implemented
in check_mntent_file(), and this function isn't called on all
operating systems. Lift this code up to ext2fs_check_mount_point(),
so that these environment variables (which are used in the regression
test suite) will work on all platforms.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The tst_* progams aren't intended to be linked 100% stically; they
just link against some of the static libraries. So use $(ALL_LDFLAGS)
and not $(LDFLAGS_STATIC).
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
There is a bug in Unix I/O manager where if an aligned I/O is
required, it does not correctly do the read-modify-write cycle
correctly. Specifically, it was not doing an lseek between the read
and the write calls, so the update was going to block N+1 instead of
block N. Oops.
Fortunately in practice we almost never use this fallback path, so
file systems weren't getting horribly corrupted, because (a) we almost
never use Direct I/O in e2fsprogs, at least not by default, and (b)
when we do the buffers end up being aligned anyway, so it's OK.
We only noticed this because the new Undo I/O manager in e2fsprogs
1.43 was doing unaligned I/O and FreeBSD requires that I/O requests be
aligned even if you are not doing Direct I/O, and the e2undo
regression tests were all failing as a result.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Make sure we use "-fPIC -shared" consistently on linker and compiler
command lines when building or linking the shared objects for the ELF
shared library.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The scratch_files feature is not really needed except on 32-bit
platforms, since tdb's performance is pretty awful given how we are
using it. Maybe SQLite would be faster, but for 64-bit platforms,
enabling swap works fairly well, especially using the rbtree for the
bitmap abstraction.
We leave tdb for Android since it's unlikely that someone will be
trying to connect petabyte+ sized file systems to a mobile handset.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If the journal superblock is corrupt and the user declines to fix it
(or runs e2fsck -n), make sure the error messages are clear and
explain that e2fsck cannot (safely) proceed.
Addresses-Debian-Bug: #768162
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Enable the following security features: stack protection, fortify,
read-only relocation tables, immediate dynamic symbol binding, and
text segment ASLR by enabling position independent executable
(PIE).
Special handling is provided for shared library and statically linked
executables. For all the gory details please see:
https://lists.debian.org/debian-devel/2016/05/msg00302.html
Distributions who want to do their own special thing can set CFLAGS,
CFLAGS_SHLIB, CLFAGS_STLIB, LDFLAGS, LDFLAGS_SHLIB and LDFLAGS_STATIC
as appropriate.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Use a large_inode so that when e2fsck is fixing a file system with
project quota enabled, the correct project id's quota is adjusted when
a corrupted inode is deleted.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Some operating systems may not define EBADMSG and EUCLEAN, so for
better portability use our own private error code numbers.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The fs_offset entry stores the filesystem offset. This allows for an
easy undo, because one does not have to remember/specify the
filesystem offset manually.
The fs_offset entry is implemented as a compatible feature.
Signed-off-by: Marcus Huewe <suse-tux@gmx.de>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Support key extension if the tdb_data_size is an arbitrary integer
multiple of the channel's block size. Before, key extension was only
possible if the tdb_data_size and the channel's block size were
equal.
Note: a key, whose data is the result of a short read, will be
extended if the tdb_data_size and the channel's block size are equal
(that's what the old code did) (if tdb_data_size is an arbitrary
integer multiple (> 1) of the channel's block size, the key might
be extended as well (depending on the keysize)).
Signed-off-by: Marcus Huewe <suse-tux@gmx.de>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The old code has some issues, for example, when backing up fs block 0
(can be reproduced via "mke2fs -z undo -b 1024 -E offset=1024 out 1024"):
* backing_blk_num is set to ULLONG_MAX instead of 0
* data is read from the beginning of the file instead of offset 1024
* data_ptr is set to read_ptr - 1024 ("invalid" address)
Hence, the wrong fs block is associated with the wrong data.
For details, see also commit 76da764639cbfcc998f13c263a11a4601bcb9961.
Signed-off-by: Marcus Huewe <suse-tux@gmx.de>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Avoid a potential out-of-bounds memory access if the group passed to
ext2fs_clear_block_uninit() or ext2fs_clear_inode_uninit() is greater
than the number of groups in the file system. This prevents a failure
in resize2fs when to allocate a block when growing the file system
significantly.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Hurd and *BSD is not going to have FS_IOC_FIEMAP, at least not at
Linux's codepoint.
Addresses-Debian-Bug: #822576
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Fix resize2fs so that the location of the backup superblocks when the
sparse_super2 feature is enabled is properly set when growing the file
system from a single block group to larger file system sizes.
Also fix a bug where the block group summary statistics in some cases
when exapnding a sparse_super2 file system.
Finally, accurately calculate the file system metadata overhead of the
last block group in sparse_super2 file systems.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Added offset support to the following functions:
- unix_cache_readahead
- unix_discard
- unix_zeroout
Signed-off-by: Marcus Huewe <suse-tux@gmx.de>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
After
http://git.kernel.org/cgit/fs/ext2/e2fsprogs.git/commit/lib/ext2fs/llseek.c?id=274d46e1d35af423d0292d63c4d0ad7a03be82ba
with
__linux__
defined(HAVE_LSEEK64) && defined(HAVE_LSEEK64_PROTOTYPE)
SIZEOF_OFF_T >= SIZEOF_LONG_LONG
it leads to ext2fs_llseek() doing a "return lseek(fd, offset, origin);"
Which fails for offsets > 32bit.
Also, with
__linux__
!(defined(HAVE_LSEEK64) && defined(HAVE_LSEEK64_PROTOTYPE))
defined(HAVE_LLSEEK)
SIZEOF_OFF_T == SIZEOF_LONG_LONG
my_llseek is not defined at all. And there is no need to define
llseek as lseek, as llseek is never used.
Luckily ext2fs_llseek() then does "return lseek(...);"
It would seem that my_llseek should be used in both places.
Addresses-Google-Bug: #13340735
Change-Id: Ie7330300c9c1ca103eaaef97536dcf10adbbba02
Signed-off-by: JP Abgrall <jpa@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
FreeBSD doesn't have libintl support built into the libc, and
libsupport requires libintl support. So we need to make sure it is
available when we link against the libsupport library.
Also, work around *BSD's interesting interpretation of how
_XOPEN_SOURCE is supposed to work.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The major() cpp macro is defined as requiring sys/types.h to be
included with _BSD_SOURCE defined. However, in older glibc's this
hasn't been strictly required and the stdlib.h header file included
sys/types.h implicitly. Fix this so that more aggressive
distributions run into build errors.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
in with andchanges. Lines starting
Glibc has depcreated _SVID_SOURCE and _BSD_SOURCE in favor of
_DEFAULT_SOURCE. So define _DEFAULT_SOURCE to shut up glibc 2.20
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Most libraries use the "-host" suffix when building for the host. This
patch renames all the libraries to use -host instead of _host.
Addresses-Google-Bug: #24619596
TEST=make dist
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
When adding 'libext2fs' as a library dependency from any other binary,
the binary needs to find the library headers. Users of this library
should not hard-code the include path themselves. This patch exports the
library headers on all three version of the libext2fs library and
removes the redundant include path from executables in e2fsprogs.
Addresses-Google-Bug: #23084776
TEST=mma
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Older kernels on 64-bit machines would incorrectly encode pre-1970
ext4 dates as post-2311 dates. Detect and correct this (assuming the
current date is before 2242).
Include tests for this, as well as changes to debugfs to correctly
set crtimes.
Signed-off-by: David Turner <novalis@novalis.org>
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If resize2fs_get_alloc_block() allocates from a BLOCK_UNINIT group, we
need to make sure that the UNINIT flag is cleared on both file system
structures which are maintained by resize2fs. This causes the
modified bitmaps to not get written out, which leads to post-resize2fs
e2fsck errors; used blocks in UNINIT groups, not marked in the block
bitmap. This was seen on r_ext4_small_bg.
This patch uses clear_block_uninit() to clear the flag,
and my problem goes away.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Replace the EXT2FS_RB_EMPTY_ROOT(), EXT2FS_RB_EMPTY_NODE(),
EXT2FS_RB_CLEAR_NODE(), IS_BLOCK_BM(), IS_INODE_BM(), and
IS_INODE_TB() macros with static inline functions to avoid
suprious compiler warnings with clang:
pass1.c:618:28: warning: equality comparison with extraneous
parentheses [-Wparentheses-equality]
if ((*((__u32 *)(entry)) == 0UL)) {
~~~~~~~~~~~~~~~~~~~~^~~~~~
pass1.c:618:28: note: remove extraneous parentheses
around the comparison to silence this warning
if ((*((__u32 *)(entry)) == 0UL)) {
^~
pass1.c:618:28: note: use '=' to turn this equality
comparison into an assignment
if ((*((__u32 *)(entry)) == 0UL)) {
^~
=
The static inline functions should compile identically, and allow
some extra compile-time checking for the arguments over macros.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Vitaliy Filippov
Theodore Ts'o