This patch instruments the libext2fs unix I/O manager and adds bytes
read/written and data rate to e2fsck -tt pass/overall timing output.
Signed-off-by: Jim Garlick <garlick@llnl.gov>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Create new functions ext2fs_{set,get}_{inode,block}_bitmap_range()
which allow programs like e2fsck, dumpe2fs, etc. to get and set chunks
of the bitmap at a time.
Move the representation details of the 32-bit old-style bitmaps into
gen_bitmap.c.
Change calls in dumpe2fs, mke2s, et. al to use the new abstractions.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Move the 32-bit specific bitmap code into gen_bitmap.c, and the
high-level interfaces into bitmaps.c. Eventually we'll move the
new-style bitmap code into gen_bitmap64.c, but first we need to
isolate the code with knowledge of the bitmap internals in one place
first.
In this patch we move allocation, free, copy, clear, set_padding, and
fudge_end function into gen_bitmap.c, and make sure that the bitmaps.c
and bitops.c no longer have any knowledge of the bitmap internals.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This changes ext2fs_fast_{mark,unmark,test}_{inode,block}_bitmap() to
be inline functions which calls ext2fs_{mark,unmark,test}_generic_bitmap().
This is part of the preparation to support the new-style bitmaps.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The test in ext2fs_check_desc() is off by one; if the inode table
goes all the way to the last block of the block group, it will
falsely assert that it has extended past it. The last block
of a range is start + len -1, not start + len.
You can create (valid) filesystems that will cause e2fsck to complain
via one of the following mkfs commands:
mkfs.ext3 -F -b 1024 /dev/sdb1 2046000000
mke2fs -j -F -b 4096 -m 0 -N 5217280 /mnt/test/fsfile2 327680
mkfs.ext2 -F -b 1024 -m 0 -g 256 -N 3744 fsfile 1024
Addresses-Red-Hat-Bugzilla: #214765
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
For some odd geometries*, mkfs will try to allocate inode tables off
the end of the block group and fail, rather than warning that too
many inodes have been requested.
This is because when ext2fs_initialize calculates metadata overhead,
it is only adding in group descriptor blocks and the superblock
if the *last* bg contains them - but the first bg also has all of
the various metadata bits taking up space.
We need to calculate the overhead both for the first block group and
the last block groups separately, since the two different tests need
to know what the overheads are for those two cases, which may be
different.
*for example "mke2fs -b 1024 -m 0 -g 256 -N 3745 fsfile 1024"
(Note, the test here is a little funky; the expected output is
actually a mkfs failure - but a proper failure instead of the
allocator catching the problem at the last minute)
Addresses-Red-Hat-Bugzilla: #241767
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
We need to set t->i_file_acl before we test it in
ext2fs_inode_data_blocks()
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Now that we are moving to x.y.z version number scheme for maintenance
releases, we ned to change ext2fs_parse_version_string and
blkid_parse_version_string to ignore the second period so we don't
have maintenance releases with a substantially bigger verison number
than the initial x.y release.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
On big-endian systems, while swapping, ext2fs_swap_inode_full() swaps
only 128+extra_isize bytes and the EAs if they are present. Now if inode
N has EAs, (and this is the inode in the "scratch inode") then inode N+1
also carries seems to have them since the "scratch inode" was never
zeroed.
Signed-off-by: Kalpak Shah <kalpak@clusterfs.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This patch changes ext2fs_open() to set EXT2_FLAG_MASTER_SB_ONLY by
default. This avoids some problems in e2fsck (reported by Jim Garlick)
where a corrupt journal can end up writing the bad superblock to the
backups. In general, only e2fsck (after the filesystem is clean),
tune2fs, and resize2fs should change the backup superblocks by default.
Most callers of ext2fs_open() should not be touching anything where the
backups should be touched. So let's change the defaults to avoid
potential problems.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
There have been reported instances of a filesystem having been mounted
at 2 places at the same time causing a lot of damage to the
filesystem. This patch reserves superblock fields and an INCOMPAT flag
for adding multiple mount protection(MMP) support within the ext4
filesystem itself. The superblock will have a block number
(s_mmp_block) which will hold a MMP structure which has a sequence
number which will be periodically updated every 5 seconds by a mounted
filesystem. Whenever a filesystem will be mounted it will wait for
s_mmp_interval seconds to make sure that the MMP sequence does not
change. To further make sure, we write a random sequence number into
the MMP block and wait for another s_mmp_interval secs. If the
sequence no. doesn't change then the mount will succeed. In case of
failure, the nodename, bdevname and the time at which the MMP block
was last updated will be displayed. tune2fs can be used to set
s_mmp_interval as desired.
Signed-off-by: Andreas Dilger <adilger@clusterfs.com>
Signed-off-by: Kalpak Shah <kalpak@clusterfs.com>
Store the RAID stride value when a filesystem is created with a requested
RAID stride, and then use it automatically in resize2fs.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Mke2fs is supposed to set the uid/gid ownership of the root directory when
a non-rooot user creates the filesystem. This wasn't working correctly
if the uid/gid was > 16 bits. In additional, debugfs wasn't displaying
large uid/gid's correctly. This patch fixes these two programs.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The l_i_version field is now defined from the old l_i_reserved1 field in
the ext2 inode. This field will be used to store high 32 bits of the
64-bit inode version number.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Fix a problem byte-swapping fast symlinks inodes that contain extended
attributes.
Addresses Red Hat Bugzilla: #232663
Addresses LTC Bugzilla: #27634
Signed-off-by: "Bryn M. Reeves" <breeves@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add support for using TDB to store the icount data, so we don't run out
of memory when checking really large filesystems.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The following patch addresses a memory leak in libext2fs
that occurs when using ext2fs_write_new_inode() on a file system
configured with large inodes.
Signed-off-by: Jim Garlick <garlick@llnl.gov>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Prevent floating point precision errors on really big filesystems from
causing the search interpolation algorithm in the icount abstraction
from looping forever.
Addresses Debian Bug: #411838
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Use pre-existing early exit label in function to handle proper
error code return and local memory allocation cleanup.
Coverity ID: 23: Resource Leak
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
This was causing dumpe2fs to crash on the ARM platform when examining
the badblocks list.
Also reverts an incorrect fix made by changeset 38078f692c20
Addresses Debian Bug: #397044
Add support for the new flag EXT2_FLAG_SOFTSUPP_FEATURES flag to
ext2fs_open() , which allows application to open filesystes with features
which are currently only partially supported by e2fsprogs.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The e2fsprogs and kernel implementation of directory hash tree has a
bug which causes the implementation to be dependent on whether
characters are signed or unsigned. Platforms such as the PowerPC,
Arm, and S/390 have signed characters by default, which means that
hash directories on those systems are incompatible with hash
directories on other systems, such as the x86.
To fix this we add a new flags field to the superblock, and define two
new bits in that field to indicate whether or not the directory should
be signed or unsigned. If the bits are not set, e2fsck and fixed
kernels will set them to the signed/unsigned value of the currently
running platform, and then respect those bits when calculating the
directory hash. This allows compatibility with current filesystems,
as well as allowing cross-architectural compatibility.
Addresses Debian Bug: #389772
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
- EXT4_FEATURE_RO_COMPAT_EXTRA_ISIZE (0x0040?) - add s_min_extra_isize and
s_want_extra_isize fields to superblock, which allow specifying
the minimum and desired i_extra_isize fields in large inodes
(for nsec+epoch timestamps, potential other uses). Needs RO_COMPAT
flag handling, needs e2fsck support, patch complete, little testing.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
- EXT4_FEATURE_INCOMPAT_64BIT (0x0080) - support for 64-bit block count
fields in the superblock (s_blocks_count_hi, s_free_blocks_count_hi),
large group descriptors (s_desc_size), extents with high 16 bits
(ee_start_hi, ei_leaf_hi), inode ACL (i_file_acl_hi). May also grow
to encompass the previously proposed BIG_BG.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
- EXT4_FEATURE_RO_COMPAT_DIR_NLINK (0x0020?) - allow directories to have
> 65000 subdirectories (i_nlinks) by setting i_nlinks = 1 for such
directories. RO_COMPAT protects old filesystems from unlinking such
directories incorrectly and losing all files therein.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
- EXT4_FEATURE_RO_COMPAT_GDT_CSUM (0x0010?) - store a crc16 checksum in
the group descriptor (s_uuid[16] | __u32 group | ext3_group_desc
(excluding gd_checksum itself)). This allows the kernel to more safely
manage UNINIT groups.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
- EXT4_FEATURE_RO_COMPAT_HUGE_FILE (0x0008) - change i_blocks to be
in units of s_blocksize units instead of 512-byte sectors, use
l_i_frag and l_i_fsize as i_blocks_hi (could also be part of 64BIT).
E2fsck and debugfs changed to support i_blocks_hi instead of l_i_frag and
l_i_fsize.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Check for potential overflow for filesystems contained in regular files
where the filesystem image size is returned by stat64().
Signed-off-by: Eric Sandeen <esandeen@sandeen.net>
Create new ext2fs library inline functions in order to calculate
the starting and ending blocks in a block group.
Signed-off-by: Eric Sandeen <esandeen@redhat.com>
There were still some %d's lurking when we print blocks & inodes; also
many of the counters in the e2fsck_struct were signed, and probably
need to be unsigned to avoid overflows.
Signed-off-by: Eric Sandeen <esandeen@redhat.com>
For loops iterating over all group descriptors, consistently define
first_block and last_block in a way that they are inclusive of the
range, and do not overflow.
Previously on the last block group we did a test of <= first +
dec_blocks; this would actually wrap back to 0 for a total block count
of 2^32-1
Also add handling of last block group which may be smaller.
Signed-off-by: Eric Sandeen <esandeen@redhat.com>
For loops such as:
for (i=1; i <= fs->super->s_blocks_count; i++) {
<do_stuff>
}
if i is an int and s_blocks_count is (2^32-1), the condition is never false.
Change these loops to:
for (i=1; i <= fs->super->s_blocks_count && i > 0; i++) {
<do_stuff>
}
to stop the loop when we overflow i
Signed-off-by: Eric Sandeen <esandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add a new function, ext2fs_div_ceil(), which correctly calculates a division
of two unsigned integer where the result is always rounded up the next
largest integer. This is used everywhere where we might have
previously caused an overflow when the number of blocks
or inodes is too close to 2**32-1.
Based on patches from Eric Sandeen, but generalized to use this new function
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Signed-off-by: Eric Sandeen <esandeen@redhat.com>
This patch allows "inode_size" to be specified in the mke2fs.conf file,
and always compiles in the "-I" option. In addition, it disallows
specifying the inode size on rev 0 filesystems, though I don't think
this was much of a danger anyways.
Clean up dead lines in ext2fs.h.
Signed-off-by: Andreas Dilger <adilger@clusterfs.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
When allocating space for the RAID filesystems with the stride parameter,
place each portion of the group's inode table right up after the superblock
(if present) in order to minimize fragmentation of the freespace.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This feature is initially intended for testing purposes; it allows an
ext2/ext3 developer to create very large filesystems using sparse files
where most of the block groups are not initialized and so do not require
much disk space. Eventually it could be used as a way of speeding up
mke2fs and e2fsck for large filesystem, but that would be best done by
adding an RO_COMPAT extension to the filesystem to allow the inode table
to be lazily initialized on a per-block basis, instead of being entirely initialized
or entirely unused on a per-blockgroup basis.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This caused FTBFS bugs on AMD64 platforms, since it uses a different
64-bit type when compared with IA64, so we need to make our
autoconfiguration system more intelligent.
Addresses Debian Bugs: #360661, #360317
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If the filesystem is opened in exclusive mode, then device will be
busy by definition, so don't return -EBUSY. This caused mke2fs -j to
fail on the 1.39-WIP (29-Mar-2006) release. (Addresses Debian Bug:
#360652)
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The x86 assembly instructures for bit test-and-set, test-and-clear,
etc., interpret the bit number as a 32-bit signed number, which is
problematic in order to support filesystems > 8TB.
Added new inline functions (in C) to implement a
ext2fs_fast_set/clear_bit() that does not return the old value of the
bit, and use it for the fast block/bitmap functions.
Added a regression test suite to test the low-level bit operations
functions to make sure they work correctly.
Note that a bitmap can address 2**32 blocks requires 2**29 bytes, or
512 megabytes. E2fsck requires 3 (and possibly 4 block bitmaps),
which means that the block bitmaps can require 2GB all by themselves,
and this doesn't include the 4 or 5 inode bitmaps (which assuming an
8k inode ratio, will take 256 megabytes each). This means that it's
more likely that a filesystem check of a filesystem greater than 2**31
blocks will fail if the e2fsck is dynamically linked (since the shared
libraries can consume a substantial portion of the 3GB address space
available to x86 userspace applications). Even if e2fsck is
statically linked, for a badly damaged filesystem, which may require
additional block and/or inode bitmaps, I am not sure e2fsck will
succeed in all cases.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This flag when specified to ext2fs_open or ext2fs_initialize indicates
that the application wants the io_channel to be opened in exclusive mode.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add a new io_channel open flag, IO_FLAG_EXCLUSIVE,which requests that
the device be opened in exclusive (O_EXCL) mode. Add support to the unix_io
implementation for this flag.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add a dependency to make sure that the subdirectories are created before
creating all of the object files.
Addresses Sourceforge Bug: #1261553
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
We no longer have the sparc assembly code in the header file any more, so we
shouldn't set _EXT2_HAVE_AS_BITOPS_. This would break compiles on the sparc
architectures when using gcc.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
#include <string.h> is needed since the inline functions use memcpy().
(Addresses Sourceforge Bug #1251062)
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Fix a bug when writing an external journal device on an big
endian machine (such as a S/390), where when the number of
block groups is zero, we never end up writing out the
primary superblock at all.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If fs->now is non-zero, use that as the time instead of the system
time when setting various filesystem fields (last modified time, last
write time, etc.)
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
In ext2fs_add_journal_inode() check for the case where the filesystem
appears to be unmounted, but the device is still apparently busy.
This can happen when the luser doesn't bother to mount /proc and has a
bogus /etc/mtab, but still wants to mount the filesystem before using
tune2fs(?!?). Add a safety check to save him from his own stupidity,
at least on 2.6 kernels. (Addresses Debian Bug #319002)
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
target is a regular file, instead of doing binary searching. It also
fixes a couple of cases where a file descriptor is leaked in the
ext2fs_getsize() routine on error.
Signed-off-by: Andreas Dilger <adilger@clusterfs.com>
ext2fs_test_bit to take an unsigned int for the bit number. Negative
bit numbers were never allowed (and didn't make any sense), so this should
be a safe change. This is needed to allow safe use of block numbers
greater than or equal to 2**31.
The trouble is that it is modifying pointers in place, but doing so via
"void *" types which alias the pointers passed in (which are typically
pointers to a struct.) The inline ext2fs_resize_mem() code may update
the pointer, but the caller is not required to reload the old value it
may have cached in a register, according to the type aliasing rules.
This is causing the caller to dereference the old pointer when compiled
with -O2, resulting in reproducible SEGV, on at least one ia64
configuration.
The compiler *is* required to reload if it sees an update to a dereferenced
char value, though, as chars are defined to alias anything; and memcpy()
is defined to operate on chars. So using memcpy() to copy the pointer
values is guaranteed to force the caller to reload. This has been
verified to fix the problem in practice.
Fixes Red Hat bug #161183.
honor the PAGER and SS_READLINE_PATH environtment variables, and the
test_io io_manager in the ext2fs library honors the TEST_IO_LOGFILE,
TEST_IO_FLAGS, TEST_IO_BLOCK, and TEST_IO_READ_ABORT environment variables.
environment variables if the libraries are called from setuid or setguid
programs, or if kernel believes that the process is not eligible to create
a core dump. In addition, if the libc has __secure_getenv(), use it so that
the libc can also do any additional limitations regarding when libraries can
trust environment variables (i.e., to integrate with systems like SELinux
and Posix capabilities).
stored in inodes into e2fsck.
There are a number of bug fixes and enhancements over the original lustre fsck
BK repository. The biggest one is that this extended attribute values must
be aligned on 4-byte boundaries.
a new inode we make sure that the extra information in the inode (any extra
fields in a large inode and any ea-in-inode information) is cleared. This
can happen when e2fsck creates a new root inode or a new lost+found directory,
or when the user uses the debugfs write, mknod, or mkdir commands. Otherwise,
the newly create inode could inherit garbage (or old EA information) from
a previously deleted inode.
than the GCC 3.4 compile code and triggers compiler warnings on
sparc64. Thanks to Matthias Andree for his analysis and suggestions.
(Addresses Debian Bug #232326)
Remove support for the --enable-old-bitops configure option which
was only for very old sparc systems.
unspecified, to avoid doing something surprising (such as unconditionally
deleting the first directory entry). Directory entries are now deleted
by coalescing them with the previous directory entry if possible, to
avoid directory fragmentation. This is not an issue with the e2fsprogs suite,
but may be a problem for some of the users of libext2fs, such as e2tools.
has its own copy of the orig_super data structure. (This
is a better way of fixing a double-free problem in
resize2fs which Fedora attempted to fix in
e2fsprogs-1.35-double_free.patch. Addresses Red Hat
Bugzilla #132707.)
byte-swapping options to e2fsck. This was the cause of some hard to
reproduce problems that had been reported in the past, and which the
resize_inode changes tickled in a much more repeatable fashion.
any debugging statements from within library code (always a bad idea), and
ext2fs_create_resize_inode() will return a proper error code if the
resize inode is corrupt, instead of returning -1.
res_gdt.c (list_backups), closefs.c (ext2fs_bg_has_super),
ext2fs.h: Move ext2fs_list_backups() to res_gdt.c, and
ext2fs_bg_has_super() back to closefs.c. There's no
reason for the new file, since list_backups() isn't being
used by any other functions, and can be made static, and
all users of the ext2fs filesystem will have to call
ext2fs_close() anyway.
correctly.
Update Makefile dependencies.
Update "make depend" production so that it filters out comments
inserted by newer gcc compilers.
Remove sync from e2fsck's "make all" target.
example, /tmp/test.img?offset=1024. Multiple options can separated using
the & character, although at the moment the only option implemented is
the offset option in the unix_io layer.
Change the maximum allowable blocksize to be 65536. This allows e2fsck to
check filesystems with a pagesize of 65536, and mke2fs to accept -b 65536.
Of course such a filesystem will not currently work on a Linux/x86 system,
at least not as of this writing!
and lib/ext2fs/getsize.c
In lib/blkid/getsize.c, include <sys/disk.h> if present since
this is where the DIOCGMEDIASIZE ioctl is defined on FreeBSD.
(Addresses Debian Bug #264630)
written last, and only after other I/O has been flushed to
disk. Thanks to Junfeng Yang from the Stanford
Metacompilation group for pointing a potential ordering
constraint problem if we don't write things out in the
right order.
or writes to a particular block. The block is specified by
TEST_IO_BLOCK environment variable, and the read/write count
by the TEST_IO_READ_ABORT and TEST_IO_WRITE_ABORT environment
variables. The block data is now only dumped if the 0x10 bit
is set in TEST_IO_FLAGS.
Theodore Ts'o