In practice, it is **extremely** rare for users to try to use more
than the first backup superblock located at the beginning of block
group #1. (i.e., at block number 32768 for file systems with a 4k
block size). This new compat feature restricts the backup superblock
to block group #1 and the last block group in the file system.
Aside from reducing the overhead of the file system by a small number
of blocks, by eliminating the rest of the backup superblocks, it
allows us to have a much more flexible metadata layout. For example,
we can force all of the allocation bitmaps and inode table blocks to
the beginning of the disk, which allows most of the disk to be
exclusively used for contiguous data blocks.
This simplifies taking advantage of certain HDD specific features,
such as Shingled Magnetic Recording (aka Shingled Drives), and the
TCG's OPAL Storage Specification where having a simple mapping between
LBA block ranges and the data blocks used by the file system can make
life much simpler.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add a LOG2_CHECK mode for check_super_value() so that it is easy
to verify values that are supposed to be power-of-two values
(s_desc_size and s_inode_size so far). In ext2fs_check_desc()
also check for a power-of-two s_desc_size.
Print out s_desc_size in debugfs "stats" and dumpe2fs output, if
it is non-zero.
It turns out that the s_desc_size validation in check_super_block()
is not currently used by e2fsck, because the group descriptors are
verified earlier by ext2fs_check_desc(), and even without an
explicit check of s_desc_size the group descriptors fail to align
correctly on disk. It makes sense to keep the check_super_block()
regardless, in case the code changes at some point in the future.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Fix all the places where we should be using a blk64_t instead of a
blk_t. These fixes are more severe because 64bit values could be
truncated silently.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Multi-mount protection is feature that allows mke2fs, e2fsck, and
others to detect if the filesystem is mounted on a remote node (on
SAN disks) and avoid corrupting the filesystem. For e2fsprogs this
means that it checks the MMP block to see if the filesystem is in use,
and marks the filesystem busy while e2fsck is running on the system.
This is useful on SAN disks that are shared between high-availability
servers, or accessible by multiple nodes that aren't in HA pairs. MMP
isn't intended to serve as a primary HA exclusion mechanism, but as a
failsafe to protect against user, software, or hardware errors.
There is no requirement that e2fsck updates the MMP block at regular
intervals, but e2fsck does this occasionally to provide useful
information to the sysadmin in case of a detected conflict.
For the kernel (since Linux 3.0) MMP adds a "heartbeat" mechanism to
periodically write to disk (every few seconds by default) to notify
other nodes that the filesystem is still in use and unsafe to modify.
Originally-by: Kalpak Shah <kalpak@clusterfs.com>
Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
This adds new APIs: ext2fs_flush2 and ext2fs_close2 which take an
extra 'int flags' parameter.
This allows us to pass in an EXT2_FLAG_FLUSH_NO_SYNC flag which avoids
fsync'ing the filesystem when closing it. For the case we have in
mind where we are just constructing a throwaway ext2 filesystem in a
file in order to boot a VM, this saves over 5 seconds during the boot
process and avoids many unnecessary disk writes.
Existing code using ext2fs_flush and ext2fs_close remains unaffected
by this change.
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The DEFS line in MCONFIG had gotten so long that it exceeded 4k, and
this was starting to cause some tools heartburn. It also made "make
V=1" almost useless, since trying to following the individual commands
run by make was lost in the noise of all of the defines.
So fix this by putting the configure-generated defines in lib/config.h
and the directory pathnames to lib/dirpaths.h.
In addition, clean up some vestigal defines in configure.in and in the
Makefiles to further shorten the cc command lines.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Code to count the number of blocks in the last partial
group is cut and pasted around the e2fsprogs codebase
a few times.
Making this a helper function should improve matters.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Filesystems with a blocksize of 1024 have the superblock starting at
block #1. However, the first data block in the superblock is 0 to
simplify the cluster calculations. So we must compensate for this in
a number of places, mostly in the ext2fs library, but also in e2fsck.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This adds the basic support for Direct I/O to unix_io.c, and adds a
new flag EXT_FLAG_DIRECT_IO which can be passed to ext2fs_open() or
ext2fs_open2() to request Direct I/O support.
Note that device mapper devices in Linux don't support Direct I/O, and
in some circumstances using Direct I/O can actually make performance
*worse*!
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The top-level COPYING file states that the e2p and ext2fs libraries
are available under the LGPLv2. The files were incorrectly labelled.
Alex Thomas/Luster has been consulted wrt to the ext3_extents.h file;
the rest of the files were primarily authored by Theodore Ts'o.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The 64-bit patches broke compiles on big endian systems. In addition
the block group checksum test was failing, due to bugs in both the
test case and the checksum code itself. This commit addresses these
problems.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
With 64-bit file systems, mke2fs can take a long time to do things
other than write inode tables. I exported the mke2fs numeric progress
meter and used it for allocating group tables and the final file
system flush.
Signed-off-by: Valerie Aurora (Henson) <vaurora@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The new ext2fs_super_and_bgd_loc2() function has some changes aside
from just blk64_t support. Lets make sure that the interfaces are
sane by adding libext2fs support early to get the new API tested here.
Signed-off-by: Jose R. Santos <jrs@us.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add new ext2fs_super_and_bgd_loc2() that returns blk64_t pointers.
The function now returns the number of blocks used by super block and
group descriptors since with flex_bg, it can no longer be assumed that
bitmaps and inode tables still resided within the block group.
Signed-off-by: Jose R. Santos <jrs@us.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If the superblock is clean, and we only need to update
s_kbytes_written, then we only need to update the superblock.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This field tracks the lifetime amount of writes to the filesystem. It
will be updated by the kernel as well as by e2fsprogs programs which
write to the filesystem.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add an explanation of exactly what ext2fs_super_and_bgd_loc() and
ext2fs_reserve_super_and_bgd_loc() do, and more importantly, exactly
what they return. Note that most callers should *not* rely on the
return value since it's rarely useful, especially once the flex_bg
feature is enabled and inode table and allocation bitmap blocks may
not be in the block group.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Now that e2fsck tries to backup the primary superblock to the backups
when the feature sets ar different, it's important when tune2fs writes
out a changed superblock, that we filter out the
EXT3_FEATURE_INCOMPAT_RECOVER feature to the backup superblocks, since
it will be removed from the primary superblock either when the
filesystem is mounted uncleanly or when journal is replayed.
Addresses-Debian-Bug: #454926
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This addresses a potential security vulnerability where an untrusted
filesystem can be corrupted in such a way that a program using
libext2fs will allocate a buffer which is far too small. This can
lead to either a crash or potentially a heap-based buffer overflow
crash. No known exploits exist, but main concern is where an
untrusted user who possesses privileged access in a guest Xen
environment could corrupt a filesystem which is then accessed by the
pygrub program, running as root in the dom0 host environment, thus
allowing the untrusted user to gain privileged access in the host OS.
Thanks to the McAfee AVERT Research group for reporting this issue.
Addresses CVE-2007-5497.
Signed-off-by: Rafal Wojtczuk <rafal_wojtczuk@mcafee.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add macros to support variable-length group descriptors for ext4.
Signed-off-by: Valerie Clement <valerie.clement@bull.net>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Use ext2fs_group_first_block() instead of the open-coded equivalent in
ext2fs_super_and_bgd_loc() and ext2fs_descriptor_block_loc().
Signed-off-by: Valerie Clement <valerie.clement@bull.net>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Fix a bug when writing an external journal device on an big
endian machine (such as a S/390), where when the number of
block groups is zero, we never end up writing out the
primary superblock at all.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If fs->now is non-zero, use that as the time instead of the system
time when setting various filesystem fields (last modified time, last
write time, etc.)
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
res_gdt.c (list_backups), closefs.c (ext2fs_bg_has_super),
ext2fs.h: Move ext2fs_list_backups() to res_gdt.c, and
ext2fs_bg_has_super() back to closefs.c. There's no
reason for the new file, since list_backups() isn't being
used by any other functions, and can be made static, and
all users of the ext2fs filesystem will have to call
ext2fs_close() anyway.
written last, and only after other I/O has been flushed to
disk. Thanks to Junfeng Yang from the Stanford
Metacompilation group for pointing a potential ordering
constraint problem if we don't write things out in the
right order.
the superblock and block group descriptors into two functions:
ext2fs_reserve_super_and_bgd, found in lib/ext2fs/alloc_sb.c, and
ext2fs_super_and_bgd_lock, found in lib/ext2fs/close.c.
Change e2fsck/pass1.c (mark_table_blocks), lib/ext2fs/closefs.c
(ext2fs_flush), lib/ext2fs/initialize.c (ext2fs_initialize),
and misc/dumpe2fs.c (list_desc) to use these functions.
e2fsck/ChangeLog
pass1.c (mark_table_blocks): Use the new function
ext2fs_reserve_super_and_bgd to calculate the blocks to be
reserved.
lib/ext2fs/ChangeLog
closefs.c (ext2fs_super_and_bgd_loc): New function which
centralizes the calculation of the superblock and block
group descriptors.
(ext2fs_flush): Use ext2fs_super_and_bgd_lock to figure
out where to write the superblock and block group
descriptors.
alloc_sb.c (ext2fs_reserve_super_and_bgd): New function which
reserves space in the block bitmap using
ext2fs_super_and_bgd_loc.
initialize.c (ext2fs_initialize): Use
ext2fs_reserve_super_and_bgd to initialize the block bitmap.
misc/ChangeLog
dumpe2fs.c (list_desc): Use ext2fs_super_and_bgd_loc to
determine the locations of the superblock and block group
descriptors.
unconditional ext2_flush() call. Also made ext2_flush() more
efficient when the superblock is modified after the filesystem has
been flushed, and make it clear the superblock dirty flag
after doing the superblock/block group descriptor flush.
Makefile.in: Add message.c and swapfs.c to the list of source files to
build the make depend.
swapfs.c, unix.c: Only support the -s and -S options to e2fsck if
ENABLE_SWAPFS is defined.
Many files:
ext2fs.h, bitops.h, block.c, bmap.c, closefs.c, dirblock.c, inode.c,
native.c, openfs.c, rw_bitmaps.c, swapfs.c: Only include the
byte-swapping logic if ENABLE_SWAPFS is turned on or if we're on a
big-endian machine.
initialize.c (ext2fs_initialize):Use WORDS_BIGENDIAN directly to set
EXT2_FLAG_SWAP_BYTES, instead of using ext2fs_native_flag.
native.c (ext2fs_native_flag): Use WORDS_BIGENDIAN provided by
autoconf to determine whether or not return EXT2_FLAG_SWAP_BYTES.
Theodore Ts'o