Allow the filesystem to store the metadata checksum seed in the
superblock and add an incompat feature to say that we're using it.
This enables tune2fs to change the UUID on a mounted metadata_csum
FS without having to (racy!) rewrite all disk metadata.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
This patch add project feature flag EXT4_FEATURE_RO_COMPAT_PROJECT.
Project feature is a read-only compat feature. Thus, an ext4 file
system with project feature enabled could only be read by ext4
kernel module without project feature support.
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Having multiple versions of jfs_user.h was confusing the Android
build. Clean up things by removing the lib/ext2fs/jfs_user.h and
misc/jfs_user.h and simplifying how we emulate the kernel
infrastructure needed by journal replay code and removing the
kernel-specific lines from kernel-jbd.h.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
It turns out that there are some serious problems with the on-disk
format of journal checksum v2. The foremost is that the function to
calculate descriptor tag size returns sizes that are too big. This
causes alignment issues on some architectures and is compounded by the
fact that some parts of jbd2 use the structure size (incorrectly) to
determine the presence of a 64bit journal instead of checking the
feature flags. These errors regrettably lead to the journal
corruption reported by Mr. Reardon.
Therefore, introduce journal checksum v3, which enlarges the
descriptor block tag format to allow for full 32-bit checksums of
journal blocks, fix the journal tag function to return the correct
sizes, and fix the jbd2 recovery code to use feature flags to
determine 64bitness.
Add a few function helpers so we don't have to open-code quite so
many pieces.
Switching to a 16-byte block size was found to increase journal size
overhead by a maximum of 0.1%, to convert a 32-bit journal with no
checksumming to a 32-bit journal with checksum v3 enabled.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: TR Reardon <thomas_reardon@hotmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
In practice, it is **extremely** rare for users to try to use more
than the first backup superblock located at the beginning of block
group #1. (i.e., at block number 32768 for file systems with a 4k
block size). This new compat feature restricts the backup superblock
to block group #1 and the last block group in the file system.
Aside from reducing the overhead of the file system by a small number
of blocks, by eliminating the rest of the backup superblocks, it
allows us to have a much more flexible metadata layout. For example,
we can force all of the allocation bitmaps and inode table blocks to
the beginning of the disk, which allows most of the disk to be
exclusively used for contiguous data blocks.
This simplifies taking advantage of certain HDD specific features,
such as Shingled Magnetic Recording (aka Shingled Drives), and the
TCG's OPAL Storage Specification where having a simple mapping between
LBA block ranges and the data blocks used by the file system can make
life much simpler.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This is what the patches from Zhen Liu uses, so let's make this change
now to keep things easier. INCOMPAT_INLINE_DATA also looks better
IMHO. :-)
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Modify the dump code to print information about jbd2 v2 checksum data.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The replica is a feature which stores multiple copies of the key
metadata blocks so a single block failure in failure-prone media
(read: certain types of flash storage) doesn't take out the entire
file system.
Discussion on the upstream list proved not to be very positive on this
feature; the arguments were that it added complexity that wasn't
warrented, since common practice in industry is to insist on reliable
media, and if media is unreliable, you're kind of toast anyway (unless
the file system is being used as the back-end store of a cluster file
system where checksuming and data replication is happening above the
local disk file system level). So, this feature is being developed
out of tree.
We reserve the code points so that other people won't accidentally
step on them. Since it's not upstream, it's a soft reservation, but
it's not like we have any shortage of RO_COMPAT features. We are a
bit more tight on reserved inodes, but EXT2_BOOT_LOADER_INO and
EXT2_UNDEL_DIR_INO are not currently used anywhere, and
EXT2_EXCLUDE_INO is a reservation for another out-of-tree feature.
There are no features currently being discussed which require a
reserved inode, but if a need were to arise, we can claw back code
point reservations that were never used or not in tree, as those will
always be considered lower priority than in-tree features.
Cc: Aditya Kali <adityakali@google.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Multi-mount protection is feature that allows mke2fs, e2fsck, and
others to detect if the filesystem is mounted on a remote node (on
SAN disks) and avoid corrupting the filesystem. For e2fsprogs this
means that it checks the MMP block to see if the filesystem is in use,
and marks the filesystem busy while e2fsck is running on the system.
This is useful on SAN disks that are shared between high-availability
servers, or accessible by multiple nodes that aren't in HA pairs. MMP
isn't intended to serve as a primary HA exclusion mechanism, but as a
failsafe to protect against user, software, or hardware errors.
There is no requirement that e2fsck updates the MMP block at regular
intervals, but e2fsck does this occasionally to provide useful
information to the sysadmin in case of a detected conflict.
For the kernel (since Linux 3.0) MMP adds a "heartbeat" mechanism to
periodically write to disk (every few seconds by default) to notify
other nodes that the filesystem is still in use and unsafe to modify.
Originally-by: Kalpak Shah <kalpak@clusterfs.com>
Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The DEFS line in MCONFIG had gotten so long that it exceeded 4k, and
this was starting to cause some tools heartburn. It also made "make
V=1" almost useless, since trying to following the individual commands
run by make was lost in the noise of all of the defines.
So fix this by putting the configure-generated defines in lib/config.h
and the directory pathnames to lib/dirpaths.h.
In addition, clean up some vestigal defines in configure.in and in the
Makefiles to further shorten the cc command lines.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Using the /* fallthrough */ comment lets Coverity (and humans)
know that we really do want to fall through in these case statements.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The set_fields commands (set_super_value, set_inode_field,
set_block_group) now handle fields which store in split fields on
ext4's on-disk format. For example, the superblock fields
s_blocks_count and s_blocks_count_hi.
The user can either set the low or high part of the field via
"blocks_count_lo" or "blocks_count_hi", or both parts can be set via
"blocks_count".
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reserve EXT4_FEATURE_RO_COMPAT_METADATA_CSUM and
EXT2_FEATURE_COMPAT_EXCLUDE_BITMAP. Also reserve fields in the
superblock and the inode for the checksums. In the block group
descriptor, reserve the exclude bitmap field for the snapshot feature,
and checksums for the inode and block allocation bitmaps.
With this commit, the metadata checksum and exclude bitmap features
should have reserved all of the fields they need in ext4's on-disk
format.
This commit also fixes an a missing byte swap for s_overhead_blocks.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: Darrick J. Wong <djwong@us.ibm.com>
Cc: Amir Goldstein <amir73il@gmail.com>
This adds the superblock fields needed so that dumpe2fs works and the
code points and renames the superblock fields from describing
fragments to clusters.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This patch adds support for detecting the new 'quota' feature in ext4.
The patch reserves code points for usr and group quota inodes and also
for the feature flag EXT4_FEATURE_RO_COMPAT_QUOTA.
Signed-off-by: Aditya Kali <adityakali@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The top-level COPYING file states that the e2p and ext2fs libraries
are available under the LGPLv2. The files were incorrectly labelled.
Alex Thomas/Luster has been consulted wrt to the ext3_extents.h file;
the rest of the files were primarily authored by Theodore Ts'o.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Print out the currently supported features of e2fsprogs/libext2fs
via a new "debugfs supported_features" command. This helps scripts
to know whether it is possible to try and enable specific features
in the filesystem.
Signed-off-by: Kalpak Shah <kalpak.shah@sun.com>
Signed-off-by: Andreas Dilger <adilger@sun.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
This was the original name used by Lustre's patches; keep the plural
when converting feature names to a feature mask for compatibility's
sake.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Allow the old name of uninit_groups when converting feature names for
backwards compatibility for scripts running mke2fs and tune2fs.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This name is a more intuitive option when running mke2fs.
Signed-off-by: Jose R. Santos <jrs@us.ibm.com>
Signed-off-by: Andreas Dilger <adilger@clusterfs.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This creates a new enhanced edit_feature function for libe2p which
supports a different set of feature flags that are OK to clear as
opposed to set, and which returns more specific information about why
the user provided an invalid edit feature command.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add FLEX_BG as a supported feature bit.
Add support to mke2fs to create filesystems with FLEX_BG.
Add support to tune2fs to add (and remove, if it won't break
filesystem consistency) the FLEX_BG feature.
Signed-off-by: Jose R. Santos <jrs@us.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
--
lib/e2p/feature.c | 2 ++
lib/ext2fs/ext2fs.h | 6 ++++--
misc/mke2fs.c | 7 ++++++-
3 files changed, 12 insertions(+), 3 deletions(-)
Add support for printing the huge_file, gdt_checksum, dir_nlink,
extra_isize, extent, and 64bit features.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This feature is initially intended for testing purposes; it allows an
ext2/ext3 developer to create very large filesystems using sparse files
where most of the block groups are not initialized and so do not require
much disk space. Eventually it could be used as a way of speeding up
mke2fs and e2fsck for large filesystem, but that would be best done by
adding an RO_COMPAT extension to the filesystem to allow the inode table
to be lazily initialized on a per-block basis, instead of being entirely initialized
or entirely unused on a per-blockgroup basis.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Change the maximum allowable blocksize to be 65536. This allows e2fsck to
check filesystems with a pagesize of 65536, and mke2fs to accept -b 65536.
Of course such a filesystem will not currently work on a Linux/x86 system,
at least not as of this writing!
feature.c: Remove backward compatibility #ifdef's for old ext2_fs.h
header files.
.del-ext2_fs.h~7a460879, ChangeLog:
ext2_fs.h (EXT3_FEATURE_INCOMPAT_JOURNAL_DEV): Add definition for use
with external journal devices.
Makefile.in (uuid_time): Compile uuid_time in two steps (first create
.o, then link it against the libraries) to work around bug in a.out
linker.
dll/jump.funcs, dll/jump.import, dll/jump.params: Update a.out shared
library control files to reflect new added files.
ChangeLog, feature.c, jump.funcs:
feature.c: Make feature_list static; it shouldn't be exported.
dll/jump.funcs: Update a.out shared library control file.
ChangeLog, badblocks.c:
badblocks.c (flush_bufs): Use fsync() if the system doesn't support
fdatasync().
feature.c: Fix GCC warnings; add const to the char * types in the
function prototypes for e2p_feature2string and e2p_edit_feature.
ChangeLog, uuid.h, uuid_time.c:
uuid_time.c (variant_string): Declare to be static to avoid gcc warnings.
uuid.h: Add function prototypes for uuid_generate_random() and
uuid_generate_time().
ChangeLog, chattr.c:
chattr.c: Add hack to compile in a definition for S_ISLNK so we can
successfully compile even with warnings turned on.
feature.c (e2p_edit_feature), e2p.h: Add a new argument which allows
the calling application to limit what features the user is allowed to
set or clear using this function. Also add support for comma
separated lists.
Makefile.in, feature.c, e2p.h: New file which is used for displaying
and editing superblock feature sets.
ls.c (print_features, list_super): Add new function print_features
which is used to display the feature bits in the superblock.
Darrick J. Wong