Teach e2fsck to (re)construct extent trees. This enables us to do
either of the following: compress a highly sparse extent tree into
fewer ETB blocks; or convert a ext3-style block mapped file to an
extent file. The reconstruction is performed during pass 1E or 3A,
as detailed below.
For files that are already extent based, this algorithm will
automatically run (pending user approval) if pass1 determines either
(1) that a whole level of extent tree will fit into a higher level of
the tree; (2) that the size of any level can be reduced by at least
one ETB block; or (3) the extent tree is unnecessarily deep. It will
not run at all if errors are found and the user declines to fix the
errors.
The option "-E bmap2extent" can be used to force e2fsck to convert all
block map files to extent trees, and to rebuild all extent files'
extent trees. After conversion, files larger than 12 blocks should be
defragmented to eliminate empty holes where a block lives.
The extent tree constructor is pretty dumb -- it creates a list of
leaf extents (adjacent extents are collapsed), marks all indirect
blocks / ETB blocks free, installs a new extent tree root in the
inode, then loads the leaf extents into the tree.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If we encounter an inode with IND/DIND/TIND blocks or internal extent
tree blocks that point into critical FS metadata such as the
superblock, the group descriptors, the bitmaps, or the inode table,
it's quite possible that the validation code for those blocks is not
going to like what it finds, and it'll ask to try to fix the block.
Unfortunately, this happens before duplicate block processing (pass
1b), which means that we can end up doing stupid things like writing
extent blocks into the inode table, which multiplies e2fsck'
destructive effect and can render a filesystem unfixable.
To solve this, create a bitmap of all the critical FS metadata. If
before pass1b runs (basically check_blocks) we find a metadata block
that points into these critical regions, continue processing that
block, but avoid making any modifications, because we could be
misinterpreting inodes as block maps. Pass 1b will find the
multiply-owned blocks and fix that situation, which means that we can
then restart e2fsck from the beginning and actually fix whatever
problems we find.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Mostly by adding static and removing excess extern qualifiers. Also
convert a few remaining non-ANSI function declarations to ANSI.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add the ability to log messages about a file system to a specified
directory, using a file name templace that can be specified in
/etc/e2fsck.conf. This allows us to suppress the output of overly
verbose e2fsck outputs while still allowing the full logging output to
go to an appropriate file.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Multi-mount protection is feature that allows mke2fs, e2fsck, and
others to detect if the filesystem is mounted on a remote node (on
SAN disks) and avoid corrupting the filesystem. For e2fsprogs this
means that it checks the MMP block to see if the filesystem is in use,
and marks the filesystem busy while e2fsck is running on the system.
This is useful on SAN disks that are shared between high-availability
servers, or accessible by multiple nodes that aren't in HA pairs. MMP
isn't intended to serve as a primary HA exclusion mechanism, but as a
failsafe to protect against user, software, or hardware errors.
There is no requirement that e2fsck updates the MMP block at regular
intervals, but e2fsck does this occasionally to provide useful
information to the sysadmin in case of a detected conflict.
For the kernel (since Linux 3.0) MMP adds a "heartbeat" mechanism to
periodically write to disk (every few seconds by default) to notify
other nodes that the filesystem is still in use and unsafe to modify.
Originally-by: Kalpak Shah <kalpak@clusterfs.com>
Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The DEFS line in MCONFIG had gotten so long that it exceeded 4k, and
this was starting to cause some tools heartburn. It also made "make
V=1" almost useless, since trying to following the individual commands
run by make was lost in the noise of all of the defines.
So fix this by putting the configure-generated defines in lib/config.h
and the directory pathnames to lib/dirpaths.h.
In addition, clean up some vestigal defines in configure.in and in the
Makefiles to further shorten the cc command lines.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
There are broken embedded devices that have system clocks that always
reset to January 1, 1970 whenever they boot (even if no power is
lost). There are also systems that have super cheap clock crystals
that can be very inaccurate. So if the option broken_system_clock is
given, disable all time based checks. E2fsck will also try to detect
incorrect system clock times, and automatically mark the system clock
as insane.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If the directory is packed with no slack space, as soon as any new
directory entries are added, leaf nodes end up getting split and
directory ends up getting very inefficient.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Track the number of non-contiguous files and directories so we can
give more detailed information in verbose mode.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Also make sure the device name has no spaces in it, to avoid confusing
displays, and make ctx->filesystem_name and ctx->device_name allocated
memory to avoid potential problems in the future.
Addresses-Launchpad-Bug: #203323
Addresses-Sourceforge-Bug: #1926023
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Allow files to be preallocated on-disk up to the next multiple of the
system's page size without complaining about extra blocks.
Signed-off-by: Andreas Dilger <adilger@clusterfs.com>
Signed-off-by: Girish Shilamkar <girish@clusterfs.com>
Signed-off-by: Kalpak Shah <kalpak@clusterfs.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
pass4.c (e2fsck_pass4): Clear inode_imagic_map after freeing it, to
prevent it from getting freed twice.
unix.c (main): Close the filesystem before freeing the context, so
that in the event of a free()-related segmentation violation, the
filesystem is properly closed and written out.
e2fsck.c (e2fsck_reset_context): When freeing ctx->inode_reg_map, we
weren't zero'ing ctx->inode_reg_map, which could cause a segfault
later on in the e2fsck run.
problem.h (PR_2_SPLIT_DOT): Fix excess zero in problem code (now
matches the standard convention).
pass4.c (e2fsck_pass4): If an inode is set in the inode_imagic_map
bitmap, don't check to see if it is disconnected from the inode tree
(because it almost certainly will be). Free inode_imagic_map at the
end of pass 4.
pass2.c (check_dir_block, check_filetype): If the FILETYPE feature is
set, check the directory entry's filetype information field, and
fix/set it if necessary. (e2fsck_pass2): Free the inode_reg_map
bitmap at the end of pass 2.
pass1.c (e2fsck_pass1, alloc_imagic_map): Allocate and fill in
information for inode_reg_map and inode_imagic_map, which indicates
which inodes are regular files and AFS inodes, respectively.
Since only the master superblock is written during a restart, force
that superblock to be used after a restart; otherwise changes to the
block group descriptors end up getting ignored.
problem.c, problemP.h: If e2fsck is run -n, make def_yn variable be 0
for "no". Add support for a new flag, PR_NO_NOMSG, which supresses
the problem message if e2fsck is run with the -n option.
problem.c, problem.h (PR_2_SET_FILETYPE, PR_2_BAD_FILETYPE): Add new
problem codes.
message.c (expand_dirent_expression): Add support for %dt which prints
the dirent type information.
e2fsck.c (e2fsck_reset_context): Free new bitmaps (inode_reg_map and
inode_imagic_map).
e2fsck.h (e2fsck_t): Add new inode_reg_map and inode_magic_map to the
context structure.
ChangeLog, nt_io.c:
nt_io.c: New file which supports I/O under Windows NT.
ChangeLog, gen_uuid_nt.c:
gen_uuid_nt.c: New file which creates a UUID under Windows NT.
Many files:
Add support for non-Unix compiles
tune2fs.8.in: Fix minor display bug in the nroff.
mke2fs.c (show_stats, write_inode_tables): Use the log10 function to
calculate the display of block numbers so that things look nice on an
80 character display.
mke2fs.c (usage): Add the sparse-super-flag to the usage message.
ChangeLog, e2fsck.c, pass1.c, pass2.c, problem.c, problem.h, unix.c:
unix.c (main): Move ext2fs_close() after e2fsck_free_context() since
e2fsck_free_context may reference data in ctx->fs.
e2fsck.c (e2fsck_reset_context): Make sure ctx->fs is non-NULL before
checking ctx->fs->dblist.
pass1.c (e2fsck_pass1): Use the device check subroutine on FIFO's and
Socket's, so that we catch bogus immutable inodes.
pass2.c (process_bad_inode): Process bad socket and fifo's.
problem.h, problem.c: Define new problem codes PR_2_BAD_FIFO and
PR_2_BAD_SOCKET.
pass2.c (e2fsck_pass2): Fix the progress accounting so that we get to
100%.
pass3.c (e2fsck_pass3): Change progress accounting to be consistent
with the other e2fsck passes.
e2fsck.c (e2fsck_run): At the end of each pass, call the progress
function with the pass number set to zero.
unix.c (e2fsck_update_progress): If the pass number is zero, ignore
the call, since that indicates that we just want to deallocate any
progress structures.
emptydir.c:
Commit partially done file.
ChangeLog, badblocks.c:
badblocks.c (ext2fs_badblocks_list_add): Use a bigger increment than
10 blocks when we need to expand the size of the badblocks list.
unix.c: Fix bug in check of feature set, to make sure we can really
fix this filesystem.
problem.h: Make blkcount type to be of type blkcnt_t. Make the num
field be a 64 bit type. Add the problem code PR_1_FEATURE_LARGE_FILES
problem.c: Add table entry for the problem code PR_1_FEATURE_LARGE_FILES.
pass1.c (e2fsck_pass1): A non-zero i_dir_acl field is only a problem
for directory inodes. (Since it is also i_size_high now.) If there
are no large_files, then clear the LARGE_FLAG feature flag. If there
are large_files, but the LARGE_FLAG feature flag is not set, complain
and offer to fix it.
(check_blocks): Add support to deal with non-directory inodes that
have i_size_high set (i.e., large_files). Don't give an error if a
directory has preallocated blocks, to support the DIR_PREALLOC
feature.
(process_block, process_bad_block): The blockcnt variable is a type of
blkcnt_t, for conversion to the new block_iterate2.
pass2.c (process_bad_inode): A non-zero i_dir_acl field is only a
problem for directory inodes. (Since it is also i_size_high now.)
message.c (expand_inode_expression): Print a 64-bits of the inode size
for non-directory inodes. (Directory inodes can only use a 32-bit
directory acl size, since i_size_high is shared with i_dir_acl.) Add
sanity check so that trying to print out the directory acl on a
non-directory inode will print zero. (expand_percent_expression): %B
and %N, which print pctx->blkcount and pctx->num, can now be 64 bit
variables. Print them using the "%lld" format if EXT2_NO_64_TYPE is
not defined.
e2fsck.h: Add the large_flagsfield to the e2fsck context.
e2fsck.c (e2fsck_reset_context): Clear the large_flags field.
ChangeLog, expect.1:
f_messy_inode: Modify test to deal with changes to support 64-bit size
files. (/MAKEDEV had i_dir_acl, now i_size_high, set.)
e2fsck.c (e2fsck_run): Since E2F_FLAG_SIGNAL_MASK doesn't
include EXT2_FLAG_RESTART anymore, we need to adjust this
routine so that it *does* return in the case of it seeing
EXT2_FLAG_RESTART.
pass1.c (e2fsck_pass1): ext2_get_next_inode() may call the group done
callback function, which may set context abort flags. So we need to
test the context abort flags after we call ext2_get_next_inode().
(process_inodes): If we abort due out of process_inodes, do a clean
exit by breaking out of the for loop instead of just returning.
e2fsck.h (E2F_FLAG_SIGNAL_MASK): EXT2_FLAG_RESTART shouldn't be
considered a SIGNAL mask (i.e., requiring an immediate abort of
processing to restart). FLAG_RESTART just means that we want to
restart once pass 1 is complete.
dirinfo.c, e2fsck.c: Don't include com_err.h; it isn't needed.
e2fsck.h: Include <time.h> since it is needed
super.c: If EXT2_SKIP_UUID is defined, then skip the UUID processing.
Change ext2fs_read_inode, ext2fs_write_inode to take the e2fsck
context as their first argument.
Change dir_info.c routines to take an e2fsck_context, renamed them to
start with e2fsck_ to avoid namespace issues, and changed them to
store the directory information inside the e2fsck context.
Added e2fsck_run() which calls all of the e2fsck passes in the correct
order, and which handles the return of abort codes.
Added abort processing, both via setjmp/longjmp and via flags in the
e2fsck context.
Use a flag in the e2fsck context instead of the restart_e2fsck global
variable.
Change uses of free and malloc to ext2fs_free_mem and ext2fs_get_mem.
pass*.c, super.c: Massive changes to avoid using printf and com_err
routines. All diagnostic messages are now routed through the
fix_problem interface.
pass2.c (check_dir_block): Check for duplicate '.' and '..' entries.
problem.c, problem.h: Add new problem codes PR_2_DUP_DOT and
PR_2_DUP_DOT_DOT.
problem.c: Added new problem codes for some of the superblock
corruption checks, and for the pass header messages. ("Pass
1: xxxxx")
util.c (print_resource_track): Now takes a description argument.
super.c, unix.c, e2fsck.c: New files to separate out the
operating-specific operations out from e2fsck.c. e2fsck.c now
contains the global e2fsck context management routines, and
super.c contains the "pass 0" initial validation of the
superblock and global block group descriptors.
pass1.c, pass2.c, pass3.c, pass4.c, pass5.c, util.c: Eliminate
(nearly) all global variables and moved them to the e2fsck
context structure.
problem.c, problem.h: Added new problem codes PR_0_SB_CORRUPT,
PR_0_FS_SIZE_WRONG, PR_0_NO_FRAGMENTS, PR_0_BLOCKS_PER_GROUP,
PR_0_FIRST_DATA_BLOCK
expect.1, expect.2:
Updated tests to align with e2fsck problem.c changes.
error_table.h:
et_name.c (error_table_name):
error_message.c (error_message.c): Make code be 16-bit safe.
ChangeLog, e2fsck.c:
e2fsck.c (check_mount): Add stronger warning message about the perils
of running e2fsck on a mounted filesystem.
Darrick J. Wong