Use a large_inode so that when e2fsck is fixing a file system with
project quota enabled, the correct project id's quota is adjusted when
a corrupted inode is deleted.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
This patch adds project quota support. An new quota type PRJQUOTA(2)
is added. EXT4_PRJ_QUOTA_INO(11) is reserved for project quota inode.
The super block reservers an field s_prj_quota_inum for saving
project quota inode. And each inode adds an internal field i_projid
for saving its project ID.
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
e2fsck pass1 is modified to use the block group data prefetch function
to try to fetch the inode tables into the pagecache before it is
needed. We iterate through the blockgroups until we have enough inode
tables that need reading such that we can issue readahead; then we sit
and wait until the last inode table block read of the last group to
start fetching the next bunch.
pass2 is modified to use the dirblock prefetching function to prefetch
the list of directory blocks that are assembled in pass1. We use the
"iterate a subset of a dblist" and avoid copying the dblist. Directory
blocks are fetched incrementally as we walk through the directory
block list. In previous iterations of this patch we would free the
directory blocks after processing, but the performance hit to e2fsck
itself wasn't worth it. Furthermore, it is anticipated that most
users will then mount the FS and start using the directories, so they
may as well remain in the page cache.
pass4 is modified to prefetch the block and inode bitmaps in
anticipation of pass 5, because pass4 is entirely CPU bound.
In general, these mechanisms can decrease fsck time by 10-40%, if the
host system has sufficient memory and the storage system can provide a
lot of IOPs. Pretty much any storage system capable of handling
multiple IOs in-flight at any time will see a fairly large performance
boost. (Single-issue USB mass storage disks seem to suffer badly.)
By default, the readahead buffer size will be set to the size of a block
group's inode table (which is 2MiB for a regular ext4 FS). The -E
readahead_kb= option can be given to specify the amount of memory to
use for readahead or zero to disable it entirely; or an option can be
given in e2fsck.conf.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
In e2fsck_pass4(), we were consulting inode_dir_map using
ext2fs_test_inode_bitmap2() for every single inode in the file system.
However, there were many cases where we never needed the result of the
test --- most notably if the inode is not in use.
I was a bit surprised that GCC 4.7 with CFLAGS set to "-g -O2" wasn't
able to optimize this out for us, but here is the pass 4 timing for an
empty 3T file system before this patch:
Pass 4: Memory used: 672k/772k (422k/251k), time: 3.67/ 3.66/ 0.00
and afterwards, we see a 43% improvement:
Pass 4: Memory used: 672k/772k (422k/251k), time: 2.09/ 2.08/ 0.00
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The DEFS line in MCONFIG had gotten so long that it exceeded 4k, and
this was starting to cause some tools heartburn. It also made "make
V=1" almost useless, since trying to following the individual commands
run by make was lost in the noise of all of the defines.
So fix this by putting the configure-generated defines in lib/config.h
and the directory pathnames to lib/dirpaths.h.
In addition, clean up some vestigal defines in configure.in and in the
Makefiles to further shorten the cc command lines.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This patch adds support for doing quota accounting during full
e2fsck scan if the 'quota' feature was set on the superblock.
If user-visible quota inodes are in use, they will be hidden
and converted to the reserved quota inodes.
Signed-off-by: Aditya Kali <adityakali@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
This fixes two possible causes for the error message:
WARNING: PROGRAMMING BUG IN E2FSCK!
OR SOME BONEHEAD (YOU) IS CHECKING A MOUNTED (LIVE) FILESYSTEM.
inode_link_info[X] is Y, inode.i_links_count is Z. They should be the same!
One cause which can trigger this message is when an inode has an
illegal link count > 65500 --- for example, 65535. This was the case
in the Debian Bug report #555456.
Another cause which could trigger this message is if an ext4 directory
previously had more than 65000 subdirectories (thus causing
i_link_count to be set to 1), but then some of the subdirectories were
deleted, such that i_link_count should now be the actual number of
subdirectories.
Addresses-Debian-Bug: #555456
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
in the case of ! defined RESOURCE_TRACK, so that we can clean up #ifdef
throughout e2fsck source.
Signed-off-by: Ken Chen <kenchen@google.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This patch includes the changes required to e2fsck to understand the
nlink count changes made in the kernel.
In e2fsck pass 4, when we fetch the actual link count, if it is
exceeds 65,000 we set the link count to 1. We silently fix the
situation where the nlink count of the directory is 1, and there are
fewer than 65,000 subdirectories, since since that can happen
naturally.
Patch originally from CFS, significantly rewritten by Theodore Ts'o.
Signed-off-by: Andreas Dilger <adilger@clusterfs.com>
Signed-off-by: Kalpak Shah <kalpak@clusterfs.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add vertificaton of the in-inode EA information, and allow in-inode
EA's to have a checksum.
Signed-off-by: Andreas Dilger <adilger@clusterfs.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Factor out code to clear a bogus inode and update e2fsck's internal
data structures accordingly into a common routine,
e2fsck_clear_inode(). This saves about 200 bytes in the compiled x86
e2fsck executable, and makes the code more maintainable in the
long-term.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This patch instruments the libext2fs unix I/O manager and adds bytes
read/written and data rate to e2fsck -tt pass/overall timing output.
Signed-off-by: Jim Garlick <garlick@llnl.gov>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
For loops such as:
for (i=1; i <= fs->super->s_blocks_count; i++) {
<do_stuff>
}
if i is an int and s_blocks_count is (2^32-1), the condition is never false.
Change these loops to:
for (i=1; i <= fs->super->s_blocks_count && i > 0; i++) {
<do_stuff>
}
to stop the loop when we overflow i
Signed-off-by: Eric Sandeen <esandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
make sure we gracefully clean up and only exit at safe points.
For fsck, we pass the SIGINT/SIGTERM signal to the child processes,
so they can do their own cleanup.
pass4.c (e2fsck_pass4): Clear inode_imagic_map after freeing it, to
prevent it from getting freed twice.
unix.c (main): Close the filesystem before freeing the context, so
that in the event of a free()-related segmentation violation, the
filesystem is properly closed and written out.
e2fsck.c (e2fsck_reset_context): When freeing ctx->inode_reg_map, we
weren't zero'ing ctx->inode_reg_map, which could cause a segfault
later on in the e2fsck run.
problem.h (PR_2_SPLIT_DOT): Fix excess zero in problem code (now
matches the standard convention).
pass4.c (e2fsck_pass4): If an inode is set in the inode_imagic_map
bitmap, don't check to see if it is disconnected from the inode tree
(because it almost certainly will be). Free inode_imagic_map at the
end of pass 4.
pass2.c (check_dir_block, check_filetype): If the FILETYPE feature is
set, check the directory entry's filetype information field, and
fix/set it if necessary. (e2fsck_pass2): Free the inode_reg_map
bitmap at the end of pass 2.
pass1.c (e2fsck_pass1, alloc_imagic_map): Allocate and fill in
information for inode_reg_map and inode_imagic_map, which indicates
which inodes are regular files and AFS inodes, respectively.
Since only the master superblock is written during a restart, force
that superblock to be used after a restart; otherwise changes to the
block group descriptors end up getting ignored.
problem.c, problemP.h: If e2fsck is run -n, make def_yn variable be 0
for "no". Add support for a new flag, PR_NO_NOMSG, which supresses
the problem message if e2fsck is run with the -n option.
problem.c, problem.h (PR_2_SET_FILETYPE, PR_2_BAD_FILETYPE): Add new
problem codes.
message.c (expand_dirent_expression): Add support for %dt which prints
the dirent type information.
e2fsck.c (e2fsck_reset_context): Free new bitmaps (inode_reg_map and
inode_imagic_map).
e2fsck.h (e2fsck_t): Add new inode_reg_map and inode_magic_map to the
context structure.
ChangeLog, nt_io.c:
nt_io.c: New file which supports I/O under Windows NT.
ChangeLog, gen_uuid_nt.c:
gen_uuid_nt.c: New file which creates a UUID under Windows NT.
Many files:
Add support for non-Unix compiles
unix.c: Add support for calculating a progress bar if the -C0 option
is given. The function e2fsck_clear_progbar() clears the progress bar
and must be called before any message is issued. SIGUSR1 will enable
the progress bar, and SIGUSR2 will disable the progress bar. This is
used by fsck to handle parallel filesystem checks. Also, set the
device_name from the filesystem label if it is available.
e2fsck.h: Add new flags E2F_FLAG_PROG_BAR and E2F_FLAG_PROG_SUPRESS.
Add new field in the e2fsck structure which contains the last tenth of
a percent printed for the user.
message.c (print_e2fsck_message): Add call to e2fsck_clear_progbar().
pass1.c (e2fsck_pass1):
pass2.c (e2fsck_pass2):
pass3.c (e2fsck_pass3):
pass4.c (e2fsck_pass4):
pass5.c (e2fsck_pass5): Add call to e2fsck_clear_progbar when printing
the resource tracking information.
pass5.c (check_block_bitmaps, check_inode_bitmaps): If there is an
error in the bitmaps, suppress printing the progress bar using the
suppression flag for the remainder of the check, in order to clean up
the display.
e2fsck.h:
pass1.c (pass1_get_blocks, pass1_read_inode, pass1_write_inode,
pass1_check_directory, e2fsck_use_inode_shortcuts): Make pass1_* be
private static functions, and create new function
e2fsck_use_inode_shortcuts which sets and clears the inode shortcut
functions in the fs structure.
e2fsck.h:
pass2.c (e2fsck_process_bad_inode): Make process_bad_inode() an
exported function.
pass4.c (e2fsck_pass4): Call e2fsck_process_bad_inode to check if a
disconnected inode has any problems before connecting it to
/lost+found. Bug and suggested fix by Pavel Machek <pavel@bug.ucw.cz>
ChangeLog, swapfs.c:
swapfs.c (ext2fs_swap_inode): Add compatibility for Linux 2.3 kernels
that use i_generation instead of i_version. Patch supplied by Jon
Bright <sircus@sircus.demon.co.uk>.
ChangeLog, mke2fs.8.in:
mke2fs.8.in: Fix typo in man page which caused the badblocks command
to not show up in the "SEE ALSO" section.
ChangeLog, expect.1, expect.2, image.gz, name:
f_recnect_bad: New test which checks the case where a disconnect inode
also bad inode fields; we need to make sure e2fsck offers to fix the
inode (or clear the inode, as necessary).
problem.c: Add PR_3_NO_DIRINFO error code.
super.c (check_super_value): Rename min and max to min_val and max_val
to avoid possible cpp macro conflicts.
pass4.c (e2fsck_pass4): Rename max to maxgroup, to avoid possible cpp
macro conflicts.
pass3.c (e2fsck_pass3): Rename max to maxdirs, to avoid possible cpp
macro conflicts.
(check_directory): Fix logic to avoid possible core dump in the
case of ext2fs_get_dir_info returning NULL. (By the time we get here,
it should never happen, but...). Also simply/streamline the control
flow of the function.
super.c, e2fsck.h: Always declare e2fsck_get_device_size() as an
extern in e2fsck.h to prevent -Wall warnings.
pass4.c (e2fsck_pass4): Remove unused variable 'j'.
super.c (check_super_block): Fix broken superblock sanity check when
calculating blocks_per_group if s_log_frag_size != s_log_block_size.
Since we don't support fragments, this isn't a bug deal, but it's good
to get it fixed.
ChangeLog, Makefile.elf-lib:
Makefile.elf-lib (installdirs): Change use of DLL_INSTALL_DIR to
ELF_INSTALL_DIR. (Doesn't make a difference currently, but it's
correct this way.)
ChangeLog, fgetversion.c, getversion.c:
getversion.c (getversion): Return 0 if there's no error.
fgetversion.c (fgetversion): Pass the address of the temporary
variable instead of the unitialized variable to the
EXT2_IOC_GETVERSION ioctl.
Change the progress function to return an integer; if returns 1, then
the progress function is expected to have set the e2fsck context flag
signalling a user abort, and the caller should also initiate a user
abort.
e2fsck.h: Add new field, priv_data to the e2fsck context structure.
It should be used by callers of the e2fsck functions only, and not by
anything in e2fsck itself.
super.c: Instead of call ext2fs_get_device_size(), define and call
e2fsck_get_device_size(). (This function may be stubbed out in
special versions of e2fsck.)
pass3.c, pass4.c: Remove extra calls to the progress function that
weren't needed.
mke2fs.8.in: Update man page to note that the format of the bad block
file is the same as the one generated by badblocks.
pass1.c, pass2.c, pass3.c, pass4.c, pass5.c: Add calls to the progress
indicator function.
pass1.c (scan_callback): Add call to the progress feedback function
(if it exists).
super.c (check_super_block): Skip the device size check if the
get_device_size returns EXT2_EXT_UNIMPLEMENTED.
iscan.c (main): Don't use fatal_error() anymore.
pass1b.c, swapfs.c, badblocks.c: Set E2F_FLAG_ABORT instead of calling
fatal_error(0).
problem.c, pass3.c (PR_3_ROOT_NOT_DIR_ABORT,
PR_3_NO_ROOT_INODE_ABORT): New problem codes.
problem.c, pass2.c (PR_2_SPLIT_DOT): New problem code.
problem.c, pass1.c (PR_1_SUPPRESS_MESSAGES): New problem code.
problemP.h: New file which separates out the private fix_problem data
structures.
util.c, dirinfo.c, pass1.c, pass1b.c, pass2.c, pass5.c, super.c,
swapfs.c util.c: allocate_memory() now takes a e2fsck context as its
first argument, and rename it to be e2fsck_allocate_memory().
problemP.h:
New file which contains the private problem abstraction definitions.
Makefile.pq:
Remove include of MAKEFILE.STD, which doesn't exist at this point.
Change ext2fs_read_inode, ext2fs_write_inode to take the e2fsck
context as their first argument.
Change dir_info.c routines to take an e2fsck_context, renamed them to
start with e2fsck_ to avoid namespace issues, and changed them to
store the directory information inside the e2fsck context.
Added e2fsck_run() which calls all of the e2fsck passes in the correct
order, and which handles the return of abort codes.
Added abort processing, both via setjmp/longjmp and via flags in the
e2fsck context.
Use a flag in the e2fsck context instead of the restart_e2fsck global
variable.
Change uses of free and malloc to ext2fs_free_mem and ext2fs_get_mem.
pass*.c, super.c: Massive changes to avoid using printf and com_err
routines. All diagnostic messages are now routed through the
fix_problem interface.
pass2.c (check_dir_block): Check for duplicate '.' and '..' entries.
problem.c, problem.h: Add new problem codes PR_2_DUP_DOT and
PR_2_DUP_DOT_DOT.
problem.c: Added new problem codes for some of the superblock
corruption checks, and for the pass header messages. ("Pass
1: xxxxx")
util.c (print_resource_track): Now takes a description argument.
super.c, unix.c, e2fsck.c: New files to separate out the
operating-specific operations out from e2fsck.c. e2fsck.c now
contains the global e2fsck context management routines, and
super.c contains the "pass 0" initial validation of the
superblock and global block group descriptors.
pass1.c, pass2.c, pass3.c, pass4.c, pass5.c, util.c: Eliminate
(nearly) all global variables and moved them to the e2fsck
context structure.
problem.c, problem.h: Added new problem codes PR_0_SB_CORRUPT,
PR_0_FS_SIZE_WRONG, PR_0_NO_FRAGMENTS, PR_0_BLOCKS_PER_GROUP,
PR_0_FIRST_DATA_BLOCK
expect.1, expect.2:
Updated tests to align with e2fsck problem.c changes.
Theodore Ts'o