The Build Log Hardening Check is a debian tool which scans the output
of a package build making sure that the security hardening flags are
used when compiling and linking all of binaries in a package.
For the most part we were passing CFLAGS, CPPFLAGS, and LDFLAGS down
to the compiler and link commands, but there there were one or two
exceptions. In addition, there where a few places in "make install"
where the V=1 option was not being honored, which triggered blhc
warnings since it couldn't analyze those commands.
The e2fsck.static was the only binary that was not getting built and
packaged with the hardening flags, but I've fixed all of the blhc
warnings so in the future it will be obvious if we regress.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
By the time we start processing the orphan inode list, we have already
calculated the total expected number of free blocks and inodes in
ctx->free_{blocks,inodes}. This is used to set the free blocks/inodes
count in the superblock in the case where we don't need to do a full
e2fsck.
We need to update these expected free block counts as we process the
orphan inode list so that superblock values are set correctly.
Otherwise we could have the following happen:
% e2fsck /tmp/test.img
e2fsck 1.42.3 (14-May-2012)
Truncating orphaned inode 12 (uid=0, gid=0, mode=0100644, size=0)
Setting free blocks count to 46 (was 79)
/tmp/test.img: clean, 12/16 files, 54/100 blocks
% e2fsck /tmp/test.img
e2fsck 1.42.3 (14-May-2012)
Setting free blocks count to 79 (was 46)
/tmp/test.img: clean, 12/16 files, 21/100 blocks
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The on-disk format for interior nodes in the extent tree does not
encode the length of each entry in the interior node; instead, it is
synthesized/simulated by the extent library code in libext2fs.
Unfortunately, this simulation is not perfect; in particular it does
not work for the last extent in the extent tree if there are
uninitialized blocks allocated using fallocate with
FALLOC_FL_KEEP_SIZE, and it leads to e2fsck incorrectly complaining
about an invalid zero-length extent.
We only need to worry about the extent length for the leaves of the
tree, since it is there were we are checking an on-disk value, as
opposed to a software-generated simulation. So restrict the check of
extent length to leaf nodes in the extent tree.
Reported-by: Tao Ma <tm@tao.ma>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If the file system is mounted read-only after a file system error has
been detected, the fact that an error occurred is written to the
journal. This is important because while the journal is getting
replayed, the error indication in the superblock may very well get
overwritten.
Unfortunately, the code to propagate the error indication from the
journal to superblock was broken because this was being done before
the old file system handle is thrown away and the file system is
re-opened to ensure that no stale data is in the file system handle.
As a result, the error indication in the superblock was never written
out.
To fix this, we need to move the check if the journal's error
indicator has been set after the file system has been freed and
re-open.
Reported-by: Ken Sumrall <ksumrall@google.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Quite some definitions in quota library are not necessary. Remove them.
Also fold quota.h file into quotaio.h since it didn't contain that many
definitions.
Signed-off-by: Jan Kara <jack@suse.cz>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If a file system was remounted read-only after a file system
corruption is detected, and then that file system is mounted and
unmounted by the kernel, the journal would have been recovered, but
the kernel currently leaves the s_errno field still set. This is
arguably a bug, since it has already propgated the non-zero s_errno
field to the file system superblock, where it will be retained until
e2fsck has been run.
However, e2fsck should handle this case for existing kernel by
checking the journal superblock's s_errno field even if journal
recovery is not required.
Without this commit, e2fsck would not notice anything wrong with the
file system, but a subsequent mount of the file system by the kernel
would mark the file system's superblock as needing checking (since the
journal's s_errno field would still be set), resulting an full e2fsck
run at the next reboot, which would find nothing wrong --- and then
when the file system was mounted, the whole cycle would repeat again.
I had seen reports of this in the past, but it wasn't until recently
that I realized exactly how this had come about, since normally e2fsck
would be run automatically before the file system is mounted again,
thus avoiding this problem. However, a user using a rescue CD who
didn't run e2fsck before mounting the a file system in this condition
could trigger this situation, and unfortunately, with previous
versions of e2fsprogs and the kernel, there would be no way out no
matter what the user tried to do.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Commit 732e26b98e added checks to
prevent e2fsck from being run in filesystem-modifying mode against
a mounted or otherwise busy device, due to several bug reports of
users doing this even with the verbose warnings in check_mount().
However, it also prevented e2fsck from checking a mounted root
filesystem, which will prevent the node from booting. Once again
allow e2fsck to run against the mounted root filesystem if it is
also mounted read-only at the time.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The creation of inline wrappers ext2fs_open_file() and ext2fs_stat()
in commit c859cb1de0 in ext2fs.h caused
difficulties with the use of headers, since the headers for open64()
and stat64() may already be included (and skip the declaration of the
64-bit variants) before ext2fs.h is ever read. There is no real way
to solve the missing prototypes and resulting compiler warnings inside
ext2fs.h.
Since ext2fs_open_file() and ext2fs_stat() are not performance
critical operations, they do not need to be inline functions at all,
and the needed function headers can be handled properly in one file.
Similarly, posix_memalloc() was having difficulties with headers, and
was being defined in ext2fs.h, but it is now only being used by a
single file, so move the required header there.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Currently, if e2fsck is run without the "-n" flag (i.e. it
might modify the filesystem), there is no guarantee that it will
open the filesystem with the EXCLUSIVE flag (i.e. O_EXCL) to
prevent the block device from being checked (in most cases this
means mounted, but it could also be an MD/LVM member device).
Conversely, if e2fsck is run with "-n" (i.e. read-only), and
/etc/mtab or /proc/mounts does not report the block device as
mounted then e2fsck thinks the filesystem is unmounted. In this
case, e2fsck incorrectly sets the EXCLUSIVE flag, which causes
the check to fail, even though e2fsck is running read-only.
To fix this, do not open with EXCLUSIVE if it is a read-only check,
and always open with EXCLUSIVE if the filesystem might be changed.
This also prevents filesystem mounts while e2fsck is running.
Also refuse allow e2fsck to run at all if the filesystem is BUSY.
The e2fsck check_mount() was checking for MOUNTED, but not BUSY,
and it should refuse to run outright if the block device is BUSY.
The previous MOUNTED heuristics pre-date the O_EXCL reservation
by the kernel, so there could be uncertainty due to stale /etc/mtab
data, but with newer kernels a busy device should never be modified.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The operator precedence bug means that we might pay atteion to
s_grp_quota_inum even if the RO_COMPAT_QUOTA feature flag is clear.
However, fortunately, this is unlikely to happen in practice.
Cc: Aditya Kali <adityakali@google.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
64-bit journal support was broken; we weren't using the high bits from
the journal descriptor blocks! We were also using "unsigned long" for
the journal block numbers, which would be a problem on 32-bit systems.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Don't consider only an error in the superblock summary as incorrect.
The kernel does not update this field except at unmount time, so
don't print errors during a "-n" run if there is nothing else wrong.
Any other unfixed errors will themselves mark the filesystem invalid.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Currently fsck recomputes quotas and overwrites quota files
whenever its run. This causes unnecessary modification of
filesystem even when quotas were never inconsistent. We also
lose the limits information because of this. With this patch,
e2fsck compares the computed quotas to the on-disk quotas
(while updating the in-memory limits) and writes out the
quota inode only if it is inconsistent.
Signed-off-by: Aditya Kali <adityakali@google.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Currently e2fsck always incorrectly detects that quota inodes
need to be hidden (even if they are already hidden) and
modifies the superblock unnecessarily. This patch fixes the
check for hidden quota files and avoids modifying the
filesystem if quota inodes are already hidden.
Also, zero-out the old quota inode so that next fsck scan
doesn't complain.
Signed-off-by: Aditya Kali <adityakali@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
We have renamed buggy_init_scripts to accept_time_fudge. Explain this
so that people who find buggy_init_scripts in older e2fsck.conf files
understand what is going on.
Addresses-Debian-Bug: #646963
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Change autoconf to test for setmntent() and use that to decide whether
to use getmntent() and setmntent(), since some systems don't have
setmntent() but they do have the mntent.h header file.
Also, remove the includes of mntent.h from e2fsck and mke2fs and other
places where it is not needed.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
We've decided to remove EOFBLOCKS_FL from the ext4 file system entirely,
because it is not actually very useful and it is causing more problems
than it solves. We're going to remove it from e2fsprogs first and then
after the new e2fsprogs version is common enough we can remove the
kernel part as well.
This commit changes e2fsck to not check for EOFBLOCKS_FL. Instead we
simply search for initialized extents past the i_size as this should not
happen. Uninitialized extents can be past the i_size as we can do
fallocate with KEEP_SIZE flag.
Also remove the EXT4_EOFBLOCKS_FL from lib/ext2fs/ext2_fs.h since it is
no longer needed.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Print the actual errors returned by ext2fs_open2() and
ext2fs_check_desc() before we fall back to the backup block group
descriptors so that it's easier to see if there is some obscure
failure that is causing e2fsck to think that it should use the backup
block group descriptors.
Addresses-Google-Bug: #6208183
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
For file systems that do not use MMP, there's no reason to close the
file system and then re-open the file system a second time, since
EXT2_FLAG_SKIP_MMP has no meaning for non-MMP file systems anyway.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: Andreas Dilger <adilger@whamcloud.com>
We were checking for ENOMEM, but in fact if the malloc() fails,
ext2fs_check_desc() will return EXT2_ET_NO_MEMORY.
Addresses-Google-Bug: #6208183
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add the ability to log messages about a file system to a specified
directory, using a file name templace that can be specified in
/etc/e2fsck.conf. This allows us to suppress the output of overly
verbose e2fsck outputs while still allowing the full logging output to
go to an appropriate file.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Also add appropriate documentation for options/max_count_problems and
problems/0xXXXXXX/max_count settings in /etc/e2fsck.conf
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This throttles the output of a particular problem type, to avoid a
bottleneck caused by (for example) printing a large number of
characters over a rate-limited a serial console.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If an extent has e_len set to zero, the kernel will oops with a
BUG_ON. Unfortunately, e2fsck wasn't catching this case. The kernel
needs to be fixed to notice this case and call ext4_error() instead of
failing an assertion check, but e2fsck should catch this case and
repair it (by deleting the errant extent).
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
We do not want to discard inode table if the underlying device does not
return zeros when reading non-provisioned blocks. The reason is that if
the inode table is not zeroed yet, then discard would not help us since
we would have to zero it anyway. In the case that inode table was
already zeroed, then the discard would cause subsequent reads to contain
non-deterministic data so we would not be able to assume that the inode
table was zeroed and we would need to zero it again, which does not
really make sense.
This commit adds check to prevent inode table from being discarded if
the discard does not zero data.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
When argument '-n' was specified and should run in read-only mode, we
should not attempt to discard anything. In order to do that we have to
check for E2F_OPT_NO flag and clear E2F_OPT_DISCARD flag if E2F_OPT_NO
is set.
This commit fixes the problem when we would mark inode tables as zeroed
(EXT2_BG_INODE_ZEROED) even when e2fsck is running in read-only mode. We
also move the check for E2F_OPT_NO so we can clear E2F_OPT_DISCARD as
early as possible.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The boolean expression (!skip_group || csum_flag) is always true,
since if csum_flag is FALSE, skip_group must also be FALSE. Hence, we
can just remove the expression from the conditional altogether, thus
simplifying the code and making it easier to read/understand.
Also, in the case where the bit is set in the bitmap, there's no point
repeatedly setting first_free to be ext2fs_block_count(fs->super).
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Previously when running e2fsck with '-E discard' argument the end of
the last group has not been discarded. This patch fixes it so we
always discard the end of the last group if needed.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
When calling e2fsck with '-E discard' option it might happen that
valid inodes are discarded accidentally. This is because we just
discard the part of inode table which lies past the free inode count.
This is terribly wrong (sorry!).
This patch fixes it so only the free parts of an inode table
is discarded, leaving used inodes intact. This was tested with highly
fragmented inode tables with block size 4k and 1k.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Reported-by: Phillip Susi <psusi@ubuntu.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Removing this check will allow us to eventually eliminate code from
the kernel which forcibly initialized the block bitmap when the inode
bitmap is first used. This would eliminate a required journal credit
and extra disk write.
Addresses-Google-Bug: #5944440
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Now that we have multiple backend implementations of the bitmap code,
this commit teaches e2fsck to use either the most appropriate backend
for each use case.
Since we don't know for sure if we will get it all right, the default
choices can be overridden via e2fsck.conf. The various definitions
are shown here, with the current defaults (which may change as we add
more bitmap implementations and as learn what works better).
; EXT2FS_BAMP64_BITARRAY is 1
; EXT2FS_BMAP64_RBTREE is 2
; EXT2FS_BMAP64_AUTODIR is 3
[bitmaps]
inode_used_map = 2 ; pass1
inode_dir_map = 3 ; pass1
inode_reg_map = 2 ; pass1
block_found_map = 2 ; pass1
inode_bad_map = 2 ; pass1
inode_imagic_map = 2 ; pass1
block_dup_map = 2 ; pass1
block_ea_map = 2 ; pass1
inode_link_info = 2 ; pass1
inode_dup_map = 2 ; pass1b
inode_done_map = 3 ; pass3
inode_loop_detect = 3 ; pass3
fs_bitmaps = 2
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The pass5 checks would fail if the expected and current {inode,block}
bitmaps used different back ends that returned different non-zero
values from the test_*_bitmap() functions. Fix this by changing
"(actual == bitmap)" to "(!actual == !bitmap)".
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Optimize how the tdb library so that running with [scratch_files] in
/etc/e2fsck.conf is more efficient. Use a better hash function,
supplied by Rogier Wolff, and supply an estimate of the size of the
hash table to tdb_open instead of using the default (which is way too
small in most cases). Also, disable the tdb locking and fsync calls,
since it's not necessary for our use in this case (which is
essentially as cheap swap space; the tdb files do not contain
persistent data.)
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
We need to do an accounting of duplicate clusters on a per-cluster
instead of a per-block basis so we know when we've correctly accounted
for all of the multiply claimed blocks in a particular inode.
Thanks to Robin Dong for reporting this bug.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
PATH_MAX is not portable (for example, it doesn't exist on the Hurd).
So replace it with a new define, which defines the maximum length of
the base quota name. As it turns out, this is substantially smaller
than PATH_MAX.
Also move the definitions relating to quotaio.c from mkquota.h to
quotaio.h, as a cleanup.
Addresses-Debian-Bug: #649689
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
When I create a non-extent file with the maximum size in ext4,
e2fsck detects the following error:
Pass 1: Checking inodes, blocks, and sizes
Inode 12, i_size is 4402345721856, should be 4402345721856. Fix?
As we know, e2fsck checks the size field of the inode in pass 1.
However, in case of the ext4 with the feature of ^extent and huge_file,
the maximum file size calculated in e2fsck is less than the real one.
The patch fixes this problem.
Signed-off-by: Kazuya Mio <k-mio@sx.jp.nec.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If we abort fsck (due to ENOMEM for example) we exit
with only the FSCK_ERROR flag. It seems useful
to do the same sorts of checks as we do on normal
exit, and return whether the filesystem was modified,
whether there are still uncorrected errors, etc, even
in the abort case.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
This patch cleans up the quota code as suggested in previous reviews. This
includes
* remove BUG_ON()s and 'exit()' calls from library code
* remove calls to malloc/free and instead use ext2fs_get/free_mem functions.
* lib/quota/common.c file in not needed anymore and is removed.
* rename exported functions to start with quota_
(ex: init_quota_context --> quota_init_context)
* better error handling in quota library
Signed-off-by: Aditya Kali <adityakali@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The ext2fs_file_acl_block() and ext2fs_set_file_acl_block() needs to
only check i_file_acl_high if the 64-bit flag is set. This is needed
because otherwise we will run into problems on Hurd systems which
actually use that field for h_i_mode_high.
This involves an ABI change since we need to pass ext2_filsys to these
functions. Fortunately these functions were first included in the
1.42-WIP series, so it's OK for us to change them now. (This is why
we have 1.42-WIP releases. :-)
Addresses-Sourceforge-Bug: #3379227
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Since the advent of 64bit filesystems, revoke blocks store 64-bit
block numbers instead of 32-bit block numbers. Therefore we need to
be able to handle that case.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o