Vitaliy Filippov
3ebbfa0428
Fix another rare OSD hang on zeroing out entries on start
2024-03-16 13:24:36 +03:00
Vitaliy Filippov
ff74b19423
Fix rare OSD hang on zeroing out bad entries on start
2024-03-06 00:41:35 +03:00
Vitaliy Filippov
f20564b44b
Fix 32-bit build warnings (99.9% in printf)
2024-02-22 12:22:16 +03:00
Vitaliy Filippov
8b8c1179a7
Use a separate used_blocks counter for free space stats to hide possibly delayed on-flush deallocation
2023-12-31 01:23:17 +03:00
Vitaliy Filippov
6d5df908a3
Fix possible out of bounds when checking invalid journal entries
2023-12-08 00:01:07 +03:00
Vitaliy Filippov
31017d8412
Allow to start with V2 journal with header size from V1, as incorrectly updated by previous versions
2023-11-04 18:13:42 +03:00
Vitaliy Filippov
a8464c19af
Support keeping checksums on disk (not in memory)
...
Definitely beneficial for SSD+HDD setups
2023-07-29 12:17:18 +03:00
Vitaliy Filippov
ddb078d5a7
Check journal entry size when checking block checksums
2023-07-29 12:17:18 +03:00
Vitaliy Filippov
d22d56f90a
Fix journal data checksum verification on start
2023-07-29 12:17:18 +03:00
Vitaliy Filippov
eb1331a079
Add more details to "journal entry data is corrupt" messages
2023-07-29 12:17:18 +03:00
Vitaliy Filippov
c5274f655b
...and partially remove the perversion with bitmap inlining
2023-07-29 12:17:18 +03:00
Vitaliy Filippov
45e07d6294
Sadly we have to refcount dyn_data...
2023-07-29 12:17:18 +03:00
Vitaliy Filippov
de48fa3fd2
Allow to forcibly set meta_format
2023-07-29 12:17:18 +03:00
Vitaliy Filippov
874a766b62
Rename meta_version to meta_format
2023-07-29 12:17:18 +03:00
Vitaliy Filippov
213a9ccb4d
Verify checksums during journal reads
2023-07-29 12:17:18 +03:00
Vitaliy Filippov
a166147110
Add backwards compatibility with non-checksum metadata and journal formats
2023-07-29 12:17:18 +03:00
Vitaliy Filippov
7d532880c3
Implement large csum_block_size support (more than 4k) + refactor blockstore_flush
2023-07-29 12:17:18 +03:00
Vitaliy Filippov
0b0405d115
Implement bitmap-granular (4k) metadata & data checksums
2023-07-29 12:17:18 +03:00
Vitaliy Filippov
b457327e77
Oops. Fix metadata read after fixes :-)
2022-12-17 17:31:57 +03:00
Vitaliy Filippov
49b88b01f9
Fix clang build
2022-12-17 16:25:26 +03:00
Vitaliy Filippov
5464821fa5
Final fix for the lack of zeroing out of old metadata entries
...
If a crash occurs during flushing a redirect-write it may happen so that
the disk contains both old and new metadata entries. This is OK, but prior
to 0.8.0 after this situation OSDs started without problem, but then they
crashed after some more overwrites with a "tried to overwrite non-zero
metadata entry" error. 0.8.0 introduced a change that was intended to fix
this situation, but rather than fixing it it prevented OSDs from starting,
now because of a "big_write journal_entry was allocated over another object"
error... :-)
This change finally fixes the original issue.
Followup to 54ef2c389f
2022-12-17 14:50:31 +03:00
Vitaliy Filippov
54ef2c389f
Followup to the "tried to overwrite" fix: also handle it in case of inmemory_meta == false
2022-08-21 01:28:29 +03:00
Vitaliy Filippov
153c73574a
Refactor blockstore_init_meta into slightly more obvious code
2022-08-21 01:21:13 +03:00
Vitaliy Filippov
d83580bd68
Fix "tried to overwrite non-zero metadata entry" when during a previous metadata
...
flush writing new entry is completed, but zeroing out an old one isn't
2022-08-21 00:31:18 +03:00
Vitaliy Filippov
ea632367e9
Do not alter dsk.meta_offset/len to skip superblock
2022-07-15 01:38:30 +03:00
Vitaliy Filippov
dfd80626bd
Extract disk opening functions to separate module
2022-07-15 01:38:30 +03:00
Vitaliy Filippov
73a363bf92
Rename some variables and constants
2022-07-15 01:38:30 +03:00
Vitaliy Filippov
bce357e2a5
Do not read all metadata into memory when dumping
2022-06-13 01:26:30 +03:00
Vitaliy Filippov
088dd15449
Exclude empty inodes from stats
2022-06-04 00:18:17 +03:00
Vitaliy Filippov
e718116f54
Fix incorrect reading of extra metadata block
2022-04-21 02:52:21 +03:00
Vitaliy Filippov
839ec9e6e0
Shard clean_db by PGs to speedup listings
2022-02-20 00:21:24 +03:00
Vitaliy Filippov
7bdd92ca4f
Fix build under clang and some warnings
...
Build problems fixed:
- void* pointer arithmetic which is a GNU extension (works as byte*)
- "variable size object may not be initialized" which is OK under GCC
- nullptr_t related error in json11 (it lacks 'operator <' in clang)
Warnings fixed:
- empty nested struct initializer { 0 } replaced by {}
- removed several unused lambda captures
2022-01-16 00:02:54 +03:00
Vitaliy Filippov
2a02f3c4c7
Add metadata superblock and check it on start
...
Refuse to start if the superblock is missing or bad version;
zero out the metadata area when initializing superblock.
2021-04-10 22:26:17 +03:00
Vitaliy Filippov
f684d9101a
Refuse to start with old journal version
2021-04-10 17:44:12 +03:00
Vitaliy Filippov
a1f2f19489
Do not increment inode statistics if the object already exists
2021-04-10 17:44:12 +03:00
Vitaliy Filippov
591cad09c9
Fix bitmaps for objects larger than 128K
2021-04-10 17:44:12 +03:00
Vitaliy Filippov
ab39ce2bbb
Use clean_entry_bitmap_size instead of entry_attr_size back because of changed bitmap handling
2021-04-10 17:44:12 +03:00
Vitaliy Filippov
c35963967f
Add inode space usage statistics tracking to blockstore
2021-04-10 17:44:12 +03:00
Vitaliy Filippov
6107a4d07b
Add "external" bitmap support to blockstore
2021-04-10 17:44:12 +03:00
Vitaliy Filippov
307c1731c1
Forget all dirty_entries before stable big_write or delete during initialisation
...
This fixes a 'double_alloc' assertion in the following case:
- big_write object #1 v1 to block #100
- big_write object #1 v2 to block #101
- big_write object #2 v1 to block #100
2021-04-07 01:30:38 +03:00
Vitaliy Filippov
ad9f619370
Skip double allocs when reading journal
2021-04-03 00:53:28 +03:00
Vitaliy Filippov
f4769ba7c7
Collapse create+delete journal entry pairs if they're already flushed
...
Old journal replay mechanism could lead to a double allocation of the same
block and a "Fatal error: tried to overwrite non-zero metadata entry"
2021-04-03 00:53:28 +03:00
Vitaliy Filippov
843b7052d2
Add an assertion when clearing deleted metadata entries, add debug details when freeing blocks
2021-04-03 00:53:28 +03:00
Vitaliy Filippov
4095bcc558
Do not ignore object deletion journal entries when they are preceded by a big write
2021-03-25 11:00:10 +03:00
Vitaliy Filippov
564d64e271
Add some details for debug prints
2021-03-25 11:00:10 +03:00
Vitaliy Filippov
bf9a175efc
Move C/C++ sources to src subdirectory
2021-02-25 23:59:03 +03:00
Vitaliy Filippov