1458 Commits (7333022adfc7905d96e8be3b7b5a977f85a567c9)

All Branches

Search

Author	SHA1	Message	Date
Vitaliy Filippov	7333022adf	Add a third I/O mode: O_DIRECT|O_SYNC, change parameters to data_io/meta_io/journal_io	2023-08-09 00:57:08 +03:00
Vitaliy Filippov	ab8627c9fa	Fix monitor retrying failed etcd connection in an infinite loop without pauses	2023-08-09 00:57:08 +03:00
Vitaliy Filippov	6acf562e01	Release 1.0.0 ... New features: - Data and metadata checksums! - Metadata checksums are always used with new disk format - Data checksums can be turned on with --data_csum_type crc32c for new OSDs - Checksum block size can be configured - inmemory_metadata now also affects keeping checksums in memory - Linux page cache I/O caching support which can be enabled separately for data, metadata (including checksums) and journal (O_SYNC instead of O_DIRECT) - Details [here](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/config/layout-osd.en.md#data_csum_type) - Backwards compatibility is preserved, you can use new OSDs with old disks Release also includes bug fixes from [0.9.6](https://git.yourcmc.ru/vitalif/vitastor/releases/tag/v0.9.6). 0.9.6 is moved to "-oldstable" repositories and will be available for some additional time.	2023-07-29 18:57:19 +03:00
Vitaliy Filippov	6f797f429e	Add a note about -oldstable	2023-07-29 18:11:13 +03:00
Vitaliy Filippov	b8a1734465	Reword checksum docs one more time	2023-07-29 14:42:56 +03:00
Vitaliy Filippov	c752b68167	Remove "without checksums" from docs :)	2023-07-29 12:19:03 +03:00
Vitaliy Filippov	564df2eb5d	Support using buffered I/O with O_SYNC instead of direct I/O	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	9a427dd70a	Allow to override OSD devices in tests	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	1a4ceb420d	Track used blocks, not object versions	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	21b5124a4b	Document data_csum_type and csum_block_size parameters	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	4181add1f4	Remove creepy "metadata copying" during overwrite ... Instead of it, just do not verify checksums of currently mutated objects. When clean data modification during flush runs in parallel to a read request, that request may read a mix of old and new data. It may even read a mix of multiple flushed versions if it lasts too long... And attempts to verify it using temporary copies of metadata make the algorithm too complex and creepy.	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	a8464c19af	Support keeping checksums on disk (not in memory) ... Definitely beneficial for SSD+HDD setups	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	819cb70cdd	Check for "Checksum mismatch" and "BUG" messages during test_heal	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	3c8e4c6b72	Use clean_dyn_size for space check	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	8ef4cf89dc	Log more details about checksum mismatch in big_writes	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	7bfb1639ea	Use find_holes() in flusher for unification	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	628e481c32	Fill journal header to know checksum type & size when dumping journal with --all	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	af6f2046fc	Fix journal read checksum verification with inmemory_journal=false	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	9357e5293e	Call fill_partial_checksum_blocks() correctly in regard to COPY_BUF_CSUM_FILL	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	12851dc07d	Wait for journal reads before checking them in clear_incomplete_csum_block_bits	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	a5753e35a3	Check for checksum mismatch absence in test_heal	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	d6ee1ca17c	Use zero checksum size for zero-length writes	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	71674d00cf	Fix journal data checksum mangling on corrupted block overwrite	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	ddb078d5a7	Check journal entry size when checking block checksums	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	d22d56f90a	Fix journal data checksum verification on start	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	eb1331a079	Add more details to "journal entry data is corrupt" messages	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	c5274f655b	...and partially remove the perversion with bitmap inlining	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	45e07d6294	Sadly we have to refcount dyn_data...	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	a8ee391e05	Fix clean block checksum read	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	de48fa3fd2	Allow to forcibly set meta_format	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	874a766b62	Rename meta_version to meta_format	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	384bd8e28f	Support old metadata format in vitastor-disk dump-meta	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	430994f48a	Fix journal big_write simple reads after checksum changes	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	3d7f838c59	Verify checksums in test_heal in different combinations	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	b909d81f41	Fix bitmap-granular checksums	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	e42975ffd1	Fix wait_journal_count not being zeroed	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	93778324e5	Rewrite and fix find_holes into a more obvious version	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	eeb6727170	Fix missing checksum read offset	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	7fe82c692e	Add a test for checksums	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	92c6e16eba	Fix checksum verification in big_write journal reads	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	213a9ccb4d	Verify checksums during journal reads	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	a166147110	Add backwards compatibility with non-checksum metadata and journal formats	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	7d532880c3	Implement large csum_block_size support (more than 4k) + refactor blockstore_flush	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	0b0405d115	Implement bitmap-granular (4k) metadata & data checksums	2023-07-29 12:17:18 +03:00
Vitaliy Filippov	e651c93a90	Release 0.9.6 ... - Fix vitastor-disk partition zeroing (sometimes it was writing garbage instead of zeroes) - Fix incorrect EC space statistics in `vitastor-cli status` - Several bug fixes for NFS: - Add . and .. in NFS directory listings - Return FILE_SYNC from NFS writes if immediate_commit is enabled - Return the same "verifier" in NFS COMMIT as in NFS WRITE - Make parallel NFS extending writes work correctly, without conflicts - Handle parallel NFS extending writes without imposing extra load on etcd - Support UTF-8 in vitastor-cli table output - Also allow "0" and "no" as false for inmemory_metadata and inmemory_journal - Use HDD defaults for HDD-only in automatic `vitastor-disk prepare` mode	2023-07-29 10:54:00 +03:00
Vitaliy Filippov	988e90be69	Fix vitastor-disk partition zeroing (it was writing random garbage instead of zeroes :D)	2023-07-28 12:29:07 +03:00
Vitaliy Filippov	272a45ad63	Fix modprobe command in docs	2023-07-27 23:57:02 +03:00
Vitaliy Filippov	25a15d24cf	Fix incorrect EC space statistics in `vitastor-cli status`	2023-07-27 02:26:17 +00:00
Vitaliy Filippov	700e0e9bff	Handle parallel NFS extending writes without imposing extra load on etcd	2023-07-27 02:26:17 +00:00
Vitaliy Filippov	ab0ca7c00f	Return FILE_SYNC from NFS writes if immediate_commit is enabled	2023-07-26 02:09:47 +03:00