vitalif
/
vitastor
5
4
Fork 9
Code Issues 1 Pull Requests 2 Actions Releases 69 Wiki Activity
1,783 Commits
44 Branches
69 Tags
14 MiB
Commit Graph

25 Commits (master)

Author SHA1 Message Date
Vitaliy Filippov f20564b44b Fix 32-bit build warnings (99.9% in printf) 2024-02-22 12:22:16 +03:00
Vitaliy Filippov 7048228678 Supposed fix for "BUG: Attempt to overwrite used offset" 2024-02-20 15:56:48 +03:00
Vitaliy Filippov ea73857450 Add asserts to catch "BUG: Attempt to overwrite used offset" 2024-02-20 15:56:48 +03:00
Vitaliy Filippov 5d3317e4f2 Followup to 1.4.2 write stall fix - sadly, the previous version was not working correctly :)
Test / test_move_reappear (push) Successful in 19s Details
Test / test_snapshot_chain (push) Successful in 1m21s Details
Test / test_snapshot_down (push) Successful in 23s Details
Test / test_snapshot_chain_ec (push) Successful in 1m50s Details
Test / test_snapshot_down_ec (push) Successful in 22s Details
Test / test_splitbrain (push) Successful in 16s Details
Test / test_etcd_fail (push) Successful in 6m42s Details
Test / test_rebalance_verify_imm (push) Successful in 2m19s Details
Test / test_rebalance_verify (push) Successful in 4m7s Details
Test / test_switch_primary (push) Successful in 36s Details
Test / test_write (push) Successful in 35s Details
Test / test_rebalance_verify_ec (push) Successful in 4m6s Details
Test / test_write_no_same (push) Successful in 22s Details
Test / test_write_xor (push) Successful in 1m34s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 6m7s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m7s Details
Test / test_heal_csum_32k_dj (push) Successful in 4m59s Details
Test / test_heal_csum_32k (push) Successful in 5m4s Details
Test / test_heal_csum_4k_dmj (push) Successful in 5m59s Details
Test / test_scrub (push) Successful in 1m9s Details
Test / test_scrub_zero_osd_2 (push) Successful in 37s Details
Test / test_scrub_xor (push) Successful in 52s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m5s Details
Test / test_heal_csum_4k_dj (push) Successful in 5m12s Details
Test / test_heal_csum_4k (push) Successful in 5m1s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m48s Details
Test / test_scrub_ec (push) Successful in 19s Details
Test / test_interrupted_rebalance (push) Successful in 1m38s Details
Test / test_heal_pg_size_2 (push) Successful in 3m20s Details
Test / test_heal_ec (push) Successful in 3m3s Details
2024-02-08 19:34:29 +03:00
Vitaliy Filippov 1cec62d25d Sync only completed writes
Test / test_move_reappear (push) Successful in 21s Details
Test / test_rm (push) Successful in 16s Details
Test / test_snapshot_down (push) Successful in 25s Details
Test / test_snapshot_down_ec (push) Successful in 35s Details
Test / test_splitbrain (push) Successful in 24s Details
Test / test_interrupted_rebalance (push) Successful in 5m14s Details
Test / test_snapshot_chain (push) Successful in 2m50s Details
Test / test_rebalance_verify_imm (push) Successful in 2m47s Details
Test / test_rebalance_verify (push) Successful in 3m42s Details
Test / test_switch_primary (push) Successful in 33s Details
Test / test_write (push) Successful in 42s Details
Test / test_write_xor (push) Successful in 44s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 2m52s Details
Test / test_write_no_same (push) Successful in 15s Details
Test / test_rebalance_verify_ec (push) Successful in 4m19s Details
Test / test_heal_ec (push) Successful in 6m20s Details
Test / test_heal_csum_32k (push) Successful in 3m29s Details
Test / test_scrub (push) Successful in 1m24s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m11s Details
Test / test_heal_csum_4k_dmj (push) Successful in 4m23s Details
Test / test_scrub_xor (push) Successful in 1m9s Details
Test / test_heal_csum_4k_dj (push) Successful in 5m29s Details
Test / test_heal_csum_4k (push) Successful in 5m36s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m53s Details
Test / test_scrub_ec (push) Successful in 29s Details
Test / test_heal_pg_size_2 (push) Successful in 3m9s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m13s Details
Test / test_heal_csum_32k_dj (push) Successful in 4m17s Details
Test / test_snapshot_chain_ec (push) Successful in 1m25s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Failing after 24s Details 
Should be a final remaining fix to EC + non-capacitor (non-immediate-commit) write hangs :).

First it was breaking non-EC ("instantly stable") writes because they sometimes
complete out of order which was leading to the following error:

terminate called after throwing an instance of 'std::runtime_error'
  what():  BUG: Unexpected dirty_entry 1000000000001:29480000 v65540 unstable state during flush: 0x151

But it is easily fixed by scanning previous and next dirty_entries in mark_stable.
 2024-01-27 15:17:22 +03:00
Vitaliy Filippov c5274f655b ...and partially remove the perversion with bitmap inlining 2023-07-29 12:17:18 +03:00
Vitaliy Filippov 45e07d6294 Sadly we have to refcount dyn_data... 2023-07-29 12:17:18 +03:00
Vitaliy Filippov 7d532880c3 Implement large csum_block_size support (more than 4k) + refactor blockstore_flush 2023-07-29 12:17:18 +03:00
Vitaliy Filippov 0b0405d115 Implement bitmap-granular (4k) metadata & data checksums 2023-07-29 12:17:18 +03:00
Vitaliy Filippov b7e4d0c9bf Fix journal dirty_start position tracking and some debug prints 
Fixes two bugs found during HDD testing :-)
1) OSD crashed with "BUG: Attempt to overwrite used offset of the journal" during
   `fio -bs=900k -iodepth=128` test with 16 MB journal
2) OSD stalled during `fio -bs=512k -iodepth=128` test with 64 MB journal
 2023-07-09 01:17:55 +03:00
Vitaliy Filippov 150968070f Slightly improve some debug prints
Test / test_change_pg_count (push) Successful in 30s Details
Test / test_change_pg_count_ec (push) Successful in 31s Details
Test / test_change_pg_size (push) Successful in 7s Details
Test / test_create_nomaxid (push) Successful in 7s Details
Test / test_etcd_fail (push) Successful in 45s Details
Test / test_failure_domain (push) Successful in 8s Details
Test / test_interrupted_rebalance (push) Successful in 1m3s Details
Test / test_interrupted_rebalance_imm (push) Successful in 55s Details
Test / test_interrupted_rebalance_ec (push) Successful in 1m30s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 57s Details
Test / test_minsize_1 (push) Successful in 20s Details
Test / test_move_reappear (push) Successful in 16s Details
Test / test_rebalance_verify (push) Successful in 1m49s Details
Test / test_rebalance_verify_imm (push) Successful in 1m40s Details
Test / test_rebalance_verify_ec (push) Successful in 2m4s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 1m51s Details
Test / test_rm (push) Successful in 14s Details
Test / test_snapshot (push) Successful in 16s Details
Test / test_snapshot_ec (push) Successful in 17s Details
Test / test_splitbrain (push) Successful in 14s Details
Test / test_write (push) Successful in 41s Details
Test / test_write_xor (push) Successful in 49s Details
Test / test_write_no_same (push) Successful in 10s Details
Test / test_heal_pg_size_2 (push) Successful in 2m58s Details
Test / test_scrub (push) Successful in 23s Details
Test / test_scrub_zero_osd_2 (push) Successful in 19s Details
Test / test_scrub_xor (push) Successful in 17s Details
Test / test_scrub_pg_size_3 (push) Successful in 24s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 28s Details
Test / test_scrub_ec (push) Successful in 25s Details
2023-05-29 01:04:16 +03:00
Vitaliy Filippov a409598b16 Wait for free space again, but count on big_write flushes instead of just flusher activity 2023-05-10 01:51:02 +03:00
Vitaliy Filippov 0fbf4c6a08 Selectively sync nonsynced objects on STABILIZE/ROLLBACK (fix for github issue #51) 2023-04-08 02:44:02 +03:00
Vitaliy Filippov d7bd36dc32 Fix another rare journal flush stall 2022-12-30 02:03:33 +03:00
Vitaliy Filippov 238037ae31 Make journal trimmer wait until reads are completed when inmemory_journal is false 
Without this new writes may in theory overwrite journal data being read at that time
 2022-11-20 01:49:21 +03:00
Vitaliy Filippov dfd80626bd Extract disk opening functions to separate module 2022-07-15 01:38:30 +03:00
Vitaliy Filippov f93491bc6c Implement journal write batching and slightly refactor journal writes 
Slightly reduces WA. For example, in 4K T1Q128 replicated randwrite tests
WA is reduced from ~3.6 to ~3.1, in T1Q64 from ~3.8 to ~3.4.

Only effective without no_same_sector_overwrites.
 2021-12-16 00:27:17 +03:00
Vitaliy Filippov ab39ce2bbb Use clean_entry_bitmap_size instead of entry_attr_size back because of changed bitmap handling 2021-04-10 17:44:12 +03:00
Vitaliy Filippov 6107a4d07b Add "external" bitmap support to blockstore 2021-04-10 17:44:12 +03:00
Vitaliy Filippov ad9f619370 Skip double allocs when reading journal 2021-04-03 00:53:28 +03:00
Vitaliy Filippov 843b7052d2 Add an assertion when clearing deleted metadata entries, add debug details when freeing blocks 2021-04-03 00:53:28 +03:00
Vitaliy Filippov 18a5fafa2a Fix rollback 2021-03-25 02:41:58 +03:00
Vitaliy Filippov 299d7d7c95 Use common macro for get_sqe 2021-03-13 17:19:45 +03:00
Vitaliy Filippov 98f1e2c277 Rework write/sync ordering 
Make syncs wait for all previous writes because it's the only way
to make sure that OSDs do not receive incomplete writes in LIST results
during peering when some writes are still in progress.

Also simplify blockstore submission queue logic.
 2021-03-08 17:04:10 +03:00
Vitaliy Filippov bf9a175efc Move C/C++ sources to src subdirectory 2021-02-25 23:59:03 +03:00