Compare commits

..

181 Commits

Author SHA1 Message Date
Vitaliy Filippov 2c0801f6e4 Configure ESLint and add it to CI
Test / test_snapshot_chain_ec (push) Successful in 3m0s Details
Test / test_rebalance_verify_imm (push) Successful in 3m20s Details
Test / test_root_node (push) Successful in 10s Details
Test / test_rebalance_verify (push) Successful in 3m50s Details
Test / test_switch_primary (push) Successful in 40s Details
Test / test_write (push) Successful in 41s Details
Test / test_write_no_same (push) Successful in 18s Details
Test / test_write_xor (push) Successful in 1m5s Details
Test / test_rebalance_verify_ec (push) Successful in 4m38s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m17s Details
Test / test_heal_pg_size_2 (push) Successful in 3m25s Details
Test / test_heal_ec (push) Successful in 4m46s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m38s Details
Test / test_heal_csum_32k_dj (push) Successful in 6m16s Details
Test / test_heal_csum_32k (push) Successful in 6m45s Details
Test / test_osd_tags (push) Successful in 27s Details
Test / test_heal_csum_4k_dmj (push) Successful in 7m12s Details
Test / test_enospc (push) Successful in 2m6s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m34s Details
Test / test_enospc_imm (push) Successful in 1m43s Details
Test / test_heal_csum_4k (push) Successful in 6m23s Details
Test / test_enospc_xor (push) Successful in 1m57s Details
Test / test_enospc_imm_xor (push) Successful in 1m0s Details
Test / test_scrub (push) Successful in 32s Details
Test / test_scrub_zero_osd_2 (push) Successful in 31s Details
Test / test_scrub_xor (push) Successful in 33s Details
Test / test_nfs (push) Successful in 18s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 27s Details
Test / test_scrub_ec (push) Successful in 28s Details
Test / test_scrub_pg_size_3 (push) Successful in 57s Details
2024-04-16 02:39:31 +03:00
Vitaliy Filippov fd83fef1d9 Fix pool deletion
Test / test_snapshot_chain_ec (push) Successful in 3m1s Details
Test / test_rebalance_verify_imm (push) Successful in 3m11s Details
Test / test_root_node (push) Successful in 9s Details
Test / test_rebalance_verify (push) Successful in 3m53s Details
Test / test_switch_primary (push) Successful in 39s Details
Test / test_write (push) Successful in 39s Details
Test / test_write_no_same (push) Successful in 18s Details
Test / test_write_xor (push) Successful in 1m9s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m53s Details
Test / test_rebalance_verify_ec (push) Successful in 4m33s Details
Test / test_heal_pg_size_2 (push) Successful in 3m27s Details
Test / test_heal_csum_4k_dmj (push) Has been cancelled Details
Test / test_heal_csum_4k_dj (push) Has been cancelled Details
Test / test_heal_csum_4k (push) Has been cancelled Details
Test / test_osd_tags (push) Has been cancelled Details
Test / test_enospc (push) Has been cancelled Details
Test / test_enospc_xor (push) Has been cancelled Details
Test / test_enospc_imm (push) Has been cancelled Details
Test / test_enospc_imm_xor (push) Has been cancelled Details
Test / test_scrub (push) Has been cancelled Details
Test / test_scrub_zero_osd_2 (push) Has been cancelled Details
Test / test_scrub_xor (push) Has been cancelled Details
Test / test_scrub_pg_size_3 (push) Has been cancelled Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Has been cancelled Details
Test / test_scrub_ec (push) Has been cancelled Details
Test / test_nfs (push) Has been cancelled Details
Test / test_heal_csum_32k_dj (push) Has been cancelled Details
Test / test_heal_ec (push) Has been cancelled Details
Test / test_heal_csum_32k_dmj (push) Has been cancelled Details
Test / test_heal_csum_32k (push) Has been cancelled Details
2024-04-16 02:20:26 +03:00
Vitaliy Filippov 8d1067971b Fix pg_effsize (and thus "used space") calculation in monitor 2024-04-16 02:20:18 +03:00
Vitaliy Filippov ae5af04fde Add noout flag for OSDs 2024-04-16 02:19:55 +03:00
Vitaliy Filippov 266d038b11 Fix 32-bit build warnings and one error again :-)
Test / test_snapshot_chain_ec (push) Successful in 2m52s Details
Test / test_rebalance_verify_imm (push) Successful in 3m7s Details
Test / test_root_node (push) Successful in 8s Details
Test / test_rebalance_verify (push) Successful in 3m36s Details
Test / test_switch_primary (push) Successful in 40s Details
Test / test_write (push) Successful in 41s Details
Test / test_write_no_same (push) Successful in 18s Details
Test / test_write_xor (push) Successful in 1m6s Details
Test / test_rebalance_verify_ec (push) Successful in 4m25s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m52s Details
Test / test_heal_pg_size_2 (push) Successful in 3m21s Details
Test / test_heal_ec (push) Successful in 5m27s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m56s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m49s Details
Test / test_heal_csum_32k (push) Successful in 6m43s Details
Test / test_osd_tags (push) Successful in 21s Details
Test / test_enospc (push) Successful in 2m18s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m43s Details
Test / test_heal_csum_4k (push) Successful in 6m27s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m29s Details
Test / test_enospc_imm (push) Successful in 1m5s Details
Test / test_enospc_xor (push) Successful in 1m38s Details
Test / test_scrub (push) Successful in 37s Details
Test / test_scrub_zero_osd_2 (push) Successful in 32s Details
Test / test_enospc_imm_xor (push) Successful in 45s Details
Test / test_scrub_xor (push) Successful in 33s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 34s Details
Test / test_scrub_ec (push) Successful in 34s Details
Test / test_scrub_pg_size_3 (push) Successful in 43s Details
Test / test_nfs (push) Successful in 13s Details
2024-04-11 22:49:33 +03:00
Vitaliy Filippov ff4414d37e Release 1.6.0
Test / test_snapshot_chain_ec (push) Successful in 3m1s Details
Test / test_rebalance_verify_imm (push) Successful in 3m25s Details
Test / test_root_node (push) Successful in 8s Details
Test / test_rebalance_verify (push) Successful in 4m4s Details
Test / test_switch_primary (push) Successful in 34s Details
Test / test_write (push) Successful in 54s Details
Test / test_write_xor (push) Successful in 52s Details
Test / test_write_no_same (push) Successful in 13s Details
Test / test_rebalance_verify_ec (push) Successful in 4m13s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m9s Details
Test / test_heal_pg_size_2 (push) Successful in 4m30s Details
Test / test_heal_ec (push) Successful in 5m16s Details
Test / test_heal_csum_32k_dmj (push) Successful in 6m21s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m40s Details
Test / test_heal_csum_32k (push) Successful in 6m42s Details
Test / test_osd_tags (push) Successful in 39s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m35s Details
Test / test_enospc (push) Successful in 1m43s Details
Test / test_heal_csum_4k (push) Successful in 6m27s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m31s Details
Test / test_enospc_xor (push) Successful in 1m42s Details
Test / test_scrub_zero_osd_2 (push) Successful in 39s Details
Test / test_scrub (push) Successful in 41s Details
Test / test_enospc_imm (push) Successful in 46s Details
Test / test_enospc_imm_xor (push) Successful in 52s Details
Test / test_scrub_xor (push) Successful in 32s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 39s Details
Test / test_scrub_ec (push) Successful in 34s Details
Test / test_nfs (push) Successful in 16s Details
Test / test_scrub_pg_size_3 (push) Successful in 46s Details
New features:

- Implement "hierarchical failure domains" and other complex distribution rules, for example
  EC 4+2 over 3 DC, with 2 chunks per each DC ([documentation](docs/config/pool.en.md#level_placement))
- Make OSDs handle ENOSPC - now cluster stays online even if some OSDs fill up
  to 100 %, only writes requiring free space hang
- Implement Stage/Unstage & volume locking for CSI to prevent parallel mounting
  and/or modifications of the same volume
- Warn about full and almost full OSDs in vitastor-cli status
- Add an experimental NBD netlink map mode as an option ([documentation](docs/usage/nbd.en.md))
- Add --pg parameter to vitastor-cli describe, print objects with 0x in human-readable format too
- Add [administration docs](docs/usage/admin.en.md)

Bug fixes:

- Fix client operation retry timeout - previously the timeout wasn't applied and writes were
  retries almost instantly
- Fix monitors crashing on invalid pool configurations
- Fix journaling - make each journal write wait for all previous journal writes
- Fix monitor thinking that OSD weight is 0 after deleting /osd/config/ key online
- Fix a write stall caused by flusher possibly not trimming journal on rollback
- Set 32k csum_block_size for HDD by default in vitastor-disk
2024-04-09 16:57:59 +03:00
Vitaliy Filippov 0fa7ecc03f Add also a test for OSD tags 2024-04-09 16:57:59 +03:00
Vitaliy Filippov c29bfe12eb Oops - fix filter_by_root_node, add a test for it 2024-04-09 15:48:44 +03:00
Vitaliy Filippov 57bf84ddb2 Fix filtering in mon 2024-04-09 14:51:05 +03:00
Vitaliy Filippov dff4879c8c Check if NBD_ATTR_BACKEND_IDENTIFIER is defined 2024-04-09 13:16:58 +03:00
Vitaliy Filippov af9a853db6 Move NBD netlink map&unmap to separate commands, add "netlink-revive" command
Test / test_splitbrain (push) Successful in 21s Details
Test / test_snapshot_chain (push) Successful in 2m57s Details
Test / test_snapshot_chain_ec (push) Successful in 3m18s Details
Test / test_rebalance_verify_imm (push) Successful in 3m40s Details
Test / test_rebalance_verify (push) Successful in 4m19s Details
Test / test_switch_primary (push) Successful in 33s Details
Test / test_write (push) Successful in 53s Details
Test / test_write_xor (push) Successful in 58s Details
Test / test_write_no_same (push) Successful in 13s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m29s Details
Test / test_rebalance_verify_ec (push) Successful in 5m12s Details
Test / test_heal_pg_size_2 (push) Successful in 3m50s Details
Test / test_heal_ec (push) Successful in 3m46s Details
Test / test_heal_csum_32k_dmj (push) Successful in 6m12s Details
Test / test_heal_csum_32k_dj (push) Successful in 6m40s Details
Test / test_heal_csum_32k (push) Successful in 6m52s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m51s Details
Test / test_enospc (push) Successful in 1m42s Details
Test / test_enospc_xor (push) Successful in 2m23s Details
Test / test_enospc_imm (push) Successful in 1m42s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m12s Details
Test / test_heal_csum_4k (push) Successful in 5m40s Details
Test / test_enospc_imm_xor (push) Successful in 1m26s Details
Test / test_scrub_zero_osd_2 (push) Successful in 32s Details
Test / test_scrub (push) Successful in 35s Details
Test / test_scrub_xor (push) Successful in 27s Details
Test / test_nfs (push) Successful in 23s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 32s Details
Test / test_scrub_ec (push) Successful in 30s Details
Test / test_scrub_pg_size_3 (push) Successful in 43s Details
2024-04-08 16:34:41 +03:00
Vitaliy Filippov b7a3275af3 Make netlink optional 2024-04-08 01:51:28 +03:00
Vitaliy Filippov 64c5c4ca26 Fix code style 2024-04-08 01:35:03 +03:00
idelson 442a9d838d nbd-proxy: add configuration via netlink to support kinds of timeouts.
PR #58 - https://github.com/vitalif/vitastor/pull/58/commits

By MIND Software LLC

By submitting this pull request, I accept Vitastor CLA
2024-04-08 00:50:08 +03:00
Vitaliy Filippov 6366972fe8 Warn about full and almost full OSDs in status
Test / test_splitbrain (push) Successful in 18s Details
Test / test_snapshot_chain (push) Successful in 2m23s Details
Test / test_snapshot_chain_ec (push) Successful in 2m53s Details
Test / test_rebalance_verify_imm (push) Successful in 3m21s Details
Test / test_rebalance_verify (push) Successful in 3m46s Details
Test / test_switch_primary (push) Successful in 33s Details
Test / test_write (push) Successful in 54s Details
Test / test_write_xor (push) Successful in 48s Details
Test / test_write_no_same (push) Successful in 14s Details
Test / test_rebalance_verify_ec (push) Successful in 4m38s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m22s Details
Test / test_heal_pg_size_2 (push) Successful in 3m34s Details
Test / test_heal_ec (push) Successful in 3m38s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m44s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m51s Details
Test / test_heal_csum_32k (push) Successful in 6m45s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m34s Details
Test / test_enospc (push) Successful in 1m47s Details
Test / test_enospc_xor (push) Successful in 2m41s Details
Test / test_enospc_imm (push) Successful in 1m31s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m39s Details
Test / test_heal_csum_4k (push) Successful in 6m15s Details
Test / test_scrub_zero_osd_2 (push) Successful in 32s Details
Test / test_scrub (push) Successful in 35s Details
Test / test_scrub_xor (push) Successful in 26s Details
Test / test_enospc_imm_xor (push) Successful in 1m13s Details
Test / test_nfs (push) Successful in 24s Details
Test / test_scrub_ec (push) Successful in 33s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 34s Details
Test / test_scrub_pg_size_3 (push) Successful in 42s Details
2024-04-07 19:39:51 +03:00
Vitaliy Filippov 2b863fb715 Add ENOSPC handling tests 2024-04-07 19:39:33 +03:00
Vitaliy Filippov 3bf4dd5abd Fix client op retry timeout - do not retry immediately 2024-04-07 19:08:36 +03:00
Vitaliy Filippov 3b84dcaedd Handle ENOSPC during write - rollback partial EC writes, remember partial replica writes
Test / test_rm (push) Successful in 14s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m59s Details
Test / test_snapshot_down (push) Successful in 28s Details
Test / test_snapshot_down_ec (push) Successful in 30s Details
Test / test_splitbrain (push) Successful in 27s Details
Test / test_snapshot_chain (push) Successful in 2m41s Details
Test / test_snapshot_chain_ec (push) Successful in 3m12s Details
Test / test_rebalance_verify_imm (push) Successful in 3m33s Details
Test / test_rebalance_verify (push) Successful in 4m24s Details
Test / test_switch_primary (push) Successful in 34s Details
Test / test_write (push) Successful in 53s Details
Test / test_write_xor (push) Successful in 51s Details
Test / test_write_no_same (push) Successful in 11s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m11s Details
Test / test_rebalance_verify_ec (push) Successful in 6m3s Details
Test / test_heal_pg_size_2 (push) Successful in 4m57s Details
Test / test_heal_ec (push) Successful in 4m52s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m37s Details
Test / test_heal_csum_32k_dj (push) Successful in 6m55s Details
Test / test_heal_csum_32k (push) Successful in 6m42s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m41s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m45s Details
Test / test_scrub_zero_osd_2 (push) Successful in 44s Details
Test / test_scrub (push) Successful in 48s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m6s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m30s Details
Test / test_scrub_ec (push) Successful in 51s Details
Test / test_nfs (push) Successful in 39s Details
Test / test_heal_csum_4k (push) Successful in 5m22s Details
Test / test_scrub_xor (push) Successful in 18s Details
2024-04-07 18:02:05 +03:00
Vitaliy Filippov 20fbc4a745 Add --pg parameter to vitastor-cli describe, print objects with 0x in human-readable format too
Test / test_rm (push) Successful in 16s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 2m4s Details
Test / test_snapshot_down (push) Successful in 26s Details
Test / test_snapshot_down_ec (push) Successful in 30s Details
Test / test_splitbrain (push) Successful in 18s Details
Test / test_snapshot_chain (push) Successful in 2m32s Details
Test / test_snapshot_chain_ec (push) Successful in 3m11s Details
Test / test_rebalance_verify_imm (push) Successful in 3m34s Details
Test / test_rebalance_verify (push) Successful in 4m12s Details
Test / test_switch_primary (push) Successful in 33s Details
Test / test_write (push) Successful in 52s Details
Test / test_write_xor (push) Successful in 54s Details
Test / test_write_no_same (push) Successful in 13s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m6s Details
Test / test_rebalance_verify_ec (push) Successful in 5m1s Details
Test / test_heal_pg_size_2 (push) Successful in 4m15s Details
Test / test_heal_ec (push) Successful in 4m11s Details
Test / test_heal_csum_32k_dmj (push) Successful in 6m4s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m59s Details
Test / test_heal_csum_32k (push) Successful in 6m42s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m40s Details
Test / test_scrub_zero_osd_2 (push) Successful in 55s Details
Test / test_scrub (push) Successful in 58s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m22s Details
Test / test_heal_csum_4k (push) Successful in 6m24s Details
Test / test_scrub_pg_size_3 (push) Successful in 2m11s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 25s Details
Test / test_scrub_ec (push) Successful in 23s Details
Test / test_nfs (push) Successful in 13s Details
Test / test_scrub_xor (push) Successful in 18s Details
2024-04-07 12:39:46 +03:00
Vitaliy Filippov 02993ee1dd Implement Stage/Unstage & volume locking for CSI to prevent parallel modifications of the same volume 2024-04-07 11:48:19 +03:00
Vitaliy Filippov 3629dbc54d Plug the new PG combinator into monitor
Test / test_move_reappear (push) Successful in 22s Details
Test / test_snapshot_down (push) Successful in 25s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 2m46s Details
Test / test_snapshot_down_ec (push) Successful in 24s Details
Test / test_splitbrain (push) Successful in 17s Details
Test / test_snapshot_chain (push) Successful in 2m36s Details
Test / test_snapshot_chain_ec (push) Successful in 3m1s Details
Test / test_rebalance_verify_imm (push) Successful in 3m17s Details
Test / test_rebalance_verify (push) Successful in 3m50s Details
Test / test_switch_primary (push) Successful in 33s Details
Test / test_write (push) Successful in 50s Details
Test / test_write_xor (push) Successful in 56s Details
Test / test_write_no_same (push) Successful in 14s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m29s Details
Test / test_rebalance_verify_ec (push) Successful in 5m23s Details
Test / test_heal_pg_size_2 (push) Successful in 4m23s Details
Test / test_heal_ec (push) Successful in 4m57s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m21s Details
Test / test_heal_csum_32k_dj (push) Successful in 6m33s Details
Test / test_heal_csum_32k (push) Successful in 6m55s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m54s Details
Test / test_scrub (push) Successful in 1m32s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m12s Details
Test / test_heal_csum_4k_dj (push) Successful in 7m12s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m1s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m41s Details
Test / test_heal_csum_4k (push) Successful in 6m22s Details
Test / test_scrub_ec (push) Successful in 44s Details
Test / test_nfs (push) Successful in 16s Details
Test / test_scrub_xor (push) Successful in 18s Details
2024-04-07 02:44:17 +03:00
Vitaliy Filippov 29284bef40 Implement new DSL/rule-based PG generation algorithm 2024-04-07 00:36:20 +03:00
Vitaliy Filippov 6a924d6066 Extract PG combinator into a separate module 2024-04-07 00:36:20 +03:00
Vitaliy Filippov 9fe779a691 Do not die on invalid pool configurations
Test / test_rm (push) Successful in 16s Details
Test / test_move_reappear (push) Successful in 24s Details
Test / test_snapshot_down (push) Successful in 26s Details
Test / test_snapshot_down_ec (push) Successful in 31s Details
Test / test_splitbrain (push) Successful in 17s Details
Test / test_snapshot_chain (push) Successful in 2m34s Details
Test / test_snapshot_chain_ec (push) Successful in 3m12s Details
Test / test_rebalance_verify_imm (push) Successful in 2m59s Details
Test / test_rebalance_verify (push) Successful in 3m27s Details
Test / test_switch_primary (push) Successful in 34s Details
Test / test_write (push) Successful in 55s Details
Test / test_write_xor (push) Successful in 54s Details
Test / test_write_no_same (push) Successful in 15s Details
Test / test_rebalance_verify_ec (push) Successful in 4m37s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m8s Details
Test / test_heal_pg_size_2 (push) Successful in 3m48s Details
Test / test_heal_ec (push) Successful in 3m47s Details
Test / test_heal_csum_32k_dmj (push) Successful in 6m8s Details
Test / test_heal_csum_32k_dj (push) Successful in 6m18s Details
Test / test_heal_csum_32k (push) Successful in 7m9s Details
Test / test_heal_csum_4k_dmj (push) Successful in 7m7s Details
Test / test_scrub (push) Successful in 1m9s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m8s Details
Test / test_scrub_xor (push) Successful in 1m7s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m20s Details
Test / test_heal_csum_4k (push) Successful in 5m58s Details
Test / test_scrub_pg_size_3 (push) Successful in 2m9s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m4s Details
Test / test_nfs (push) Successful in 15s Details
Test / test_scrub_ec (push) Successful in 21s Details
2024-04-07 00:36:20 +03:00
Vitaliy Filippov 31c2751b9b Move NBD/VDUSE map/unmap functions to a separate file 2024-04-07 00:36:09 +03:00
Vitaliy Filippov c5195666cd Fix journal sequencing: make each journal write wait for all previous journal writes
Test / test_snapshot_ec (push) Successful in 31s Details
Test / test_rm (push) Successful in 15s Details
Test / test_snapshot_down (push) Successful in 30s Details
Test / test_snapshot_down_ec (push) Successful in 32s Details
Test / test_splitbrain (push) Successful in 25s Details
Test / test_snapshot_chain (push) Successful in 2m11s Details
Test / test_snapshot_chain_ec (push) Successful in 3m1s Details
Test / test_rebalance_verify_imm (push) Successful in 2m35s Details
Test / test_rebalance_verify (push) Successful in 3m10s Details
Test / test_switch_primary (push) Successful in 39s Details
Test / test_write (push) Successful in 43s Details
Test / test_write_no_same (push) Successful in 18s Details
Test / test_write_xor (push) Successful in 1m3s Details
Test / test_rebalance_verify_ec (push) Successful in 4m38s Details
Test / test_heal_pg_size_2 (push) Successful in 3m22s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 5m11s Details
Test / test_heal_ec (push) Successful in 4m23s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m55s Details
Test / test_heal_csum_32k_dj (push) Successful in 6m31s Details
Test / test_heal_csum_32k (push) Successful in 6m29s Details
Test / test_heal_csum_4k_dmj (push) Successful in 7m18s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m0s Details
Test / test_scrub (push) Failing after 3m19s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m39s Details
Test / test_scrub_xor (push) Successful in 58s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m13s Details
Test / test_scrub_ec (push) Successful in 50s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m51s Details
Test / test_heal_csum_4k (push) Successful in 5m13s Details
Test / test_nfs (push) Successful in 23s Details
2024-04-06 23:53:12 +03:00
Vitaliy Filippov f36d7eb76c Fix monitor thinking that OSD weight is 0 after deleting /osd/config/ key
Test / test_rm (push) Successful in 16s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m30s Details
Test / test_snapshot_down (push) Successful in 31s Details
Test / test_snapshot_down_ec (push) Successful in 32s Details
Test / test_splitbrain (push) Successful in 23s Details
Test / test_snapshot_chain (push) Successful in 2m13s Details
Test / test_snapshot_chain_ec (push) Successful in 3m1s Details
Test / test_rebalance_verify_imm (push) Successful in 3m22s Details
Test / test_rebalance_verify (push) Successful in 4m5s Details
Test / test_switch_primary (push) Successful in 34s Details
Test / test_write (push) Successful in 52s Details
Test / test_write_xor (push) Successful in 52s Details
Test / test_write_no_same (push) Successful in 14s Details
Test / test_rebalance_verify_ec (push) Successful in 4m41s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m51s Details
Test / test_heal_pg_size_2 (push) Successful in 3m55s Details
Test / test_heal_ec (push) Successful in 4m35s Details
Test / test_heal_csum_32k_dmj (push) Successful in 6m0s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m51s Details
Test / test_heal_csum_32k (push) Successful in 6m48s Details
Test / test_heal_csum_4k_dmj (push) Successful in 7m7s Details
Test / test_scrub (push) Successful in 1m36s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m20s Details
Test / test_scrub_xor (push) Successful in 56s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m39s Details
Test / test_heal_csum_4k (push) Successful in 6m37s Details
Test / test_nfs (push) Successful in 18s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 47s Details
Test / test_scrub_ec (push) Successful in 27s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m3s Details
2024-04-05 23:14:46 +03:00
Vitaliy Filippov dd7f651de1 Add --max-request-bytes=104857600 to etcd params in tests 2024-04-05 23:14:46 +03:00
Vitaliy Filippov a2994ecd0d Fix flusher possibly not trimming journal on rollback 2024-04-05 23:14:39 +03:00
Vitaliy Filippov 5d3aaf016b Add administration docs 2024-03-31 01:54:52 +03:00
Vitaliy Filippov 0b097ca3f2 Set 32k csum_block_size for HDD by default 2024-03-30 16:16:49 +03:00
Vitaliy Filippov 989675a780 s/etcd_ws_keepalive_timeout/etcd_ws_keepalive_interval/ in docs 2024-03-26 01:56:08 +03:00
Vitaliy Filippov f8c403ec9e Add newer benchmark results 2024-03-23 18:28:48 +03:00
Vitaliy Filippov bfbb85e653 Replace -Oanything with -O3, not just -O/-O1/-O2
Test / test_move_reappear (push) Successful in 22s Details
Test / test_rm (push) Successful in 13s Details
Test / test_snapshot_down (push) Successful in 31s Details
Test / test_snapshot_down_ec (push) Successful in 32s Details
Test / test_splitbrain (push) Successful in 21s Details
Test / test_snapshot_chain (push) Successful in 2m11s Details
Test / test_snapshot_chain_ec (push) Successful in 3m14s Details
Test / test_rebalance_verify_imm (push) Successful in 3m29s Details
Test / test_rebalance_verify (push) Successful in 4m4s Details
Test / test_switch_primary (push) Successful in 40s Details
Test / test_write (push) Successful in 42s Details
Test / test_write_no_same (push) Successful in 17s Details
Test / test_write_xor (push) Successful in 1m3s Details
Test / test_rebalance_verify_ec (push) Successful in 4m36s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m43s Details
Test / test_heal_pg_size_2 (push) Successful in 3m36s Details
Test / test_heal_ec (push) Successful in 6m7s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m39s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m33s Details
Test / test_heal_csum_32k (push) Successful in 6m38s Details
Test / test_scrub (push) Successful in 1m52s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m32s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m30s Details
Test / test_heal_csum_4k (push) Successful in 6m25s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m32s Details
Test / test_scrub_xor (push) Successful in 31s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 35s Details
Test / test_scrub_pg_size_3 (push) Successful in 43s Details
Test / test_nfs (push) Successful in 13s Details
Test / test_scrub_ec (push) Successful in 19s Details
2024-03-18 02:03:44 +03:00
Vitaliy Filippov 9ad6822353 Release 1.5.0
Test / test_rm (push) Successful in 14s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m36s Details
Test / test_snapshot_down (push) Successful in 31s Details
Test / test_snapshot_down_ec (push) Successful in 30s Details
Test / test_splitbrain (push) Successful in 24s Details
Test / test_snapshot_chain (push) Successful in 2m20s Details
Test / test_snapshot_chain_ec (push) Successful in 3m5s Details
Test / test_rebalance_verify_imm (push) Successful in 5m11s Details
Test / test_rebalance_verify (push) Successful in 5m55s Details
Test / test_switch_primary (push) Successful in 33s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m26s Details
Test / test_write (push) Successful in 54s Details
Test / test_write_xor (push) Successful in 57s Details
Test / test_write_no_same (push) Successful in 19s Details
Test / test_rebalance_verify_ec (push) Successful in 7m21s Details
Test / test_heal_pg_size_2 (push) Successful in 4m36s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m33s Details
Test / test_heal_ec (push) Successful in 6m15s Details
Test / test_heal_csum_32k_dj (push) Successful in 6m31s Details
Test / test_heal_csum_32k (push) Successful in 6m29s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m15s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m16s Details
Test / test_scrub (push) Successful in 1m18s Details
Test / test_scrub_xor (push) Successful in 1m13s Details
Test / test_heal_csum_4k_dj (push) Successful in 7m10s Details
Test / test_scrub_ec (push) Successful in 56s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 59s Details
Test / test_heal_csum_4k (push) Successful in 6m2s Details
Test / test_scrub_pg_size_3 (push) Successful in 2m11s Details
Test / test_nfs (push) Successful in 11s Details
After half a year of hard work, VitastorFS is finally here ! :-)

New features:
- VitastorFS, a full-featured clustered (read-write-many) file system.
  Documentation: [VitastorFS](docs/usage/nfs.en.md)
- Embedded key-value database implementation based on Parallel Optimistic B-Tree
  algorithm and used for the metadata of VitastorFS
- Pool management commands in vitastor-cli (create-pool, list-pools, rm-pool, modify-pool).
  Thanks MIND Software (https://mindsw.io) for their contribution!
  [Documentation](docs/usage/cli.en.md#create-pool)

Bug fixes:
- Fix a very rare "infinite loop" in the client library
- Fix a rare OSD hang on during start when zeroing out bad metadata entries left from the previous run
2024-03-16 15:35:10 +03:00
Vitaliy Filippov 2043b4e374 Fix build errors for gcc 8 2024-03-16 15:35:10 +03:00
Vitaliy Filippov de840e6fe3 Reduce kv-cli loadjson load parallelism to 16 2024-03-16 15:35:10 +03:00
Vitaliy Filippov b5e04bf809 Fix build warning 2024-03-16 15:35:10 +03:00
Vitaliy Filippov 8807a1623b Fix markdown tables 2024-03-16 15:35:10 +03:00
Vitaliy Filippov f12855c31b Add vitastor-kv to packages 2024-03-16 15:35:10 +03:00
Vitaliy Filippov e75dcc9a71 Add documentation for VitastorFS 2024-03-16 15:16:43 +03:00
Vitaliy Filippov 88516ab4bd Remove extra log 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 6221126b4f Allow to print simple-offsets just given the device size 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 6783d4a13c Implement fool protection for FS pools 2024-03-16 13:24:36 +03:00
Vitaliy Filippov dcbe1afac3 Store pool ID in inode metadata 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 0bde28c24a Make nfs_do_rmw a library function 2024-03-16 13:24:36 +03:00
Vitaliy Filippov bb8ca6184e Support setattr guard 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 87310ef7bb Support ctime 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 4f4b2dab80 Log NFS liveness checks 2024-03-16 13:24:36 +03:00
Vitaliy Filippov f70da82317 Add loadjson command to vitastor-kv 2024-03-16 13:24:36 +03:00
Vitaliy Filippov e42148f347 Allow to specify KV commands on command line 2024-03-16 13:24:36 +03:00
Vitaliy Filippov c289584469 Add JSON dump format 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 018e89f867 Erase verf key left from creation from ientries on every modification 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 603dc68f11 Implement async mtime change 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 7b12342933 Allow to specify additional NFS mount options 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 44bf0f16ee Fix malloc/free in nfs_kv_read/write 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 8840c84572 Fix "bad key in etcd" in mon for FS pools 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 5b747c12ec Check if already mounted before mounting 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 05f5f46162 Fix zero used space, update mtime when moving/changing inode 2024-03-16 13:24:36 +03:00
Vitaliy Filippov b5604191c8 Ignore ECANCELED in nfs-proxy (happens in io_uring on fork) 2024-03-16 13:24:36 +03:00
Vitaliy Filippov e871de27de Support unaligned shared_offsets, align shared file data instead of header 2024-03-16 13:24:36 +03:00
Vitaliy Filippov f600ce98e2 Implement auto-unmount local NFS server mode for vitastor-nfs 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 57605a5c13 Return error on failed shrink 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 29bd4561bb Implement rename over an existing file/directory 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 7142460ec8 Support --logfile in nfs-proxy 2024-03-16 13:24:36 +03:00
Vitaliy Filippov d03f19ebe5 Fix shared file overlap, add FIXMEs 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 88f9d18be3 Create inode, then direntry, not direntry, then inode; retry ID collisions 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 6213fbd8c6 Fix NFS shared/aligned write FIXMEs 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 3aee37eadd Allow to disable per-inode stats for VitastorFS pools 2024-03-16 13:24:36 +03:00
Vitaliy Filippov ecfc753e93 Add basic NFS tests, fix bugs 2024-03-16 13:24:36 +03:00
Vitaliy Filippov a574f9ad71 Return block NFS implementation back as an option too 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 7c235c9103 Move KV FS header into a separate file 2024-03-16 13:24:36 +03:00
Vitaliy Filippov e5bb986164 Implement packing small files into shared inodes 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 181795d748 Split new NFS proxy implementation into multiple files 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 8cdc38805b WIP VitastorFS with metadata storage in VitastorKV 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 0cd455d17f First just recheck version without actually re-reading block in vitastor-kv 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 32ba653ba6 Fix vitastor-kv hang on reopen & unfinished closed listing 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 231d4b15fc Add loadable dump format to vitastor-kv (dump) 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 9dc4d5fd7b Fix freeing r/w buffers on errors in kv_db 2024-03-16 13:24:36 +03:00
Vitaliy Filippov e58538fa47 Fix eviction when random_pos selects the end 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 11ac9e7024 Implement min/max list_count to make listings during performance test reasonable 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 511bc3df1c Fix and improve parallel allocation
- Do not try to allocate more DB blocks in an inode block until it's "confirmed" and "locked" by the first write
- Do not recheck for new zero DB blocks on first write into an inode block - a CAS failure means someone else is already writing into it
- Throw new allocation blocks away regardless of whether the known_version is 0 on a CAS failure
2024-03-16 13:24:36 +03:00
Vitaliy Filippov a64f0d1f73 Implement key_prefix for K/V stress test 2024-03-16 13:24:36 +03:00
Vitaliy Filippov ec5f7c6b87 More fixes
- do not overwrite a block with older version if known version is newer
  (read may start before update and end after update)
- invalidated block versions can't be remembered and trusted
- right boundary for split blocks is right_half when diving down, not key_lt
- restart update also when block is "invalidated", not just on version mismatch
- copy callback in listings to avoid closure destruction bugs too
2024-03-16 13:24:36 +03:00
Vitaliy Filippov 3ebed9a749 Add logging and one more assert 2024-03-16 13:24:36 +03:00
Vitaliy Filippov eab67a6e8f Make get_block() wait for updating when unrelated block is found along the path 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 20993d9b7a Fix a race condition where changed blocks were parsed over existing cached blocks and getting a mix of data 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 5cf9b343c0 Simplify code by removing an unneeded "optimisation" 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 79ae0aadcd Add kv_log_level, print warnings on level 1, trace ops on level 10 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 605afc3583 Fix duplicate keys in listings on parallel updates -- do not rewind key "iterator position" 2024-03-16 13:24:36 +03:00
Vitaliy Filippov c0681d8242 Implement key suffix to avoid collisions of multiple test workers 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 763e77b4f4 Do not complain on empty first block 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 19426aa4c5 Add JSON output for stress-tester 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 08f586bcec Print total stats 2024-03-16 13:24:36 +03:00
Vitaliy Filippov f1cd87473a Do not send more than op_count operations (fix segfault on finish) 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 1bd8d2da56 Add some more resiliency to serialize() 2024-03-16 13:24:36 +03:00
Vitaliy Filippov a7396d2baf Invalidate blocks being updated too 2024-03-16 13:24:36 +03:00
Vitaliy Filippov e98a38810d Change new block allocation method: make each writer choose multiple empty PG blocks and place blocks in them 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 28c4324c36 Remove blocks from cache on unsuccessful updates 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 31ec3fa8f5 Allow to track multiple updates per block (it should never happen though) 2024-03-16 13:24:36 +03:00
Vitaliy Filippov e4fa26f60a Do not call stop_updating after failed write_new_block and after clear_block (both delete the item) 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 59ae27f9e5 Track versions of parent blocks and recheck if changed during update 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 2c6a301d9b Fix resume_split condition (key_lt can also be "") 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 01558349f8 Experiment: transform offsets for better sharding 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 36f4717d0d More post-stress-test fixes
- Prevent _split types of new blocks
- Stop updating new blocks only after the whole update, otherwise pointers
  may become invalid
- Use recheck_none for updates initially
- Use UINT64_MAX as initial block version when postponing ops, otherwise the
  check fails when the block is initially empty. This for example leads to
  writing both leaf items & block pointers (which is incorrect) into the root
  block when starting stress-test with --parallelism 32
- Fix -EINTR comparison
2024-03-16 13:24:36 +03:00
Vitaliy Filippov babaf2a0ce Print operation statistics 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 5773f1a375 K/V fixes after stress-test :-)
- track block versions correctly - per inode block (128kb) instead of tree block (4kb)
- prevent multiple parallel CAS writes of the same inode block
- add logging for EILSEQ which means invalid data in the tree
- fix get_block updated flag which was true for blocks already in cache and was leading to infinite loops on "unrelated block" errors
- apply changes to blocks in cache only after successful writes (using "virtual changes")
- do not replace cached block with an older version from disk
- recheck "unrelated blocks" (read/update collisions) until data stops changing
- track tree path correctly - do not treat split block as parent of its right half
- correctly move blocks when finding new empty place on disk
- restart updates from the beginning when one of blocks is changed by a parallel update
- fix delete using SET opcode and setting key to the empty value instead
- prevent changing the same key more than 1 time in parallel
- fix listing verification
- resume continue_updates in update_find (required because it uses continue_update itself)
- add allow_old_cached parameter to get()
2024-03-16 13:24:36 +03:00
Vitaliy Filippov 57222a9f79 Implement K/V DB stress tester 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 61ef000c6e Evict blocks based on memory limit & block usage 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 7d5e1cc393 Track blocks per level 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 5e7f27a02d Track block level 2024-03-16 13:24:36 +03:00
Vitaliy Filippov fd1d8a8520 Experimental B-Tree Vitastor embedded K/V database implementation! 2024-03-16 13:24:36 +03:00
Vitaliy Filippov c364e14c40 Stop then retry, not retry then stop 2024-03-16 13:24:36 +03:00
Vitaliy Filippov 3ebbfa0428 Fix another rare OSD hang on zeroing out entries on start 2024-03-16 13:24:36 +03:00
Vitaliy Filippov aa79d1db1c Fix incorrect "changing scheme" message in modify-pool
Test / test_rm (push) Successful in 14s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m32s Details
Test / test_move_reappear (push) Successful in 20s Details
Test / test_snapshot_down (push) Successful in 29s Details
Test / test_snapshot_down_ec (push) Successful in 29s Details
Test / test_splitbrain (push) Successful in 28s Details
Test / test_snapshot_chain (push) Successful in 2m5s Details
Test / test_snapshot_chain_ec (push) Successful in 3m3s Details
Test / test_rebalance_verify_imm (push) Successful in 4m0s Details
Test / test_rebalance_verify (push) Successful in 4m40s Details
Test / test_switch_primary (push) Successful in 38s Details
Test / test_write (push) Successful in 41s Details
Test / test_write_no_same (push) Successful in 17s Details
Test / test_write_xor (push) Successful in 1m2s Details
Test / test_rebalance_verify_ec (push) Successful in 5m34s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 5m34s Details
Test / test_heal_pg_size_2 (push) Successful in 3m22s Details
Test / test_heal_ec (push) Successful in 4m58s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m37s Details
Test / test_heal_csum_32k_dj (push) Successful in 6m21s Details
Test / test_heal_csum_32k (push) Successful in 7m1s Details
Test / test_scrub (push) Successful in 1m37s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m59s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m26s Details
Test / test_scrub_xor (push) Successful in 1m3s Details
Test / test_heal_csum_4k_dj (push) Successful in 7m20s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m7s Details
Test / test_scrub_ec (push) Successful in 36s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m37s Details
Test / test_heal_csum_4k (push) Successful in 6m23s Details
2024-03-06 00:41:35 +03:00
Vitaliy Filippov a1fecb7eff Move callback away when calling it in cluster_client 2024-03-06 00:41:35 +03:00
Vitaliy Filippov ff74b19423 Fix rare OSD hang on zeroing out bad entries on start 2024-03-06 00:41:35 +03:00
Vitaliy Filippov 4cf6dceed7 Merge branch 'rel-1.4'
Test / test_minsize_1 (push) Has been cancelled Details
Test / test_move_reappear (push) Has been cancelled Details
Test / test_rm (push) Has been cancelled Details
Test / test_snapshot_chain (push) Has been cancelled Details
Test / test_snapshot_chain_ec (push) Has been cancelled Details
Test / test_snapshot_down (push) Has been cancelled Details
Test / test_snapshot_down_ec (push) Has been cancelled Details
Test / test_splitbrain (push) Has been cancelled Details
Test / test_rebalance_verify (push) Has been cancelled Details
Test / test_rebalance_verify_imm (push) Has been cancelled Details
Test / test_rebalance_verify_ec (push) Has been cancelled Details
Test / test_rebalance_verify_ec_imm (push) Has been cancelled Details
Test / test_switch_primary (push) Has been cancelled Details
Test / test_write (push) Has been cancelled Details
Test / test_write_xor (push) Has been cancelled Details
Test / test_write_no_same (push) Has been cancelled Details
Test / test_heal_pg_size_2 (push) Has been cancelled Details
Test / test_heal_ec (push) Has been cancelled Details
Test / test_heal_csum_32k_dmj (push) Has been cancelled Details
Test / test_heal_csum_32k_dj (push) Has been cancelled Details
Test / test_heal_csum_32k (push) Has been cancelled Details
Test / test_heal_csum_4k_dmj (push) Has been cancelled Details
Test / test_heal_csum_4k_dj (push) Has been cancelled Details
Test / test_heal_csum_4k (push) Has been cancelled Details
Test / test_scrub (push) Has been cancelled Details
Test / test_scrub_zero_osd_2 (push) Has been cancelled Details
Test / test_scrub_xor (push) Has been cancelled Details
Test / test_scrub_pg_size_3 (push) Has been cancelled Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Has been cancelled Details
Test / test_scrub_ec (push) Has been cancelled Details
2024-02-29 09:59:01 +03:00
Vitaliy Filippov 38b8963330 Release 1.4.8
Test / test_rm (push) Successful in 19s Details
Test / test_move_reappear (push) Successful in 26s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m40s Details
Test / test_snapshot_down (push) Successful in 31s Details
Test / test_snapshot_down_ec (push) Successful in 34s Details
Test / test_splitbrain (push) Successful in 27s Details
Test / test_snapshot_chain (push) Successful in 2m18s Details
Test / test_snapshot_chain_ec (push) Successful in 2m59s Details
Test / test_rebalance_verify_imm (push) Successful in 5m32s Details
Test / test_rebalance_verify (push) Successful in 6m11s Details
Test / test_switch_primary (push) Successful in 41s Details
Test / test_write (push) Successful in 45s Details
Test / test_write_no_same (push) Successful in 23s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 5m2s Details
Test / test_write_xor (push) Successful in 55s Details
Test / test_rebalance_verify_ec (push) Successful in 6m22s Details
Test / test_heal_pg_size_2 (push) Successful in 5m41s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m59s Details
Test / test_heal_csum_32k_dj (push) Successful in 7m19s Details
Test / test_heal_csum_32k (push) Successful in 7m17s Details
Test / test_heal_csum_4k_dmj (push) Successful in 7m14s Details
Test / test_scrub (push) Successful in 1m12s Details
Test / test_heal_ec (push) Successful in 9m2s Details
Test / test_scrub_xor (push) Successful in 56s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m8s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 2m1s Details
Test / test_heal_csum_4k_dj (push) Successful in 4m45s Details
Test / test_scrub_pg_size_3 (push) Successful in 2m31s Details
Test / test_heal_csum_4k (push) Successful in 4m54s Details
Test / test_scrub_ec (push) Successful in 46s Details
- Do not use \r if output is not a terminal (should fix unexpected job output in proxmox)
- Fix rm/rm-data error return code, add --down-ok option to bypass the error
- Add EIO retry timeout and allow to disable these retries, rename up_wait_retry_interval to client_retry_interval
- Add ubuntu jammy build
- Wait for blockstore initialisation before starting OSD (prevent timeouts when init takes time)
- Fix a rare use-after-free in automatic sync after delete in blockstore
2024-02-29 09:58:34 +03:00
Vitaliy Filippov 77167e2920 Do not use \r if output is not a terminal 2024-02-29 00:21:17 +03:00
Vitaliy Filippov 5af23672d0 Fix rm/rm-data error return code, add --down-ok option to bypass the error 2024-02-29 00:20:10 +03:00
Vitaliy Filippov 6bf1f539a6 Add EIO retry timeout and allow to disable these retries, rename up_wait_retry_interval to client_retry_interval 2024-02-28 13:10:02 +03:00
Vitaliy Filippov 4eab26f968 Add documentation and a very basic test for pool management commands
Test / test_snapshot_ec (push) Successful in 31s Details
Test / test_rm (push) Successful in 17s Details
Test / test_move_reappear (push) Successful in 24s Details
Test / test_snapshot_down (push) Successful in 27s Details
Test / test_snapshot_down_ec (push) Successful in 33s Details
Test / test_splitbrain (push) Successful in 20s Details
Test / test_snapshot_chain (push) Successful in 2m15s Details
Test / test_snapshot_chain_ec (push) Successful in 2m58s Details
Test / test_rebalance_verify_imm (push) Successful in 5m3s Details
Test / test_rebalance_verify (push) Successful in 5m36s Details
Test / test_switch_primary (push) Successful in 37s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m3s Details
Test / test_write_no_same (push) Successful in 21s Details
Test / test_write (push) Successful in 58s Details
Test / test_write_xor (push) Successful in 1m31s Details
Test / test_rebalance_verify_ec (push) Successful in 6m20s Details
Test / test_heal_pg_size_2 (push) Successful in 4m7s Details
Test / test_heal_ec (push) Successful in 4m33s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m53s Details
Test / test_heal_csum_32k_dj (push) Successful in 6m17s Details
Test / test_heal_csum_32k (push) Successful in 7m23s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m56s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m26s Details
Test / test_scrub (push) Successful in 1m29s Details
Test / test_heal_csum_4k_dj (push) Successful in 7m1s Details
Test / test_scrub_xor (push) Successful in 1m1s Details
Test / test_heal_csum_4k (push) Successful in 6m34s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 32s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m19s Details
Test / test_scrub_ec (push) Successful in 24s Details
2024-02-28 13:08:04 +03:00
Vitaliy Filippov 86243b7101 Rework & fix pool-create / pool-modify / pool-ls 2024-02-28 13:08:04 +03:00
idelson dc92851322 vitastor-cli: add commands to control pools: pool-create, pool-ls, pool-modify, pool-rm
PR #59 - https://github.com/vitalif/vitastor/pull/58/commits

By MIND Software LLC

By submitting this pull request, I accept Vitastor CLA
2024-02-28 13:08:04 +03:00
Zibort Cloud 02d1f16bbd Add ubuntu jammy build
PR #62 #62

I accept Vitastor CLA agreement: https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/CLA-en.md
2024-02-28 11:43:54 +03:00
Vitaliy Filippov fc413038d1 Wait for blockstore initialisation before starting OSD
Test / test_cas (push) Has been cancelled Details
Test / test_change_pg_count (push) Has been cancelled Details
Test / test_change_pg_count_ec (push) Has been cancelled Details
Test / test_change_pg_size (push) Has been cancelled Details
Test / test_create_nomaxid (push) Has been cancelled Details
Test / test_etcd_fail (push) Has been cancelled Details
Test / test_interrupted_rebalance (push) Has been cancelled Details
Test / test_interrupted_rebalance_imm (push) Has been cancelled Details
Test / test_interrupted_rebalance_ec (push) Has been cancelled Details
Test / test_interrupted_rebalance_ec_imm (push) Has been cancelled Details
Test / test_failure_domain (push) Has been cancelled Details
Test / test_snapshot (push) Has been cancelled Details
Test / test_snapshot_ec (push) Has been cancelled Details
Test / test_minsize_1 (push) Has been cancelled Details
Test / test_move_reappear (push) Has been cancelled Details
Test / test_rm (push) Has been cancelled Details
Test / test_snapshot_chain (push) Has been cancelled Details
Test / test_snapshot_chain_ec (push) Has been cancelled Details
Test / test_snapshot_down (push) Has been cancelled Details
Test / test_snapshot_down_ec (push) Has been cancelled Details
Test / test_splitbrain (push) Has been cancelled Details
Test / test_rebalance_verify (push) Has been cancelled Details
Test / test_rebalance_verify_imm (push) Has been cancelled Details
Test / test_rebalance_verify_ec (push) Has been cancelled Details
Test / test_rebalance_verify_ec_imm (push) Has been cancelled Details
Test / test_switch_primary (push) Has been cancelled Details
Test / test_write (push) Has been cancelled Details
Test / test_write_xor (push) Has been cancelled Details
Test / test_write_no_same (push) Has been cancelled Details
Test / test_heal_pg_size_2 (push) Has been cancelled Details
2024-02-27 02:20:04 +03:00
Vitaliy Filippov 1bc0b5aab3 Fix a rare use-after-free in automatic sync after delete in blockstore
Test / test_interrupted_rebalance_ec (push) Successful in 2m49s Details
Test / test_rm (push) Successful in 14s Details
Test / test_move_reappear (push) Successful in 21s Details
Test / test_snapshot_down (push) Successful in 31s Details
Test / test_snapshot_down_ec (push) Successful in 30s Details
Test / test_splitbrain (push) Successful in 23s Details
Test / test_snapshot_chain (push) Successful in 2m29s Details
Test / test_snapshot_chain_ec (push) Successful in 2m48s Details
Test / test_rebalance_verify_imm (push) Successful in 4m9s Details
Test / test_rebalance_verify (push) Successful in 4m42s Details
Test / test_switch_primary (push) Successful in 41s Details
Test / test_write (push) Successful in 43s Details
Test / test_write_no_same (push) Successful in 21s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m37s Details
Test / test_write_xor (push) Successful in 1m11s Details
Test / test_rebalance_verify_ec (push) Successful in 7m14s Details
Test / test_heal_pg_size_2 (push) Successful in 4m3s Details
Test / test_heal_ec (push) Successful in 4m18s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m5s Details
Test / test_heal_csum_32k_dj (push) Successful in 6m52s Details
Test / test_heal_csum_32k (push) Successful in 6m23s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m23s Details
Test / test_scrub (push) Successful in 1m30s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m18s Details
Test / test_heal_csum_4k_dj (push) Successful in 7m9s Details
Test / test_scrub_xor (push) Successful in 57s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m5s Details
Test / test_scrub_ec (push) Successful in 1m6s Details
Test / test_scrub_pg_size_3 (push) Successful in 2m3s Details
Test / test_heal_csum_4k (push) Successful in 4m54s Details
ASan report: [0] READ of size 16 at operator() /root/vitastor/src/blockstore_write.cpp:100
...[5] blockstore_impl_t::ack_sync(blockstore_op_t*) /root/vitastor/src/blockstore_sync.cpp:232
2024-02-24 00:06:36 +03:00
Vitaliy Filippov 5e934264cf Release 1.4.7
- Fix another old "BUG: Attempt to overwrite used offset" in a very simple
  case: bs=4k rw=write iodepth=16 from OSD start; add this case to tests
- Fix a rare crash with "unexpected state during flush: 0x51" possible with
  EC since 1.4.2 during rebalance and OSD outages
- Fix a rare write stall with EC & immediate_commit=none caused by sync
  operations reserving unneeded space in the journal
- Fix 32-bit build warnings, most in printf/scanf format strings
2024-02-22 12:45:52 +03:00
Vitaliy Filippov f20564b44b Fix 32-bit build warnings (99.9% in printf) 2024-02-22 12:22:16 +03:00
Vitaliy Filippov b3c15db331 32M journal by default in simple-offsets
Test / test_snapshot_ec (push) Successful in 30s Details
Test / test_rm (push) Successful in 18s Details
Test / test_move_reappear (push) Successful in 24s Details
Test / test_snapshot_down (push) Successful in 26s Details
Test / test_snapshot_down_ec (push) Successful in 30s Details
Test / test_splitbrain (push) Successful in 23s Details
Test / test_snapshot_chain (push) Successful in 2m17s Details
Test / test_snapshot_chain_ec (push) Successful in 2m55s Details
Test / test_rebalance_verify_imm (push) Successful in 2m46s Details
Test / test_rebalance_verify (push) Successful in 3m9s Details
Test / test_switch_primary (push) Successful in 39s Details
Test / test_write (push) Successful in 43s Details
Test / test_write_no_same (push) Successful in 19s Details
Test / test_write_xor (push) Successful in 55s Details
Test / test_rebalance_verify_ec (push) Successful in 3m35s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m37s Details
Test / test_heal_pg_size_2 (push) Successful in 3m36s Details
Test / test_heal_ec (push) Successful in 5m47s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m21s Details
Test / test_heal_csum_32k_dj (push) Successful in 6m16s Details
Test / test_heal_csum_32k (push) Successful in 6m45s Details
Test / test_scrub (push) Successful in 1m56s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m39s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m42s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m16s Details
Test / test_scrub_xor (push) Successful in 47s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m26s Details
Test / test_heal_csum_4k (push) Successful in 6m32s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 48s Details
Test / test_scrub_ec (push) Successful in 49s Details
2024-02-21 15:25:02 +03:00
Vitaliy Filippov 685bcd6ef9 Do not reserve extra space for big_writes during sync - sync itself is needed to commit and clear them 2024-02-21 13:00:14 +03:00
Vitaliy Filippov 3eb389b321 Supposed fix for "unexpected state during flush: 0x51" with EC
Test / test_move_reappear (push) Successful in 22s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m32s Details
Test / test_rm (push) Successful in 16s Details
Test / test_snapshot_down (push) Successful in 31s Details
Test / test_snapshot_down_ec (push) Successful in 32s Details
Test / test_splitbrain (push) Successful in 25s Details
Test / test_snapshot_chain (push) Successful in 2m4s Details
Test / test_snapshot_chain_ec (push) Successful in 2m51s Details
Test / test_rebalance_verify_imm (push) Successful in 2m47s Details
Test / test_rebalance_verify (push) Successful in 3m30s Details
Test / test_switch_primary (push) Successful in 38s Details
Test / test_write (push) Successful in 51s Details
Test / test_write_no_same (push) Successful in 16s Details
Test / test_write_xor (push) Successful in 52s Details
Test / test_rebalance_verify_ec (push) Successful in 3m32s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m7s Details
Test / test_scrub_zero_osd_2 (push) Successful in 59s Details
Test / test_scrub (push) Successful in 1m2s Details
Test / test_scrub_xor (push) Successful in 36s Details
Test / test_scrub_ec (push) Successful in 38s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 40s Details
Test / test_scrub_pg_size_3 (push) Successful in 49s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m12s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m8s Details
Test / test_heal_csum_32k (push) Successful in 4m55s Details
Test / test_heal_ec (push) Failing after 10m14s Details
Test / test_heal_csum_4k_dmj (push) Successful in 4m59s Details
Test / test_heal_csum_4k_dj (push) Successful in 5m5s Details
Test / test_heal_pg_size_2 (push) Successful in 3m54s Details
Test / test_heal_csum_4k (push) Successful in 3m49s Details
2024-02-21 01:32:06 +03:00
Vitaliy Filippov 3d16cde23c Fix assertions, add small sequential write test
Test / test_snapshot_down_ec (push) Successful in 32s Details
Test / test_splitbrain (push) Successful in 22s Details
Test / test_snapshot_chain (push) Successful in 2m8s Details
Test / test_snapshot_chain_ec (push) Successful in 2m48s Details
Test / test_rebalance_verify_imm (push) Successful in 2m57s Details
Test / test_rebalance_verify (push) Successful in 3m29s Details
Test / test_switch_primary (push) Successful in 36s Details
Test / test_write (push) Successful in 54s Details
Test / test_write_xor (push) Successful in 51s Details
Test / test_write_no_same (push) Successful in 16s Details
Test / test_rebalance_verify_ec (push) Successful in 3m40s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m20s Details
Test / test_scrub (push) Successful in 1m1s Details
Test / test_scrub_zero_osd_2 (push) Successful in 46s Details
Test / test_scrub_xor (push) Successful in 41s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m0s Details
Test / test_scrub_ec (push) Successful in 58s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m45s Details
Test / test_heal_pg_size_2 (push) Failing after 4m52s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m36s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m33s Details
Test / test_interrupted_rebalance_imm (push) Successful in 1m35s Details
Test / test_interrupted_rebalance (push) Successful in 2m28s Details
Test / test_interrupted_rebalance_ec (push) Successful in 2m30s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 2m41s Details
Test / test_heal_ec (push) Failing after 10m20s Details
Test / test_heal_csum_4k_dmj (push) Successful in 4m21s Details
Test / test_heal_csum_32k (push) Successful in 5m15s Details
Test / test_heal_csum_4k_dj (push) Successful in 5m48s Details
Test / test_heal_csum_4k (push) Successful in 5m32s Details
2024-02-20 19:41:48 +03:00
Vitaliy Filippov c6406d67fc Fix journal space_check incorrectly checking for space at the beginning 2024-02-20 19:40:56 +03:00
Vitaliy Filippov f87964861d Release 1.4.6
Test / test_snapshot_ec (push) Successful in 29s Details
Test / test_rm (push) Successful in 18s Details
Test / test_move_reappear (push) Successful in 26s Details
Test / test_snapshot_down (push) Successful in 28s Details
Test / test_snapshot_down_ec (push) Successful in 32s Details
Test / test_splitbrain (push) Successful in 23s Details
Test / test_snapshot_chain (push) Successful in 2m3s Details
Test / test_snapshot_chain_ec (push) Successful in 2m46s Details
Test / test_rebalance_verify_imm (push) Successful in 3m1s Details
Test / test_rebalance_verify (push) Successful in 3m30s Details
Test / test_switch_primary (push) Successful in 38s Details
Test / test_write (push) Successful in 32s Details
Test / test_write_no_same (push) Successful in 17s Details
Test / test_write_xor (push) Successful in 38s Details
Test / test_rebalance_verify_ec (push) Successful in 4m38s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m57s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m14s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m21s Details
Test / test_heal_csum_32k (push) Successful in 5m45s Details
Test / test_heal_csum_4k_dmj (push) Successful in 5m27s Details
Test / test_scrub (push) Successful in 1m30s Details
Test / test_heal_csum_4k_dj (push) Successful in 5m26s Details
Test / test_scrub_zero_osd_2 (push) Successful in 38s Details
Test / test_scrub_xor (push) Successful in 40s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m8s Details
Test / test_scrub_ec (push) Successful in 1m5s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m49s Details
Test / test_heal_csum_4k (push) Successful in 5m41s Details
Test / test_heal_ec (push) Successful in 4m11s Details
Test / test_heal_pg_size_2 (push) Successful in 4m22s Details
Unwavering stabilization of 1.4.x, continued :-)

- Include the accidentally lost part of 1.4.5 journal trimming fix
- Fix a possible OSD crash with "BUG: Attempt to overwrite used offset"
  which was probably present for long time, but became apparent after
  fixing flapping tests in CI
- Fix remaining flapping tests in CI. It was the first time when tests
  actually passed without retries :-)
2024-02-20 17:01:26 +03:00
Vitaliy Filippov 62a4f45160 Raise test_scrub waiting timeout
Test / test_snapshot_ec (push) Successful in 27s Details
Test / test_rm (push) Successful in 19s Details
Test / test_move_reappear (push) Successful in 25s Details
Test / test_snapshot_down (push) Successful in 28s Details
Test / test_snapshot_down_ec (push) Successful in 33s Details
Test / test_splitbrain (push) Successful in 28s Details
Test / test_snapshot_chain (push) Successful in 2m17s Details
Test / test_snapshot_chain_ec (push) Successful in 3m5s Details
Test / test_rebalance_verify_imm (push) Successful in 3m0s Details
Test / test_rebalance_verify (push) Successful in 3m43s Details
Test / test_switch_primary (push) Successful in 40s Details
Test / test_write (push) Successful in 41s Details
Test / test_write_no_same (push) Successful in 14s Details
Test / test_write_xor (push) Successful in 42s Details
Test / test_rebalance_verify_ec (push) Successful in 3m55s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m6s Details
Test / test_heal_pg_size_2 (push) Successful in 4m51s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m47s Details
Test / test_heal_csum_32k_dmj (push) Successful in 5m50s Details
Test / test_heal_csum_32k (push) Successful in 5m42s Details
Test / test_heal_ec (push) Failing after 10m30s Details
Test / test_heal_csum_4k_dmj (push) Successful in 5m22s Details
Test / test_scrub (push) Successful in 1m21s Details
Test / test_scrub_xor (push) Successful in 46s Details
Test / test_scrub_zero_osd_2 (push) Successful in 53s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m21s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m56s Details
Test / test_scrub_ec (push) Successful in 55s Details
Test / test_heal_csum_4k (push) Successful in 4m28s Details
Test / test_heal_csum_4k_dj (push) Failing after 10m15s Details
2024-02-20 16:26:09 +03:00
Vitaliy Filippov 7048228678 Supposed fix for "BUG: Attempt to overwrite used offset" 2024-02-20 15:56:48 +03:00
Vitaliy Filippov ea73857450 Add asserts to catch "BUG: Attempt to overwrite used offset" 2024-02-20 15:56:48 +03:00
Vitaliy Filippov 6cfe38ec04 Followup to empty cur.oid as stop condition for forced trim fix 2024-02-20 15:56:38 +03:00
Vitaliy Filippov 7ae5766fdb Wait to clear has_degraded in test_heal - should fix flaps of test_heal_* in CI 2024-02-20 15:56:27 +03:00
Vitaliy Filippov f882c7dd87 Release 1.4.5
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m23s Details
Test / test_rm (push) Successful in 15s Details
Test / test_move_reappear (push) Successful in 21s Details
Test / test_snapshot_down (push) Successful in 26s Details
Test / test_snapshot_down_ec (push) Successful in 30s Details
Test / test_splitbrain (push) Successful in 29s Details
Test / test_snapshot_chain (push) Successful in 2m17s Details
Test / test_snapshot_chain_ec (push) Successful in 3m14s Details
Test / test_rebalance_verify_imm (push) Successful in 3m24s Details
Test / test_rebalance_verify (push) Successful in 3m59s Details
Test / test_switch_primary (push) Successful in 35s Details
Test / test_write_xor (push) Successful in 32s Details
Test / test_write_no_same (push) Successful in 13s Details
Test / test_rebalance_verify_ec (push) Successful in 3m46s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m13s Details
Test / test_heal_pg_size_2 (push) Successful in 3m52s Details
Test / test_heal_ec (push) Successful in 5m25s Details
Test / test_heal_csum_32k_dj (push) Successful in 4m24s Details
Test / test_heal_csum_4k_dmj (push) Successful in 4m23s Details
Test / test_heal_csum_4k_dj (push) Successful in 4m17s Details
Test / test_scrub (push) Successful in 38s Details
Test / test_scrub_zero_osd_2 (push) Successful in 29s Details
Test / test_scrub_xor (push) Successful in 30s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 43s Details
Test / test_scrub_ec (push) Successful in 32s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m46s Details
Test / test_heal_csum_4k (push) Successful in 4m4s Details
Test / test_write (push) Successful in 1m38s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m5s Details
Test / test_heal_csum_32k (push) Successful in 4m15s Details
- Fix a write stall caused by incorrect journal trimming introduced in 1.4.4 :)
- Fix PGs sometimes hanging in "starting" state on mass OSD restarts
- Fix a rare crash with "map::at" during OSD pings
- Use new defaults for non-capacitor (desktop) SSDs - improves T1Q256 random write from ~6k iops to ~45k iops
- Make journal_trim_interval configurable
2024-02-16 10:13:33 +03:00
Vitaliy Filippov 26dd863c8d Fix sometimes possible crash on clients.at() during pings 2024-02-16 10:13:33 +03:00
Vitaliy Filippov 2ae859fbc6 Use min/max_flusher_count=32/256, 128M journal and autosync_writes=512 for non-capacitor SSDs by default 2024-02-16 10:13:33 +03:00
Vitaliy Filippov f6cd9f9153 Add a note about pg_minsize 2024-02-15 23:38:52 +03:00
Vitaliy Filippov 8389c0f33b Fix PGs sometimes hanging in "starting" state on mass OSD restarts 2024-02-15 23:38:52 +03:00
Vitaliy Filippov 9db2196aef Make journal_trim_interval configurable 2024-02-15 23:38:51 +03:00
Vitaliy Filippov 8d6ae662fe Use empty cur.oid as stop condition for forced trim, not journal_trim_counter 2024-02-15 23:27:17 +03:00
Vitaliy Filippov c777a0041a Release 1.4.4
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m23s Details
Test / test_move_reappear (push) Successful in 21s Details
Test / test_rm (push) Successful in 16s Details
Test / test_snapshot_down (push) Successful in 30s Details
Test / test_snapshot_down_ec (push) Successful in 30s Details
Test / test_splitbrain (push) Successful in 25s Details
Test / test_snapshot_chain (push) Successful in 2m18s Details
Test / test_snapshot_chain_ec (push) Successful in 3m13s Details
Test / test_rebalance_verify_imm (push) Successful in 3m8s Details
Test / test_rebalance_verify (push) Successful in 3m41s Details
Test / test_switch_primary (push) Successful in 36s Details
Test / test_write (push) Successful in 40s Details
Test / test_write_no_same (push) Successful in 16s Details
Test / test_write_xor (push) Successful in 39s Details
Test / test_rebalance_verify_ec (push) Successful in 4m56s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m21s Details
Test / test_heal_pg_size_2 (push) Successful in 4m15s Details
Test / test_heal_ec (push) Successful in 5m1s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m32s Details
Test / test_heal_csum_32k (push) Successful in 5m38s Details
Test / test_heal_csum_4k_dmj (push) Successful in 5m43s Details
Test / test_scrub (push) Successful in 1m31s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m17s Details
Test / test_heal_csum_4k_dj (push) Successful in 5m57s Details
Test / test_scrub_xor (push) Successful in 30s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m7s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 41s Details
Test / test_scrub_ec (push) Successful in 24s Details
Test / test_heal_csum_32k_dmj (push) Successful in 3m56s Details
Test / test_heal_csum_4k (push) Successful in 3m16s Details
A couple of fixes for EC pools

- Fix a segfault possible on partial EC overwrite in 1234 -> 5030 rebalance scenario
- Fix two problems leading to EC pools stalling on rebalance & parallel sudden stops
  of OSDs, for example during a sudden poweroff of a host:
  - Recovery auto-tuning (1.4.0 feature) could apply too large delays and stall
    the EC journal - fixed by limiting delays with a new recovery_tune_sleep_cutoff_us
    parameter (10 seconds by default) and applying recovery pauses before write
    operations, not after them, to not occupy space in the journal for long time
  - Dynamic journal space reservation (1.3.0 feature) wasn't accounting new writes
    when checking the limit so OSDs could still fill the journal fully and stall -
    fixed by including new writes into the limit
- Print etcd dbSize instead of dbSizeInUse in status
2024-02-11 16:23:08 +03:00
Vitaliy Filippov 2947ea93e8 Raise test_snapshot_chain_ec timeout to 6 minutes 2024-02-11 16:13:52 +03:00
Vitaliy Filippov 978bdc128a Apply recovery pause before writes, after commits, and do not apply it to syncs to not block EC pools from functioning 2024-02-11 16:13:52 +03:00
Vitaliy Filippov bb2f395f1e Add cutoff threshold for recovery auto-tuning 2024-02-11 16:13:52 +03:00
Vitaliy Filippov b127da40f7 Add a FIXME about incomplete PGs 2024-02-11 13:42:51 +03:00
Vitaliy Filippov ca34a6047a Fix dynamic journal space reservation: include the new write itself, too 2024-02-11 13:42:51 +03:00
Vitaliy Filippov 38ba76e893 Fix flusher sometimes being unable to trim journal when the flush queue is empty 2024-02-11 13:42:51 +03:00
Vitaliy Filippov 1e3c4edea0 Print etcd dbSize instead of dbSizeInUse in status 2024-02-11 13:42:51 +03:00
Vitaliy Filippov e7ac855b07 Fix that EC segfault (1234 -> 5030 partial overwrite) 2024-02-11 13:42:51 +03:00
Vitaliy Filippov c53357ac45 Add a test for EC segfault with partial overwrite in 1234 -> 5030 rebalance scenario 2024-02-11 13:42:51 +03:00
Vitaliy Filippov 27e9f244ec Release 1.4.3
Test / test_move_reappear (push) Successful in 22s Details
Test / test_rm (push) Successful in 15s Details
Test / test_snapshot_down (push) Successful in 36s Details
Test / test_snapshot_down_ec (push) Successful in 30s Details
Test / test_interrupted_rebalance (push) Successful in 5m3s Details
Test / test_splitbrain (push) Successful in 20s Details
Test / test_snapshot_chain (push) Successful in 3m1s Details
Test / test_snapshot_chain_ec (push) Successful in 3m13s Details
Test / test_rebalance_verify_imm (push) Successful in 3m0s Details
Test / test_rebalance_verify (push) Successful in 3m29s Details
Test / test_switch_primary (push) Successful in 37s Details
Test / test_write (push) Successful in 44s Details
Test / test_write_xor (push) Successful in 39s Details
Test / test_write_no_same (push) Successful in 16s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m13s Details
Test / test_rebalance_verify_ec (push) Successful in 5m31s Details
Test / test_heal_ec (push) Successful in 4m54s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m25s Details
Test / test_heal_csum_32k (push) Successful in 6m8s Details
Test / test_heal_csum_4k_dmj (push) Successful in 6m17s Details
Test / test_scrub (push) Successful in 1m8s Details
Test / test_scrub_zero_osd_2 (push) Successful in 55s Details
Test / test_scrub_xor (push) Successful in 45s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m22s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m11s Details
Test / test_scrub_ec (push) Successful in 46s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m39s Details
Test / test_heal_csum_4k (push) Successful in 6m8s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m15s Details
Test / test_heal_pg_size_2 (push) Successful in 4m41s Details
Hotfix for hotfix O:-)

- "Write stall fix" was incomplete and EC write stalls could
  continue even on 1.4.2. Now they're finally fixed O:-)
- Make monitor ignore statistics of stopped OSDs. Previously if you stopped all
  OSDs the last total I/O numbers would remain the same indefinitely
2024-02-09 00:29:31 +03:00
Vitaliy Filippov 8e25a28a08 Ignore down OSDs in monitor statistics aggregation
Test / test_move_reappear (push) Successful in 20s Details
Test / test_interrupted_rebalance_ec (push) Successful in 2m50s Details
Test / test_snapshot_down (push) Successful in 22s Details
Test / test_snapshot_down_ec (push) Successful in 25s Details
Test / test_splitbrain (push) Successful in 18s Details
Test / test_snapshot_chain (push) Successful in 2m10s Details
Test / test_snapshot_chain_ec (push) Failing after 3m8s Details
Test / test_rebalance_verify (push) Successful in 3m6s Details
Test / test_interrupted_rebalance (push) Failing after 10m52s Details
Test / test_rebalance_verify_imm (push) Successful in 5m28s Details
Test / test_switch_primary (push) Successful in 37s Details
Test / test_write (push) Successful in 42s Details
Test / test_write_xor (push) Successful in 38s Details
Test / test_write_no_same (push) Successful in 16s Details
Test / test_rebalance_verify_ec (push) Successful in 6m7s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 6m3s Details
Test / test_heal_pg_size_2 (push) Successful in 4m12s Details
Test / test_heal_ec (push) Successful in 5m20s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m53s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m23s Details
Test / test_heal_csum_32k (push) Successful in 5m59s Details
Test / test_scrub_zero_osd_2 (push) Has been cancelled Details
Test / test_scrub_xor (push) Has been cancelled Details
Test / test_heal_csum_4k_dmj (push) Has been cancelled Details
Test / test_scrub_pg_size_3 (push) Has been cancelled Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Has been cancelled Details
Test / test_scrub_ec (push) Has been cancelled Details
Test / test_scrub (push) Has been cancelled Details
Test / test_heal_csum_4k_dj (push) Has been cancelled Details
Test / test_heal_csum_4k (push) Has been cancelled Details
2024-02-09 00:22:36 +03:00
Vitaliy Filippov 5d3317e4f2 Followup to 1.4.2 write stall fix - sadly, the previous version was not working correctly :)
Test / test_move_reappear (push) Successful in 19s Details
Test / test_snapshot_chain (push) Successful in 1m21s Details
Test / test_snapshot_down (push) Successful in 23s Details
Test / test_snapshot_chain_ec (push) Successful in 1m50s Details
Test / test_snapshot_down_ec (push) Successful in 22s Details
Test / test_splitbrain (push) Successful in 16s Details
Test / test_etcd_fail (push) Successful in 6m42s Details
Test / test_rebalance_verify_imm (push) Successful in 2m19s Details
Test / test_rebalance_verify (push) Successful in 4m7s Details
Test / test_switch_primary (push) Successful in 36s Details
Test / test_write (push) Successful in 35s Details
Test / test_rebalance_verify_ec (push) Successful in 4m6s Details
Test / test_write_no_same (push) Successful in 22s Details
Test / test_write_xor (push) Successful in 1m34s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 6m7s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m7s Details
Test / test_heal_csum_32k_dj (push) Successful in 4m59s Details
Test / test_heal_csum_32k (push) Successful in 5m4s Details
Test / test_heal_csum_4k_dmj (push) Successful in 5m59s Details
Test / test_scrub (push) Successful in 1m9s Details
Test / test_scrub_zero_osd_2 (push) Successful in 37s Details
Test / test_scrub_xor (push) Successful in 52s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m5s Details
Test / test_heal_csum_4k_dj (push) Successful in 5m12s Details
Test / test_heal_csum_4k (push) Successful in 5m1s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m48s Details
Test / test_scrub_ec (push) Successful in 19s Details
Test / test_interrupted_rebalance (push) Successful in 1m38s Details
Test / test_heal_pg_size_2 (push) Successful in 3m20s Details
Test / test_heal_ec (push) Successful in 3m3s Details
2024-02-08 19:34:29 +03:00
Vitaliy Filippov 016115c0d4 Release 1.4.2
Test / test_rm (push) Successful in 16s Details
Test / test_move_reappear (push) Successful in 20s Details
Test / test_snapshot_down (push) Successful in 25s Details
Test / test_snapshot_down_ec (push) Successful in 39s Details
Test / test_interrupted_rebalance (push) Successful in 4m52s Details
Test / test_splitbrain (push) Successful in 20s Details
Test / test_snapshot_chain (push) Successful in 3m11s Details
Test / test_rebalance_verify_imm (push) Successful in 3m16s Details
Test / test_rebalance_verify (push) Successful in 3m45s Details
Test / test_switch_primary (push) Successful in 36s Details
Test / test_write (push) Successful in 40s Details
Test / test_write_xor (push) Successful in 40s Details
Test / test_write_no_same (push) Successful in 17s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 3m8s Details
Test / test_rebalance_verify_ec (push) Successful in 5m57s Details
Test / test_heal_pg_size_2 (push) Successful in 4m22s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m20s Details
Test / test_heal_ec (push) Successful in 5m54s Details
Test / test_heal_csum_32k_dj (push) Successful in 5m24s Details
Test / test_heal_csum_32k (push) Successful in 6m3s Details
Test / test_heal_csum_4k_dmj (push) Successful in 5m54s Details
Test / test_scrub_zero_osd_2 (push) Successful in 53s Details
Test / test_scrub (push) Successful in 55s Details
Test / test_heal_csum_4k_dj (push) Successful in 6m14s Details
Test / test_scrub_xor (push) Successful in 1m1s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m50s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 57s Details
Test / test_scrub_ec (push) Successful in 52s Details
Test / test_heal_csum_4k (push) Successful in 5m47s Details
Test / test_snapshot_chain_ec (push) Successful in 1m24s Details
- Log to systemd by default
- Fix excessive autosyncs after every operation with disabled immediate_commit (introduced in 1.1.0)
- Fix a possible write stall with EC due to the lack of OSD wakeup after stabilizing previous writes
- Change sync operation semantics as a final fix to possible write stalls with EC and disabled immediate_commit
- Sync after deleting data in CLI rm / rm-data if immediate_commit is disabled
- Fix OSDs ignoring syncs & autosyncs for delete operations
- Fix OSD space reporting sometimes adding garbage zeros for deleted inodes (causing extra pool/stats etcd keys for deleted pools)
- Speed up monitor failover - change default etcd_mon_ttl from 30 to 5 seconds
- Speed up operation retries - change default up_wait_retry_interval to 50 ms
- Add patch for libvirt 9.10
2024-02-04 02:23:49 +03:00
Vitaliy Filippov e026de95d5 Log to systemd by default
Test / test_move_reappear (push) Successful in 20s Details
Test / test_etcd_fail (push) Successful in 5m19s Details
Test / test_snapshot_chain (push) Successful in 1m26s Details
Test / test_snapshot_down (push) Successful in 26s Details
Test / test_snapshot_down_ec (push) Successful in 28s Details
Test / test_splitbrain (push) Successful in 19s Details
Test / test_snapshot_chain_ec (push) Failing after 3m8s Details
Test / test_interrupted_rebalance (push) Successful in 7m44s Details
Test / test_rebalance_verify_imm (push) Successful in 3m11s Details
Test / test_switch_primary (push) Successful in 34s Details
Test / test_write (push) Successful in 34s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 2m41s Details
Test / test_rebalance_verify_ec (push) Successful in 3m18s Details
Test / test_write_no_same (push) Successful in 22s Details
Test / test_write_xor (push) Successful in 1m41s Details
Test / test_heal_pg_size_2 (push) Failing after 3m54s Details
Test / test_rebalance_verify (push) Successful in 9m38s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m4s Details
Test / test_heal_csum_32k_dj (push) Successful in 4m23s Details
Test / test_heal_csum_32k (push) Successful in 5m24s Details
Test / test_heal_ec (push) Failing after 10m18s Details
Test / test_heal_csum_4k_dmj (push) Successful in 5m31s Details
Test / test_scrub (push) Successful in 1m18s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m0s Details
Test / test_scrub_xor (push) Successful in 51s Details
Test / test_heal_csum_4k_dj (push) Successful in 5m10s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 48s Details
Test / test_scrub_pg_size_3 (push) Successful in 1m59s Details
Test / test_scrub_ec (push) Successful in 48s Details
Test / test_heal_csum_4k (push) Successful in 4m39s Details
2024-02-04 01:21:31 +03:00
Vitaliy Filippov 77c10fd1f8 In fact, do not autosync blockstore when autosync_writes=0
Test / test_move_reappear (push) Successful in 19s Details
Test / test_rm (push) Successful in 14s Details
Test / test_snapshot_down (push) Successful in 24s Details
Test / test_snapshot_down_ec (push) Successful in 25s Details
Test / test_splitbrain (push) Successful in 17s Details
Test / test_snapshot_chain (push) Successful in 1m57s Details
Test / test_snapshot_chain_ec (push) Successful in 2m41s Details
Test / test_rebalance_verify (push) Successful in 3m5s Details
Test / test_rebalance_verify_imm (push) Successful in 2m26s Details
Test / test_switch_primary (push) Successful in 45s Details
Test / test_write (push) Successful in 33s Details
Test / test_write_xor (push) Successful in 33s Details
Test / test_rebalance_verify_ec (push) Successful in 3m42s Details
Test / test_write_no_same (push) Successful in 14s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 4m57s Details
Test / test_heal_csum_32k_dmj (push) Successful in 4m24s Details
Test / test_heal_csum_32k_dj (push) Successful in 4m29s Details
Test / test_heal_csum_4k_dmj (push) Successful in 5m10s Details
Test / test_heal_csum_32k (push) Successful in 5m13s Details
Test / test_scrub (push) Successful in 1m5s Details
Test / test_scrub_zero_osd_2 (push) Successful in 1m1s Details
Test / test_scrub_xor (push) Successful in 1m2s Details
Test / test_heal_csum_4k_dj (push) Successful in 5m2s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 57s Details
Test / test_scrub_ec (push) Successful in 50s Details
Test / test_scrub_pg_size_3 (push) Successful in 2m1s Details
Test / test_heal_csum_4k (push) Successful in 4m40s Details
Test / test_interrupted_rebalance (push) Successful in 1m38s Details
Test / test_heal_pg_size_2 (push) Successful in 4m2s Details
Test / test_heal_ec (push) Successful in 5m17s Details
2024-02-03 20:37:36 +03:00
Vitaliy Filippov 581d02e581 Mark secondary OSDs with deletions as dirty to not forget to sync & autosync them
Test / test_change_pg_count (push) Has been cancelled Details
Test / test_rm (push) Has been cancelled Details
Test / test_snapshot_chain (push) Has been cancelled Details
Test / test_snapshot_chain_ec (push) Has been cancelled Details
Test / test_snapshot_down (push) Has been cancelled Details
Test / test_snapshot_down_ec (push) Has been cancelled Details
Test / test_splitbrain (push) Has been cancelled Details
Test / test_rebalance_verify (push) Has been cancelled Details
Test / test_rebalance_verify_imm (push) Has been cancelled Details
Test / test_rebalance_verify_ec (push) Has been cancelled Details
Test / test_rebalance_verify_ec_imm (push) Has been cancelled Details
Test / test_switch_primary (push) Has been cancelled Details
Test / test_write (push) Has been cancelled Details
Test / test_write_xor (push) Has been cancelled Details
Test / test_write_no_same (push) Has been cancelled Details
Test / test_heal_pg_size_2 (push) Has been cancelled Details
Test / test_heal_ec (push) Has been cancelled Details
Test / test_heal_csum_32k_dmj (push) Has been cancelled Details
Test / test_cas (push) Has been cancelled Details
Test / test_heal_csum_32k_dj (push) Has been cancelled Details
Test / test_heal_csum_32k (push) Has been cancelled Details
Test / test_heal_csum_4k_dmj (push) Has been cancelled Details
Test / test_heal_csum_4k_dj (push) Has been cancelled Details
Test / test_heal_csum_4k (push) Has been cancelled Details
Test / test_scrub (push) Has been cancelled Details
Test / test_scrub_zero_osd_2 (push) Has been cancelled Details
Test / test_scrub_xor (push) Has been cancelled Details
Test / test_scrub_pg_size_3 (push) Has been cancelled Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Has been cancelled Details
Test / test_scrub_ec (push) Has been cancelled Details
2024-02-03 20:31:08 +03:00
Vitaliy Filippov f03a9db4d9 Fix OSD space reporting sometimes adding garbage zeros for deleted inodes (causing extra pool/stats etcd keys for deleted pools) 2024-02-03 20:31:08 +03:00
Vitaliy Filippov cb9c30bc31 Sync after sending all deletes to each PG in cli rm-data 2024-02-03 20:31:08 +03:00
Vitaliy Filippov a86a380d20 Fix invalid parsing of autosync_writes in blockstore leading to autosyncs after every operation with disabled immediate_commit :D 2024-02-03 20:31:08 +03:00
Vitaliy Filippov d2b43cb118 Change default etcd_mon_ttl
Test / test_move_reappear (push) Successful in 35s Details
Test / test_interrupted_rebalance_ec (push) Successful in 3m29s Details
Test / test_interrupted_rebalance (push) Successful in 4m47s Details
Test / test_snapshot_down (push) Successful in 29s Details
Test / test_snapshot_down_ec (push) Successful in 25s Details
Test / test_splitbrain (push) Successful in 24s Details
Test / test_snapshot_chain (push) Successful in 2m46s Details
Test / test_snapshot_chain_ec (push) Failing after 3m10s Details
Test / test_rebalance_verify_imm (push) Successful in 4m24