Commit Graph

16 Commits (master)

Author SHA1 Message Date
Vitaliy Filippov f757a35a8d Retry PG changes without re-running lpsolve when pool configuration and OSD tree don't change
OSDs often change their /pg/history keys during rebalance, so monitor receives additional
transaction failures from etcd if it re-runs lpsolve which sometimes may even lead to monitor
being unable to apply PG changes at all until rebalance completes
2023-12-31 01:23:17 +03:00
Vitaliy Filippov a4186e20aa First derive, then sum per-OSD statistics instead of first summing and then deriving
Test / test_change_pg_count (push) Successful in 43s Details
Test / test_change_pg_count_ec (push) Successful in 37s Details
Test / test_change_pg_size (push) Successful in 8s Details
Test / test_create_nomaxid (push) Successful in 8s Details
Test / test_failure_domain (push) Successful in 16s Details
Test / test_interrupted_rebalance (push) Successful in 1m49s Details
Test / test_interrupted_rebalance_imm (push) Successful in 1m38s Details
Test / test_interrupted_rebalance_ec (push) Successful in 1m49s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m23s Details
Test / test_minsize_1 (push) Successful in 13s Details
Test / test_move_reappear (push) Successful in 16s Details
Test / test_rebalance_verify (push) Successful in 3m2s Details
Test / test_rebalance_verify_imm (push) Successful in 2m53s Details
Test / test_rebalance_verify_ec (push) Successful in 3m9s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 5m27s Details
Test / test_rm (push) Successful in 17s Details
Test / test_snapshot (push) Successful in 34s Details
Test / test_snapshot_ec (push) Successful in 29s Details
Test / test_splitbrain (push) Successful in 22s Details
Test / test_write (push) Successful in 37s Details
Test / test_write_xor (push) Successful in 44s Details
Test / test_write_no_same (push) Successful in 16s Details
Test / test_heal_pg_size_2 (push) Successful in 3m31s Details
Test / test_heal_ec (push) Successful in 4m20s Details
Test / test_scrub (push) Successful in 38s Details
Test / test_scrub_zero_osd_2 (push) Successful in 30s Details
Test / test_scrub_xor (push) Successful in 32s Details
Test / test_scrub_pg_size_3 (push) Successful in 42s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 37s Details
Test / test_scrub_ec (push) Successful in 34s Details
This makes statistics reported by vitastor-cli status much smoother
2023-06-18 01:32:24 +03:00
Vitaliy Filippov e69d459d43 Allow rebalance to start in test_interrupted_rebalance, raise etcd start timeout 2023-05-13 15:16:28 +03:00
Vitaliy Filippov be4087d9d2 Add a FIXME to test_interrupted_rebalance 2022-06-05 00:06:56 +03:00
Vitaliy Filippov c4eb46600d Merge run_3osds and run_7osds scripts 2022-06-03 01:56:36 +03:00
Vitaliy Filippov 834554c523 LD_PRELOAD=libasan.so.5 fio in tests fails when vitastor is built with ASan 2022-05-05 02:11:34 +03:00
Vitaliy Filippov 0eb929f1ba Fix change_pg_count test (statistic reporting may take some time) 2022-04-08 11:58:53 +03:00
Vitaliy Filippov 8a6b07d8f7 Add a 2/5 etcd failure test 2022-01-20 00:43:22 +03:00
Vitaliy Filippov eb7ad2c114 Fix empty size syntax, use C version of simple-offsets in tests 2021-11-20 23:51:26 +03:00
Vitaliy Filippov 05db1308aa Fix two potential read/write ordering problems (even though not yet seen in tests)
- Write operations could be 'stabilized' and previous versions could be
  purged from OSDs before the removal of version_override and following
  reads could potentially hit different version in EC pools
- Object was marked clean after completing the delete during recovery, so
  reads could in theory hit a deleted version and return nothing
2021-03-24 14:20:56 +03:00
Vitaliy Filippov dd76eda5e5 Test multiple interrupted rebalancings
Currently only passes with immediate_commit=all configuration
(env variable IMMEDIATE_COMMIT=1 for the bash script)
2021-03-12 12:55:44 +03:00
Vitaliy Filippov 2e0c853180 Make test_change_pg_count check if any objects are lost during the test 2021-03-08 23:15:07 +03:00
Vitaliy Filippov 97795ea1b1 Use pg_minsize=2 in the pg_count change test
Also don't check for has_degraded because it's not a bug that objects
are _temporarily_ listed as degraded during PG peering as it's not
required for the new primary to connect to _all_ older peers to start
peering. The test may be improved in the future by temporarily disabling
degraded recovery during it and returning the has_degraded check back.
2021-03-01 16:18:08 +03:00
Vitaliy Filippov 24e7075f08 Fix monitor's statistics aggregation 2021-02-28 19:51:16 +03:00
Vitaliy Filippov 7d49706c07 Improve the pg_count change test: add more OSDs and actually move data between them 2021-02-28 19:36:59 +03:00
Vitaliy Filippov 69c87009e9 Add a test for changing PG count 2021-02-25 23:59:33 +03:00