Compare commits

...

140 Commits

Author SHA1 Message Date
Vitaliy Filippov cdfc74665b Close client FDs only when destroying the client, after handling all async reads/writes
Test / test_change_pg_count (push) Successful in 50s Details
Test / test_change_pg_count_ec (push) Successful in 58s Details
Test / test_change_pg_size (push) Successful in 17s Details
Test / test_create_nomaxid (push) Successful in 19s Details
Test / test_etcd_fail (push) Successful in 58s Details
Test / test_failure_domain (push) Successful in 14s Details
Test / test_interrupted_rebalance (push) Successful in 1m31s Details
Test / test_interrupted_rebalance_imm (push) Successful in 1m0s Details
Test / test_interrupted_rebalance_ec (push) Successful in 1m23s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m16s Details
Test / test_minsize_1 (push) Successful in 38s Details
Test / test_move_reappear (push) Successful in 54s Details
Test / test_rebalance_verify (push) Successful in 2m26s Details
Test / test_rebalance_verify_imm (push) Successful in 2m7s Details
Test / test_rebalance_verify_ec (push) Successful in 2m51s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 2m15s Details
Test / test_rm (push) Successful in 22s Details
Test / test_snapshot (push) Successful in 40s Details
Test / test_snapshot_ec (push) Successful in 34s Details
Test / test_splitbrain (push) Successful in 23s Details
Test / test_write (push) Successful in 1m7s Details
Test / test_write_xor (push) Successful in 2m13s Details
Test / test_write_no_same (push) Successful in 18s Details
Test / test_heal_pg_size_2 (push) Successful in 5m14s Details
Test / test_scrub (push) Successful in 36s Details
Test / test_scrub_zero_osd_2 (push) Successful in 40s Details
Test / test_scrub_xor (push) Successful in 54s Details
Test / test_scrub_pg_size_3 (push) Successful in 54s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 1m12s Details
Test / test_scrub_ec (push) Successful in 1m10s Details
Fixes "Client XX command out of sync" sometimes happening on reconnections
3 days ago
Vitaliy Filippov 3f60fecd7c Fix typo 7 days ago
Vitaliy Filippov 3b4cf29e65 Release 0.9.0
New features:
- Scrubbing! Check documentation: [auto_scrub](src/branch/master/docs/config/osd.en.md#auto_scrub)
- Document online-updatable configuration parameters

Bug fixes:
- Fix NaN during PG optimisation if there are nonexisting OSDs in node_placement
- Fix monitor crash on pool deletion
- Clear journal_device and meta_device before initialising the next OSD in automatic mode
- Sync unsynced deletes before overwriting them with a lower version
  (reproducted mostly/only after scrubbing)
7 days ago
Vitaliy Filippov eeaba11ebd Use fio 3.27-8 for alma9 7 days ago
Vitaliy Filippov aea567cfbd Slightly improve scrub docs
Test / test_cas (push) Successful in 9s Details
Test / test_change_pg_count (push) Successful in 52s Details
Test / test_change_pg_count_ec (push) Successful in 1m0s Details
Test / test_change_pg_size (push) Successful in 16s Details
Test / test_create_nomaxid (push) Successful in 16s Details
Test / test_etcd_fail (push) Successful in 56s Details
Test / test_failure_domain (push) Successful in 13s Details
Test / test_interrupted_rebalance (push) Successful in 1m24s Details
Test / test_interrupted_rebalance_imm (push) Successful in 1m10s Details
Test / test_interrupted_rebalance_ec (push) Successful in 1m9s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m6s Details
Test / test_minsize_1 (push) Failing after 19s Details
Test / test_move_reappear (push) Successful in 28s Details
Test / test_rebalance_verify (push) Successful in 2m25s Details
Test / test_rebalance_verify_imm (push) Successful in 2m19s Details
Test / test_rebalance_verify_ec (push) Successful in 3m3s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 2m20s Details
Test / test_rm (push) Successful in 16s Details
Test / test_snapshot (push) Successful in 21s Details
Test / test_snapshot_ec (push) Successful in 28s Details
Test / test_splitbrain (push) Successful in 20s Details
Test / test_write_xor (push) Has started running Details
Test / test_heal_pg_size_2 (push) Has started running Details
Test / test_write (push) Has started running Details
Test / test_scrub (push) Has been cancelled Details
Test / test_scrub_zero_osd_2 (push) Has been cancelled Details
Test / test_scrub_xor (push) Has been cancelled Details
Test / test_scrub_pg_size_3 (push) Has been cancelled Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Has been cancelled Details
Test / test_scrub_ec (push) Has been cancelled Details
7 days ago
Vitaliy Filippov ce02f47de6 Allow to disable scrub_find_best 7 days ago
Vitaliy Filippov 5fd3208616 Add version archive link to docs 7 days ago
Vitaliy Filippov 5997b76535 Remove -runtime=10 from fio params in test_scrub, it was breaking the test in CI :D
Test / test_change_pg_count (push) Successful in 49s Details
Test / test_change_pg_count_ec (push) Successful in 2m53s Details
Test / test_change_pg_size (push) Successful in 17s Details
Test / test_create_nomaxid (push) Successful in 13s Details
Test / test_etcd_fail (push) Successful in 1m0s Details
Test / test_failure_domain (push) Successful in 13s Details
Test / test_interrupted_rebalance (push) Successful in 1m18s Details
Test / test_interrupted_rebalance_imm (push) Successful in 1m3s Details
Test / test_interrupted_rebalance_ec (push) Successful in 1m40s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 52s Details
Test / test_minsize_1 (push) Successful in 22s Details
Test / test_move_reappear (push) Successful in 23s Details
Test / test_rebalance_verify (push) Successful in 2m32s Details
Test / test_rebalance_verify_imm (push) Successful in 2m29s Details
Test / test_rebalance_verify_ec (push) Successful in 2m55s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 2m30s Details
Test / test_rm (push) Successful in 22s Details
Test / test_snapshot (push) Successful in 26s Details
Test / test_snapshot_ec (push) Successful in 38s Details
Test / test_splitbrain (push) Successful in 28s Details
Test / test_write (push) Successful in 1m5s Details
Test / test_write_xor (push) Successful in 2m13s Details
Test / test_write_no_same (push) Successful in 18s Details
Test / test_heal_ec (push) Successful in 5m27s Details
Test / test_scrub (push) Successful in 36s Details
Test / test_scrub_zero_osd_2 (push) Successful in 39s Details
Test / test_scrub_xor (push) Successful in 1m2s Details
Test / test_scrub_pg_size_3 (push) Successful in 50s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 29s Details
Test / test_scrub_ec (push) Successful in 1m25s Details
7 days ago
Vitaliy Filippov f1961157f0 Fix brute-force error locator for EC n+k with k > 2
Test / test_change_pg_count_ec (push) Successful in 2m23s Details
Test / test_change_pg_size (push) Successful in 20s Details
Test / test_create_nomaxid (push) Successful in 16s Details
Test / test_etcd_fail (push) Successful in 55s Details
Test / test_failure_domain (push) Successful in 12s Details
Test / test_interrupted_rebalance (push) Successful in 1m18s Details
Test / test_interrupted_rebalance_imm (push) Successful in 1m9s Details
Test / test_interrupted_rebalance_ec (push) Successful in 1m23s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m8s Details
Test / test_minsize_1 (push) Successful in 22s Details
Test / test_move_reappear (push) Successful in 28s Details
Test / test_rebalance_verify (push) Successful in 2m17s Details
Test / test_rebalance_verify_imm (push) Successful in 2m19s Details
Test / test_rebalance_verify_ec (push) Successful in 3m4s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 2m22s Details
Test / test_rm (push) Successful in 23s Details
Test / test_snapshot (push) Successful in 20s Details
Test / test_snapshot_ec (push) Successful in 34s Details
Test / test_splitbrain (push) Successful in 33s Details
Test / test_write (push) Successful in 1m15s Details
Test / test_write_xor (push) Successful in 2m6s Details
Test / test_write_no_same (push) Successful in 16s Details
Test / test_heal_pg_size_2 (push) Successful in 5m22s Details
Test / test_heal_ec (push) Successful in 5m31s Details
Test / test_scrub (push) Successful in 29s Details
Test / test_scrub_zero_osd_2 (push) Successful in 27s Details
Test / test_scrub_xor (push) Successful in 22s Details
Test / test_scrub_pg_size_3 (push) Failing after 37s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 29s Details
Test / test_scrub_ec (push) Failing after 33s Details
1 week ago
Vitaliy Filippov 88c1ba0790 Fix compile errors with gcc 10
Test / build (push) Has started running Details
Test / buildenv (push) Successful in 11s Details
Test / make_test (push) Has been cancelled Details
Test / test_add_osd (push) Has been cancelled Details
Test / test_cas (push) Has been cancelled Details
Test / test_change_pg_count (push) Has been cancelled Details
Test / test_change_pg_count_ec (push) Has been cancelled Details
Test / test_change_pg_size (push) Has been cancelled Details
Test / test_create_nomaxid (push) Has been cancelled Details
Test / test_etcd_fail (push) Has been cancelled Details
Test / test_failure_domain (push) Has been cancelled Details
Test / test_interrupted_rebalance (push) Has been cancelled Details
Test / test_interrupted_rebalance_imm (push) Has been cancelled Details
Test / test_interrupted_rebalance_ec (push) Has been cancelled Details
Test / test_interrupted_rebalance_ec_imm (push) Has been cancelled Details
Test / test_minsize_1 (push) Has been cancelled Details
Test / test_move_reappear (push) Has been cancelled Details
Test / test_rebalance_verify (push) Has been cancelled Details
Test / test_rebalance_verify_imm (push) Has been cancelled Details
Test / test_rebalance_verify_ec (push) Has been cancelled Details
Test / test_rebalance_verify_ec_imm (push) Has been cancelled Details
Test / test_rm (push) Has been cancelled Details
Test / test_snapshot (push) Has been cancelled Details
Test / test_snapshot_ec (push) Has been cancelled Details
Test / test_splitbrain (push) Has been cancelled Details
Test / test_write (push) Has been cancelled Details
Test / test_write_xor (push) Has been cancelled Details
Test / test_write_no_same (push) Has been cancelled Details
Test / test_heal_pg_size_2 (push) Has been cancelled Details
Test / test_heal_ec (push) Has been cancelled Details
1 week ago
Vitaliy Filippov b5bd611683 Add scrub tests to CI 1 week ago
Vitaliy Filippov fa90b5a4e7 Schedule automatic scrubs correctly (not just after previous scrub) 1 week ago
Vitaliy Filippov 8d40ad99a6 Add scrub documentation 1 week ago
Vitaliy Filippov 3475772b07 Add configuration online update documentation 1 week ago
Vitaliy Filippov 25fcedf6e7 Enable vitastor-cli fix in test 1 week ago
Vitaliy Filippov 6ca20aa194 Allow scrub to fix corrupted object states 1 week ago
Vitaliy Filippov 4bfd994341 Sync unsynced deletes before overwriting them with a lower version 1 week ago
Vitaliy Filippov 59e959dcbb Do not die when "different versions are returned from subops" 1 week ago
Vitaliy Filippov a9581f0739 Handle dirty deletes during read correctly O_o 1 week ago
Vitaliy Filippov 105a405b0a Implement vitastor-cli fix 1 week ago
Vitaliy Filippov d55d7d5326 Add scrub test 1 week ago
Vitaliy Filippov 0e5d0e02a9 Add "vitastor-cli describe" command 1 week ago
Vitaliy Filippov 0439981a66 Implement "describe object(s)" operation
Required to implement fixing inconsistent objects in vitastor-cli
1 week ago
Vitaliy Filippov 6648f6bb6e Implement ambiguity detection during scrub 1 week ago
Vitaliy Filippov 281be547eb Implement brute-force error locator for EC 1 week ago
Vitaliy Filippov 0c78dd7178 Add no_scrub flag 1 week ago
Vitaliy Filippov 3c924397e7 Store next scrub timestamp instead of last scrub timestamp 1 week ago
Vitaliy Filippov c3bd26193d Implement PG scrub runner 1 week ago
Vitaliy Filippov 43b77d7619 Implement scrubbing "data path" - OSD_OP_SCRUB 1 week ago
Vitaliy Filippov a6d846863b Add min/max stripe and limit to OP_LIST 1 week ago
Vitaliy Filippov 8dc427b43c Retry failed reads (including chained and RMW) from other replicas 1 week ago
Vitaliy Filippov bf2112653b Refcount object_states 1 week ago
Vitaliy Filippov 0538a484b3 Add corrupted object state 1 week ago
Vitaliy Filippov 97720fa6b4 Remove unused capture
Test / buildenv (push) Successful in 12s Details
Test / build (push) Has started running Details
Test / make_test (push) Has been cancelled Details
Test / test_add_osd (push) Has been cancelled Details
Test / test_cas (push) Has been cancelled Details
Test / test_change_pg_count (push) Has been cancelled Details
Test / test_change_pg_count_ec (push) Has been cancelled Details
Test / test_change_pg_size (push) Has been cancelled Details
Test / test_create_nomaxid (push) Has been cancelled Details
Test / test_etcd_fail (push) Has been cancelled Details
Test / test_failure_domain (push) Has been cancelled Details
Test / test_interrupted_rebalance (push) Has been cancelled Details
Test / test_interrupted_rebalance_imm (push) Has been cancelled Details
Test / test_interrupted_rebalance_ec (push) Has been cancelled Details
Test / test_interrupted_rebalance_ec_imm (push) Has been cancelled Details
Test / test_minsize_1 (push) Has been cancelled Details
Test / test_move_reappear (push) Has been cancelled Details
Test / test_rebalance_verify (push) Has been cancelled Details
Test / test_rebalance_verify_imm (push) Has been cancelled Details
Test / test_rebalance_verify_ec (push) Has been cancelled Details
Test / test_rebalance_verify_ec_imm (push) Has been cancelled Details
Test / test_rm (push) Has been cancelled Details
Test / test_snapshot (push) Has been cancelled Details
Test / test_snapshot_ec (push) Has been cancelled Details
Test / test_splitbrain (push) Has been cancelled Details
Test / test_write (push) Has been cancelled Details
Test / test_write_xor (push) Has been cancelled Details
Test / test_write_no_same (push) Has been cancelled Details
Test / test_heal_pg_size_2 (push) Has been cancelled Details
Test / test_heal_ec (push) Has been cancelled Details
1 week ago
Vitaliy Filippov e60e352df6 Improve vitastor-nbd documentation 1 week ago
Vitaliy Filippov 98077a1712 Remove unused dependencies from CSI 1 week ago
Vitaliy Filippov 1c7d53996d Reweight only 2 OSDs to zero in test_rebalance_verify, otherwise the test does not pass with EC 3+2
Test / buildenv (push) Successful in 9s Details
Test / build (push) Successful in 2m20s Details
Test / test_cas (push) Successful in 11s Details
Test / make_test (push) Successful in 35s Details
Test / test_change_pg_size (push) Successful in 22s Details
Test / test_change_pg_count (push) Successful in 52s Details
Test / test_create_nomaxid (push) Successful in 19s Details
Test / test_change_pg_count_ec (push) Successful in 1m3s Details
Test / test_failure_domain (push) Successful in 13s Details
Test / test_etcd_fail (push) Successful in 1m0s Details
Test / test_interrupted_rebalance_imm (push) Successful in 1m3s Details
Test / test_interrupted_rebalance (push) Successful in 1m14s Details
Test / test_minsize_1 (push) Successful in 22s Details
Test / test_move_reappear (push) Successful in 18s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m1s Details
Test / test_interrupted_rebalance_ec (push) Successful in 1m38s Details
Test / test_rebalance_verify (push) Successful in 2m20s Details
Test / test_rebalance_verify_imm (push) Successful in 2m1s Details
Test / test_rm (push) Successful in 26s Details
Test / test_rebalance_verify_ec (push) Successful in 2m30s Details
Test / test_snapshot (push) Successful in 22s Details
Test / test_snapshot_ec (push) Successful in 28s Details
Test / test_splitbrain (push) Successful in 20s Details
Test / test_write (push) Successful in 48s Details
Test / test_write_no_same (push) Successful in 15s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 2m11s Details
Test / test_write_xor (push) Successful in 1m28s Details
Test / test_heal_pg_size_2 (push) Successful in 4m48s Details
Test / test_heal_ec (push) Successful in 5m12s Details
Test / test_add_osd (push) Successful in 1m20s Details
1 week ago
Vitaliy Filippov 2ca07b1ea7 Raise timeout in test_rebalance_verify
Test / buildenv (push) Successful in 10s Details
Test / build (push) Successful in 2m27s Details
Test / test_cas (push) Successful in 11s Details
Test / make_test (push) Successful in 34s Details
Test / test_change_pg_size (push) Successful in 22s Details
Test / test_change_pg_count (push) Successful in 52s Details
Test / test_create_nomaxid (push) Successful in 8s Details
Test / test_failure_domain (push) Successful in 12s Details
Test / test_etcd_fail (push) Successful in 1m0s Details
Test / test_interrupted_rebalance (push) Successful in 1m15s Details
Test / test_add_osd (push) Successful in 2m33s Details
Test / test_interrupted_rebalance_imm (push) Successful in 1m4s Details
Test / test_change_pg_count_ec (push) Successful in 2m52s Details
Test / test_minsize_1 (push) Successful in 19s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 53s Details
Test / test_move_reappear (push) Successful in 21s Details
Test / test_interrupted_rebalance_ec (push) Successful in 1m36s Details
Test / test_rebalance_verify (push) Successful in 2m22s Details
Test / test_rebalance_verify_imm (push) Successful in 2m22s Details
Test / test_rm (push) Successful in 15s Details
Test / test_snapshot (push) Successful in 19s Details
Test / test_snapshot_ec (push) Successful in 27s Details
Test / test_rebalance_verify_ec (push) Failing after 3m6s Details
Test / test_splitbrain (push) Successful in 17s Details
Test / test_write_no_same (push) Successful in 20s Details
Test / test_rebalance_verify_ec_imm (push) Failing after 3m9s Details
Test / test_write (push) Successful in 49s Details
Test / test_write_xor (push) Successful in 1m17s Details
Test / test_heal_ec (push) Successful in 4m53s Details
Test / test_heal_pg_size_2 (push) Failing after 10m10s Details
2 weeks ago
Vitaliy Filippov 022176aa98 Fix NaN during PG optimisation if there are nonexisting OSDs in node_placement
Test / buildenv (push) Successful in 11s Details
Test / build (push) Successful in 2m28s Details
Test / test_cas (push) Successful in 12s Details
Test / make_test (push) Successful in 40s Details
Test / test_change_pg_size (push) Successful in 23s Details
Test / test_change_pg_count (push) Successful in 1m1s Details
Test / test_create_nomaxid (push) Successful in 7s Details
Test / test_failure_domain (push) Successful in 11s Details
Test / test_change_pg_count_ec (push) Successful in 1m35s Details
Test / test_etcd_fail (push) Successful in 51s Details
Test / test_add_osd (push) Successful in 2m27s Details
Test / test_interrupted_rebalance (push) Successful in 1m14s Details
Test / test_interrupted_rebalance_imm (push) Successful in 1m3s Details
Test / test_minsize_1 (push) Successful in 28s Details
Test / test_move_reappear (push) Successful in 41s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m13s Details
Test / test_interrupted_rebalance_ec (push) Successful in 1m49s Details
Test / test_rebalance_verify (push) Successful in 2m21s Details
Test / test_rm (push) Successful in 15s Details
Test / test_rebalance_verify_imm (push) Successful in 2m12s Details
Test / test_snapshot (push) Successful in 20s Details
Test / test_snapshot_ec (push) Successful in 28s Details
Test / test_splitbrain (push) Successful in 23s Details
Test / test_write_no_same (push) Successful in 17s Details
Test / test_write (push) Successful in 1m6s Details
Test / test_write_xor (push) Successful in 1m42s Details
Test / test_heal_pg_size_2 (push) Successful in 4m57s Details
Test / test_heal_ec (push) Successful in 4m42s Details
Test / test_rebalance_verify_ec_imm (push) Failing after 2m19s Details
Test / test_rebalance_verify_ec (push) Failing after 2m25s Details
2 weeks ago
Vitaliy Filippov 120e3fa7bc Fix pool deletion
Test / buildenv (push) Successful in 10s Details
Test / build (push) Successful in 2m32s Details
Test / test_cas (push) Successful in 13s Details
Test / make_test (push) Successful in 35s Details
Test / test_change_pg_size (push) Successful in 21s Details
Test / test_change_pg_count (push) Successful in 53s Details
Test / test_create_nomaxid (push) Successful in 17s Details
Test / test_change_pg_count_ec (push) Successful in 1m3s Details
Test / test_failure_domain (push) Successful in 16s Details
Test / test_etcd_fail (push) Successful in 1m3s Details
Test / test_add_osd (push) Successful in 2m36s Details
Test / test_interrupted_rebalance_imm (push) Successful in 1m10s Details
Test / test_interrupted_rebalance (push) Successful in 1m24s Details
Test / test_minsize_1 (push) Failing after 28s Details
Test / test_interrupted_rebalance_ec (push) Successful in 1m8s Details
Test / test_move_reappear (push) Failing after 1m2s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m8s Details
Test / test_rebalance_verify_imm (push) Successful in 2m12s Details
Test / test_rebalance_verify (push) Successful in 2m22s Details
Test / test_rm (push) Successful in 21s Details
Test / test_snapshot (push) Successful in 24s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 2m19s Details
Test / test_snapshot_ec (push) Successful in 27s Details
Test / test_splitbrain (push) Successful in 20s Details
Test / test_rebalance_verify_ec (push) Successful in 2m33s Details
Test / test_write_no_same (push) Successful in 15s Details
Test / test_write (push) Successful in 1m14s Details
Test / test_write_xor (push) Successful in 2m9s Details
Test / test_heal_ec (push) Successful in 4m25s Details
Test / test_heal_pg_size_2 (push) Successful in 4m59s Details
2 weeks ago
Vitaliy Filippov 629999f789 Clear journal_device and meta_device before initialising the next OSD in automatic mode 2 weeks ago
Vitaliy Filippov 93eca11ba2 Fix rhel 9 installation docs 2 weeks ago
Vitaliy Filippov 5a9e1ede52 Release 0.8.9
Test / buildenv (push) Successful in 9s Details
Test / build (push) Successful in 2m31s Details
Test / test_cas (push) Successful in 12s Details
Test / make_test (push) Successful in 33s Details
Test / test_change_pg_size (push) Successful in 19s Details
Test / test_change_pg_count (push) Successful in 55s Details
Test / test_create_nomaxid (push) Successful in 21s Details
Test / test_change_pg_count_ec (push) Successful in 58s Details
Test / test_failure_domain (push) Successful in 13s Details
Test / test_etcd_fail (push) Successful in 1m4s Details
Test / test_interrupted_rebalance (push) Successful in 1m13s Details
Test / test_interrupted_rebalance_imm (push) Successful in 1m7s Details
Test / test_add_osd (push) Successful in 2m59s Details
Test / test_move_reappear (push) Successful in 24s Details
Test / test_interrupted_rebalance_ec (push) Successful in 1m22s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m1s Details
Test / test_rebalance_verify (push) Successful in 2m12s Details
Test / test_minsize_1 (push) Successful in 15s Details
Test / test_rebalance_verify_imm (push) Successful in 2m4s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 2m9s Details
Test / test_rm (push) Successful in 17s Details
Test / test_snapshot (push) Successful in 23s Details
Test / test_rebalance_verify_ec (push) Successful in 2m31s Details
Test / test_splitbrain (push) Successful in 23s Details
Test / test_snapshot_ec (push) Successful in 30s Details
Test / test_write_no_same (push) Successful in 16s Details
Test / test_write (push) Successful in 53s Details
Test / test_write_xor (push) Successful in 1m19s Details
Test / test_heal_pg_size_2 (push) Successful in 4m30s Details
Test / test_heal_ec (push) Successful in 4m32s Details
- The tests are now stable and run in a CI system based on Gitea CI
- The release includes final bug fixes for EC:
  - Implement missing EC recovery of allocation bitmap when built with ISA-L
  - Fix broken snapshot export with EC (allocation bitmap reads were giving incorrect results previously)
- Also fixed bugs manifesting under heavy load:
  - Fix monitor possibly applying incorrect PG history on retries
  - Fix monitor incorrectly changing PG count when last_clean_pgs contains less PGs than the new number
  - Allow writes to wait for free space again, but now correctly (previously dropped in 0.8.2)
  - Fix a rare segfault in client (handle client stop during incoming stream handling in 1 more place)
  - Make monitor correctly handle etcd connection errors - it could die instead of connecting to another etcd
  - Fix OSD rarely being unable to report PG states after a PG was taken over by another OSD
- Fixed return code for incomplete EC objects (now EIO) and made cluster client retry this error
- Made other small changes for tests: timeouts, nice/ionice for etcd, waiting conditions, NBD device checks and so on
2 weeks ago
Vitaliy Filippov 1c9a188600 Add tests to CI
Test / buildenv (push) Successful in 10s Details
Test / build (push) Successful in 10s Details
Test / test_cas (push) Successful in 12s Details
Test / make_test (push) Successful in 34s Details
Test / test_change_pg_size (push) Successful in 17s Details
Test / test_create_nomaxid (push) Successful in 9s Details
Test / test_change_pg_count (push) Successful in 1m29s Details
Test / test_failure_domain (push) Successful in 11s Details
Test / test_change_pg_count_ec (push) Successful in 1m35s Details
Test / test_etcd_fail (push) Successful in 52s Details
Test / test_add_osd (push) Successful in 2m13s Details
Test / test_interrupted_rebalance_imm (push) Successful in 1m4s Details
Test / test_interrupted_rebalance (push) Successful in 1m28s Details
Test / test_minsize_1 (push) Successful in 21s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m4s Details
Test / test_move_reappear (push) Successful in 30s Details
Test / test_interrupted_rebalance_ec (push) Successful in 1m53s Details
Test / test_rebalance_verify_imm (push) Successful in 2m14s Details
Test / test_rebalance_verify (push) Successful in 2m16s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 2m4s Details
Test / test_rm (push) Successful in 22s Details
Test / test_snapshot (push) Successful in 28s Details
Test / test_rebalance_verify_ec (push) Successful in 2m27s Details
Test / test_splitbrain (push) Successful in 24s Details
Test / test_snapshot_ec (push) Successful in 34s Details
Test / test_write_no_same (push) Successful in 19s Details
Test / test_write (push) Successful in 1m19s Details
Test / test_write_xor (push) Successful in 1m36s Details
Test / test_heal_pg_size_2 (push) Successful in 4m34s Details
Test / test_heal_ec (push) Successful in 4m21s Details
2 weeks ago
Vitaliy Filippov de3e609166 Add a FIXME about QEMU driver thread safety 2 weeks ago
Vitaliy Filippov 11481170f5 Add a FIXME about ENOSPC 2 weeks ago
Vitaliy Filippov e69d459d43 Allow rebalance to start in test_interrupted_rebalance, raise etcd start timeout 2 weeks ago
Vitaliy Filippov da82754baa Wait for conditions in test_move_reappear instead of waiting a fixed amount of time 2 weeks ago
Vitaliy Filippov d356aca030 Add missing $NO_SAME OSD argument to test_splitbrain 2 weeks ago
Vitaliy Filippov 04a273d213 Raise NBD timeout in tests 2 weeks ago
Vitaliy Filippov 6442010f93 Skip offline PGs during state reporting when the state is already deleted or taken over by another OSD
This fixes OSDs being unable to report PG states in rare conditions
2 weeks ago
Vitaliy Filippov 6f4dc16c59 Handle etcd connection errors correctly in mon (unhandled error events) 2 weeks ago
Vitaliy Filippov ce4a8067b5 Handle client stop during incoming stream handling in 1 more place 2 weeks ago
Vitaliy Filippov e431ecb715 Make tests more stable in CI 2 weeks ago
Vitaliy Filippov 8cac795445 Return EIO instead of EINVAL for incomplete EC objects 2 weeks ago
Vitaliy Filippov a409598b16 Wait for free space again, but count on big_write flushes instead of just flusher activity 3 weeks ago
Vitaliy Filippov f4c6765522 Ignore ENOENT in epoll_ctl 3 weeks ago
Vitaliy Filippov ad2916068a Fix test_add_osd rebalance timeout check 3 weeks ago
Vitaliy Filippov 321cb435a6 Fix monitor incorrectly changing PG count when last_clean_pgs contains less PGs than the new number 3 weeks ago
Vitaliy Filippov cfcf4f4355 Support checking /dev/nbdX nodes in Docker 3 weeks ago
Vitaliy Filippov e0fb17bfee Make etcd more stable in tests (add ionice and raise timeout) 3 weeks ago
Vitaliy Filippov 5b9031fecc Fix monitor possibly applying incorrect PG history under heavy load
Monitor could deceive itself by immediately saving PG configuration changes
which weren't applied to etcd yet in memory, and apply incorrect PG history
changes next time if the first update fails.

This usually only happened under heavy load and was caught in CI. :-)
3 weeks ago
Vitaliy Filippov 5da1d8e1b5 Fix EC just-bitmap reads (len=0) (fixes SCHEME=ec test_snapshot.sh) 3 weeks ago
Vitaliy Filippov 44f86f1999 Add a basic EC 2+2 recovery test (not really required, but let it be there) 3 weeks ago
Vitaliy Filippov 2d9a80c6f6 Implement missing bitmap recovery with ISA-L \(°□°)/ 3 weeks ago
Vitaliy Filippov 5e295e346e Do not make vitastor-mon part of vitastor.target 4 weeks ago
Vitaliy Filippov d9c0898b7c Notes about config and vitastor-disk cache status 4 weeks ago
Vitaliy Filippov 04cfb48361 Add a note about PVE 7.4 4 weeks ago
Vitaliy Filippov ab615849d6 Release 0.8.8
- Fix vitastor-cli rm/rm-data broken in 0.8.6 (missing messenger initialization)
- Prepare OSD read handler for upcoming version with scrub - allow "secondary reads" to return errors
- Fix OSDs re-peering PGs infinitely with a big number of PGs (reproduced in test_add_osd)
- Fix another variant of flusher sync-waiting stall (reproduced in test_write)
- Fix other tests in tests/ (will add them to Gitea CI soon)
- Add patches for QEMU 6.2-8.0
- Fix QEMU driver compatibility with QEMU 8.0
- Build packages for RHEL 9 clones (based on AlmaLinux 9)
4 weeks ago
Vitaliy Filippov 38be9a49c0 Add AlmaLinux 9 build to documentation 1 month ago
Vitaliy Filippov 7d6bf84a3e Add scripts/meson-buildoptions.sh to QEMU patches 1 month ago
Vitaliy Filippov 41a40a4123 Add QEMU spec patch for Alma/Rocky/RH 9 1 month ago
Vitaliy Filippov b94587ef0e Fix some build warnings 1 month ago
Vitaliy Filippov 2a2f4f6738 Add Almalinux 9 build 1 month ago
Vitaliy Filippov c768a9015f Fix QEMU driver compatibility with QEMU 8.0 1 month ago
Vitaliy Filippov 0d9e10cf96 Add patches for QEMU 6.2-8.0 1 month ago
Vitaliy Filippov b74ccb613c Fix another variant of flusher sync-waiting stall 1 month ago
Vitaliy Filippov 5052174918 Fix test_write_no_same (too large image) 1 month ago
Vitaliy Filippov eec9cf5575 Fix test_snapshot.sh - qemu-img requires explicit backing_fmt 1 month ago
Vitaliy Filippov a04dab0840 Initialize messenger in cluster_client listings 1 month ago
Vitaliy Filippov 160863f707 Print op pointer values in slow log 1 month ago
Vitaliy Filippov 2f16c32eb4 Fix test_minsize_1 (left_on_dead) 1 month ago
Vitaliy Filippov 2877cd0adb Allow OP_SEC_READ to return errors (do not hang the connection) 1 month ago
Vitaliy Filippov 480509f5b9 Fix pg_data_size > 1 for replicas (harmless bug) 1 month ago
Vitaliy Filippov 46462da45e Preload own PG history updates to fix PG state loop possibly applying the old metadata version 1 month ago
Vitaliy Filippov 024c8658f6 Fix missing } in quick start documentation 2 months ago
Vitaliy Filippov 7e958afeda Release 0.8.7
This release includes a bunch of important bugfixes for erasure-coded setups
with disabled immediate_commit. After these fixes, "test_heal" OSD killing test
now passes fine with EC:

- Fix cluster write stalls with "Error while doing flush on OSD xx: -16 (Device or resource busy)"
  in OSD logs possible in EC setups with disabled immediate_commit by selectively
  syncing nonsynced objects on STABILIZE/ROLLBACK (https://github.com/vitalif/vitastor/issues/51)
- Fix other EC + disabled immediate_commit problems:
  - Fix "opcode=5 retval=-2" errors happening on SYNC retries
  - Fix non-working "pagination" during PG dirty object flushing
  - Fix write operations not continued correctly after dirty object flushing
- Fix incorrect parity read-modify-write calculation when writing into a lost chunk
- Fix OSDs losing left_on_dead PG state of non-clean PGs and thus not removing junk data in the cluster
- Fix a small memory leak caused by bad indexing of EC recovery matrices
- Fix a rare use-after-free in cluster_client caused by a reenterability issue
- Fix vitastor-cli create command syntax in the CSI driver
- Allow to start OSDs without local store for tests
- Fix memory allocation error in disk_tool_meta for non-standard metadata block sizes
- Fix delete operations received before loading pool metadata crashing OSDs with "null pointer exception"
- Improve "theoretical performance" Russian documentation

New features:

- Implement online configuration update for some parameters. Documentation is coming soon :)
2 months ago
Vitaliy Filippov 2f5e769a29 Fix a small memory leak caused by bad indexing of EC recovery matrices 2 months ago
Vitaliy Filippov 28d5e53c6c Add test_heal to run_tests 2 months ago
Vitaliy Filippov d9f55f11d8 More logs (log_level 10), append to log instead of overwriting on restart in tests 2 months ago
Vitaliy Filippov 3237014608 Fix incorrect parity read-modify-write calculation when writing into a lost chunk 2 months ago
Vitaliy Filippov baaf8f6f44 Fix write operations not continued correctly after flush 2 months ago
Vitaliy Filippov 1d83fdcd17 Add debug logs to osd_flush 2 months ago
Vitaliy Filippov 0ddd787c38 Fix non-working "pagination" during PG dirty object flushing 2 months ago
Vitaliy Filippov 6eff3a60a5 Do not lose left_on_dead PG state of non-clean PGs 2 months ago
Vitaliy Filippov 888a6975ab Fix a rare use-after-free in cluster_client caused by a reenterability issue 2 months ago
Vitaliy Filippov cd1e890bd4 Fix "opcode=5 retval=-2" errors sometimes possible with EC 2 months ago
Vitaliy Filippov 0fbf4c6a08 Selectively sync nonsynced objects on STABILIZE/ROLLBACK (fix for github issue #51) 2 months ago
Vitaliy Filippov d06ed2b0e7 Implement online config update 2 months ago
Vitaliy Filippov 3bbc46543d Fix vitastor-cli create syntax 2 months ago
Vitaliy Filippov 2fb0c85618 Allow to start OSDs without local store (only for tests) 2 months ago
Vitaliy Filippov d81a6c04fc Update cmake min version so it does not complain about deprecation 2 months ago
Vitaliy Filippov 7b35801647 Fix possible bad realloc in disk_tool_meta for non-standard metadata block sizes 2 months ago
Vitaliy Filippov f3228d5c07 Fix typo (did not affect execution though) 2 months ago
Vitaliy Filippov 18366f5055 Fix read/write return type in rw_blocking 2 months ago
Vitaliy Filippov 851507c147 Add missing close() in test stubs 2 months ago
Vitaliy Filippov 9aaad28488 Fix "null pointer exception" for unhandled OSD_OP_DELETEs (when pool is not loaded yet) 3 months ago
Vitaliy Filippov dd57d086fe Add a missing part of the "theoretical performance" to the Russian version 3 months ago
Vitaliy Filippov 8810eae8fb Release 0.8.6
Important fixes:

- Fix possibly incorrect EC parity chunk updates with EC n+k, k > 1 and when
  the first parity chunk is missing

Minor fixes and improvements:

- Fix incorrect EC free space statistics in vitastor-cli df output
- Speedup vitastor-cli startup in clusters with RDMA
- Remove unused PG "peered" state (previously used to update PG epoch)
- Use sfdisk with just --json in vitastor-disk (--dump --json isn't needed)
- Allow trailing comma in sfdisk output (fixes sfdisk 2.36 compatibility)
- Slightly improve RDMA send/receive code
- Reduce RDMA memory consumption by default (rdma_max_recv/send = 16/8)
- Use vitastor-cli instead of direct etcd interaction in the CSI driver
3 months ago
Vitaliy Filippov c1365f46c9 Use vitastor-cli instead of direct etcd interaction in the CSI driver 3 months ago
Vitaliy Filippov 14d6acbcba Set default rdma_max_recv/send to 16/8, fix documentation 3 months ago
Vitaliy Filippov 1e307069bc Fix missing parity chunk calculation for EC n+k, k > 1 and first parity chunk missing 3 months ago
Vitaliy Filippov c3e80abad7 Allow to send more than 1 operation at a time 3 months ago
Vitaliy Filippov 138ffe4032 Reuse incoming RDMA buffers 3 months ago
Vitaliy Filippov 8139a34e97 Fix json11: allow trailing comma 3 months ago
Vitaliy Filippov 4ab630b44d Use just sfdisk --json, --dump is not needed 3 months ago
Vitaliy Filippov 2c8241b7db Remove PG "peered" state 3 months ago
Vitaliy Filippov 36a7dd3671 Move tests to "make test" 3 months ago
Vitaliy Filippov 936122bbcf Initialize msgr lazily in client to speedup vitastor-cli with RDMA enabled 3 months ago
Vitaliy Filippov 1a1ba0d1e7 Add set_immediate to ringloop and use it for bs/osd ops to prevent reenterability issues 4 months ago
Vitaliy Filippov 3d09c9cec7 Remove unused wait_sqe() from ringloop 4 months ago
Vitaliy Filippov 3d08a1ad6c Fix cluster_client test after last reenterability fixes 4 months ago
Vitaliy Filippov 499881d81c Fix typo 4 months ago
Vitaliy Filippov aba93b951b Fix incorrect EC free space statistics in vitastor-cli df output 4 months ago
Vitaliy Filippov d125fb1f30 Release 0.8.5
- Fix a possible "double free" bug in the client library happening on OSD restart
- Fix a possible write hang on PG history update when only epoch is changed
- Fix incorrect systemd target "local.target" in mon/make-etcd
- Allow "content" option in PVE storage plugin to allow to enable containers
- Build client library without tcmalloc which fixes "attempt to free invalid pointer"
  errors when, for example, trying to run QEMU with both Vitastor and Ceph RBD disks
4 months ago
Vitaliy Filippov 9d3fd72298 Require liburing < 2 in rpm specs 4 months ago
Vitaliy Filippov 8b552a01f9 Do not retry successful operation parts in client (could lead to "double free" bugs) 4 months ago
Vitaliy Filippov 0385b2f9e8 Fix write hangs on PG epoch update - always set pg.history_changed to true 4 months ago
Vitaliy Filippov 749c837045 Replace non-existing local.target with multi-user.target 4 months ago
Vitaliy Filippov 98001d845b Remove version from vitastor-release.rpm links 4 months ago
Vitaliy Filippov c96bcae74b Allow "content" option in PVE storage plugin to allow to enable containers 4 months ago
Vitaliy Filippov 9f4e34a8cc Build client library without tcmalloc
Fixes "[src/tcmalloc.cc:332] Attempt to free invalid pointer ..." when trying
to run QEMU with both Vitastor and Ceph RBD disks and other possible allocator
collisions.
4 months ago
Vitaliy Filippov 81fc8bb94c Release 0.8.4
New features:
- Implement QCOW2 image/snapshot export via qemu-img (bdrv_co_block_status in the driver)
- Remove OSDs from PG history during `vitastor-cli rm-osd` to prevent `left_on_dead` PG states after deletion
- Add a new recovery_pg_switch setting to mix all PGs during recovery, to almost
  fully reduce the probability of ENOSPC during rebalance
- Introduce partial ENOSPC ("OSD is full") handling - now ENOSPC doesn't turn
  into cascades of crashes
- Add migration support to Proxmox VE Vitastor driver
- Track last_clean_pgs on a per-pool basis thus reducing data movement in a cluster
  with pools remaining unclean/degraded for a long time

Bug fixes:
- Fix a bug where monitor could generate degraded PGs if one of the hosts had no OSDs
- Fix a bug where monitor could skip PG redistribution with a lot of OSDs in cluster
- Report PG history synchronously on the first write, which improves PG consistency
  and availability at the same time, because history now gets reported correctly
  and doesn't get reported without the need for it
- Fix possible write and recovery stalls which could happen in a cluster with both EC and replicated pools
- Make OSD and monitors sanitize & deduplicate PG history items in etcd
- Fix non-working OSD peer config safety check
- Fix a rare journal flush stall where flushing wasn't activated with full journal, but with empty flush queue
- Fix builds without ISA-L (jerasure-only) crashing with EC N+K, K>=2 due to the lack of 16-byte buffer alignment
- Fix a possible crash for EC N+K, K>=2 when calculating a parity chunk with previous parity chunk missing
- Fix a bug where vitastor-disk purge with suppressed warnings didn't work
4 months ago
Vitaliy Filippov bc465c16de Fix arithmetic on void* for clang 4 months ago
Vitaliy Filippov 8763e9211c Fix qemu driver compilation warning/error 4 months ago
Vitaliy Filippov 9e1a80bd17 Replace apt-key with trusted.gpg.d 4 months ago
Vitaliy Filippov 3e280f2f08 Mark vitastor as shared storage in PVE driver 5 months ago
Vitaliy Filippov fe87b4076b Fix backwards compatibility in cluster_client 5 months ago
Vitaliy Filippov a38957c1a7 Skip empty hosts in lp-optimizer 5 months ago
Vitaliy Filippov 137309cf29 Implement bdrv_co_block_status for snapshot export support 5 months ago

@ -0,0 +1,36 @@
FROM node:16-bullseye
WORKDIR /root
ADD ./docker/vitastor.gpg /etc/apt/trusted.gpg.d
RUN echo 'deb http://deb.debian.org/debian bullseye-backports main' >> /etc/apt/sources.list; \
echo 'deb http://vitastor.io/debian bullseye main' >> /etc/apt/sources.list; \
echo >> /etc/apt/preferences; \
echo 'Package: *' >> /etc/apt/preferences; \
echo 'Pin: release a=bullseye-backports' >> /etc/apt/preferences; \
echo 'Pin-Priority: 500' >> /etc/apt/preferences; \
echo >> /etc/apt/preferences; \
echo 'Package: *' >> /etc/apt/preferences; \
echo 'Pin: origin "vitastor.io"' >> /etc/apt/preferences; \
echo 'Pin-Priority: 1000' >> /etc/apt/preferences; \
grep '^deb ' /etc/apt/sources.list | perl -pe 's/^deb/deb-src/' >> /etc/apt/sources.list; \
echo 'APT::Install-Recommends false;' >> /etc/apt/apt.conf; \
echo 'APT::Install-Suggests false;' >> /etc/apt/apt.conf
RUN apt-get update
RUN apt-get -y install etcd qemu-system-x86 qemu-block-extra qemu-utils fio libasan5 \
liburing1 liburing-dev libgoogle-perftools-dev devscripts libjerasure-dev cmake libibverbs-dev libisal-dev
RUN apt-get -y build-dep fio qemu=`dpkg -s qemu-system-x86|grep ^Version:|awk '{print $2}'`
RUN apt-get -y install jq lp-solve sudo
RUN apt-get --download-only source fio qemu=`dpkg -s qemu-system-x86|grep ^Version:|awk '{print $2}'`
RUN set -ex; \
mkdir qemu-build; \
cd qemu-build; \
dpkg-source -x /root/qemu*.dsc; \
cd qemu*/; \
debian/rules configure-qemu || debian/rules b/configure-stamp; \
cd b/qemu; \
make -j8 config-poison.h || true; \
make -j8 qapi/qapi-builtin-types.h

@ -0,0 +1,16 @@
FROM git.yourcmc.ru/vitalif/vitastor/buildenv
ADD . /root/vitastor
RUN set -e -x; \
mkdir -p /root/fio-build/; \
cd /root/fio-build/; \
dpkg-source -x /root/fio*.dsc; \
cd /root/vitastor; \
ln -s /root/fio-build/fio-*/ ./fio; \
ln -s /root/qemu-build/qemu-*/ ./qemu; \
ls /usr/include/linux/raw.h || cp ./debian/raw.h /usr/include/linux/raw.h; \
mkdir build; \
cd build; \
cmake .. -DWITH_ASAN=yes -DWITH_QEMU=yes; \
make -j16

@ -0,0 +1,660 @@
name: Test
on:
push:
branches:
- '*'
paths:
- '.gitea/**'
- 'src/**'
- 'mon/**'
- 'json11'
- 'cpp-btree'
- 'tests/**'
env:
BUILDENV_IMAGE: git.yourcmc.ru/vitalif/vitastor/buildenv
TEST_IMAGE: git.yourcmc.ru/vitalif/vitastor/test
OSD_ARGS: '--etcd_quick_timeout 2000'
concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true
jobs:
buildenv:
runs-on: ubuntu-latest
container: git.yourcmc.ru/vitalif/gitea-ci-dind
steps:
- uses: actions/checkout@v3
- name: Build and push
run: |
set -ex
if ! docker manifest inspect $BUILDENV_IMAGE >/dev/null; then
docker build -t $BUILDENV_IMAGE -f .gitea/workflows/buildenv.Dockerfile .
docker login git.yourcmc.ru -u vitalif -p "${{secrets.TOKEN}}"
docker push $BUILDENV_IMAGE
fi
build:
runs-on: ubuntu-latest
needs: buildenv
container: git.yourcmc.ru/vitalif/gitea-ci-dind
steps:
- uses: actions/checkout@v3
with:
submodules: true
- name: Build and push
run: |
set -ex
if ! docker manifest inspect $TEST_IMAGE:$GITHUB_SHA >/dev/null; then
docker build -t $TEST_IMAGE:$GITHUB_SHA -f .gitea/workflows/test.Dockerfile .
docker login git.yourcmc.ru -u vitalif -p "${{secrets.TOKEN}}"
docker push $TEST_IMAGE:$GITHUB_SHA
fi
make_test:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
# leak sanitizer sometimes crashes
- run: cd /root/vitastor/build && ASAN_OPTIONS=detect_leaks=0 make -j16 test
test_add_osd:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: /root/vitastor/tests/test_add_osd.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_cas:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: /root/vitastor/tests/test_cas.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_change_pg_count:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: /root/vitastor/tests/test_change_pg_count.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_change_pg_count_ec:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: SCHEME=ec /root/vitastor/tests/test_change_pg_count.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_change_pg_size:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: /root/vitastor/tests/test_change_pg_size.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_create_nomaxid:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: /root/vitastor/tests/test_create_nomaxid.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_etcd_fail:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 10
run: /root/vitastor/tests/test_etcd_fail.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_failure_domain:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: /root/vitastor/tests/test_failure_domain.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_interrupted_rebalance:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 10
run: /root/vitastor/tests/test_interrupted_rebalance.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_interrupted_rebalance_imm:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 10
run: IMMEDIATE_COMMIT=1 /root/vitastor/tests/test_interrupted_rebalance.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_interrupted_rebalance_ec:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 10
run: SCHEME=ec /root/vitastor/tests/test_interrupted_rebalance.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_interrupted_rebalance_ec_imm:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 10
run: SCHEME=ec IMMEDIATE_COMMIT=1 /root/vitastor/tests/test_interrupted_rebalance.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_minsize_1:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: /root/vitastor/tests/test_minsize_1.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_move_reappear:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: /root/vitastor/tests/test_move_reappear.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_rebalance_verify:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: /root/vitastor/tests/test_rebalance_verify.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_rebalance_verify_imm:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: IMMEDIATE_COMMIT=1 /root/vitastor/tests/test_rebalance_verify.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_rebalance_verify_ec:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: SCHEME=ec /root/vitastor/tests/test_rebalance_verify.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_rebalance_verify_ec_imm:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: SCHEME=ec IMMEDIATE_COMMIT=1 /root/vitastor/tests/test_rebalance_verify.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_rm:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: /root/vitastor/tests/test_rm.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_snapshot:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: /root/vitastor/tests/test_snapshot.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_snapshot_ec:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: SCHEME=ec /root/vitastor/tests/test_snapshot.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_splitbrain:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: /root/vitastor/tests/test_splitbrain.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_write:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: /root/vitastor/tests/test_write.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_write_xor:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: SCHEME=xor /root/vitastor/tests/test_write.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_write_no_same:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: /root/vitastor/tests/test_write_no_same.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_heal_pg_size_2:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 10
run: PG_SIZE=2 /root/vitastor/tests/test_heal.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_heal_ec:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 10
run: SCHEME=ec /root/vitastor/tests/test_heal.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_scrub:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: /root/vitastor/tests/test_scrub.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_scrub_zero_osd_2:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: ZERO_OSD=2 /root/vitastor/tests/test_scrub.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_scrub_xor:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: SCHEME=xor /root/vitastor/tests/test_scrub.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_scrub_pg_size_3:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: PG_SIZE=3 /root/vitastor/tests/test_scrub.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: PG_SIZE=6 PG_MINSIZE=4 OSD_COUNT=6 SCHEME=ec /root/vitastor/tests/test_scrub.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done
test_scrub_ec:
runs-on: ubuntu-latest
needs: build
container: ${{env.TEST_IMAGE}}:${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: 3
run: SCHEME=ec /root/vitastor/tests/test_scrub.sh
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- $i --------"
cat $i
echo ""
done

@ -0,0 +1,68 @@
#!/usr/bin/perl
use strict;
for my $line (<>)
{
if ($line =~ /\.\/(test_[^\.]+)/s)
{
chomp $line;
my $test_name = $1;
my $timeout = 3;
if ($test_name eq 'test_etcd_fail' || $test_name eq 'test_heal' || $test_name eq 'test_interrupted_rebalance')
{
$timeout = 10;
}
while ($line =~ /([^\s=]+)=(\S+)/gs)
{
if ($1 eq 'SCHEME' && $2 eq 'ec')
{
$test_name .= '_ec';
}
elsif ($1 eq 'SCHEME' && $2 eq 'xor')
{
$test_name .= '_xor';
}
elsif ($1 eq 'IMMEDIATE_COMMIT')
{
$test_name .= '_imm';
}
else
{
$test_name .= '_'.lc($1).'_'.$2;
}
}
$line =~ s!\./test_!/root/vitastor/tests/test_!;
# Gitea CI doesn't support artifacts yet, lol
#- name: Upload results
# uses: actions/upload-artifact\@v3
# if: always()
# with:
# name: ${test_name}_result
# path: |
# /root/vitastor/testdata
# !/root/vitastor/testdata/*.bin
# retention-days: 5
print <<"EOF"
$test_name:
runs-on: ubuntu-latest
needs: build
container: \${{env.TEST_IMAGE}}:\${{github.sha}}
steps:
- name: Run test
id: test
timeout-minutes: $timeout
run: $line
- name: Print logs
if: always() && steps.test.outcome == 'failure'
run: |
for i in /root/vitastor/testdata/*.log /root/vitastor/testdata/*.txt; do
echo "-------- \$i --------"
cat \$i
echo ""
done
EOF
;
}
}

@ -1,7 +1,7 @@
cmake_minimum_required(V