Vitaliy Filippov
97efb9e299
Do not crash on PG re-peering events when operations are in progress
2021-04-07 11:06:31 +03:00
Vitaliy Filippov
75a6a556b5
Shuffle PGs for better data device utilisation
2021-04-07 01:30:38 +03:00
Vitaliy Filippov
8f8b90be7a
Add min_flusher_count configuration
2021-04-03 00:53:28 +03:00
Vitaliy Filippov
df99e232ee
Deduplicate osd_sets in pg history + raise request size limit for etcd
2021-04-03 00:53:28 +03:00
Vitaliy Filippov
3a40fa4127
Fix monitor errors in case of OSD removal
2021-03-27 01:15:18 +03:00
Vitaliy Filippov
435045751d
Delete objects only after a SYNC during rebalance in the non-immediate_commit mode
...
Previously OSDs could commit deletes before writes during recovery or rebalance
in the "lazy fsync" (immediate_commit=off) mode which could result in lost objects
2021-03-16 12:48:26 +03:00
Vitaliy Filippov
9f59381bea
Re-distribute PG primaries over OSDs that come up after a short downtime
2021-03-16 12:48:26 +03:00
Vitaliy Filippov
87dbd8fa57
Use empty hash as the default value for some etcd keys in the monitor
2021-03-12 12:40:15 +03:00
Vitaliy Filippov
b44f49aab2
Ignore zero OSDs in history osd_sets
2021-03-12 12:40:15 +03:00
Vitaliy Filippov
af5155fcd9
Implement "no_recovery" and "no_rebalance" flags
2021-03-11 00:36:31 +03:00
Vitaliy Filippov
0d2efbecc9
Preserve previous PG history when changing PG distribution
...
Fixes incorrect PG history in case when a new rebalance is started
before the finish of the previous one which could make primary OSDs unable
to locate some objects on some secondaries.
2021-03-11 00:16:10 +03:00
Vitaliy Filippov
e62e8b6bae
Use real pg configuration instead of the "last clean" one for generating PG history
...
Basically fixes the bug introduced in 0.5.7 where an rebalance interrupted
by the monitor could result in forgetting objects moved to the new place
2021-03-10 02:01:44 +03:00
Vitaliy Filippov
7006875a24
Make monitor stick to one etcd until the restart
2021-03-09 02:15:38 +03:00
Vitaliy Filippov
836635c518
Use osd_out_time = 10 minutes by default
2021-03-09 02:15:38 +03:00
Vitaliy Filippov
2a5036669d
Fix PG count change procedure
...
In previous versions PG histories were calculated incorrectly during
PG count change which led to objects being lost on OSDs not in PG's osd set.
2021-03-08 23:15:58 +03:00
Vitaliy Filippov
07912fd670
Use history/last_clean_pgs to avoid extra data move when observing a series of changes in the cluster
2021-03-08 17:04:10 +03:00
Vitaliy Filippov
24e7075f08
Fix monitor's statistics aggregation
2021-02-28 19:51:16 +03:00
Vitaliy Filippov
e899ed2c25
Make OSDs with 256 flushers (as they are now dynamic)
2021-02-26 01:59:18 +03:00
Vitaliy Filippov
e16b87ecc8
Rename random_combinations() parameter from "unordered" to "ordered" as it's more correct
2021-02-25 23:59:34 +03:00
Vitaliy Filippov
fcb4aa0a11
Fix Monitor being unable to change PG sizes
2021-02-25 23:59:34 +03:00
Vitaliy Filippov
7f15e0c084
Add a simple test for the PG optimizer
2021-02-25 23:59:33 +03:00
Vitaliy Filippov
08d4bef419
Fix PG optimizer removing PGs without adding new ones
...
This happened when the distribution was already valid for the current OSD tree,
but didn't use all OSDs. For example, OSDs 1 2 3 and all PGs equal to [ 1, 2 ]
remained unchanged.
2021-02-25 23:59:33 +03:00
Vitaliy Filippov
2d73b19a6c
Fix online PG count change bugs
2021-02-25 23:59:33 +03:00
Vitaliy Filippov
00e98f64f3
A little better systemd unit generator
2021-02-25 23:59:33 +03:00
Vitaliy Filippov
4a2dcf7b6b
Update the license to VNPL 1.1
...
VNPL 1.1 is slightly reworded to make it clear that proprietary software
interacting with Vitastor and providing some kind of service to end users isn't
a "Proxy Program" if it's not specially designed to be used with Vitastor.
For example, Windows OS running in a virtual machine stored in a Vitastor
cluster clearly isn't.
2021-02-25 23:55:33 +03:00
Vitaliy Filippov
8d48cc56b0
Generate randomly permutated OSD combinations when optimizing for compressed chunks
2021-02-25 23:55:33 +03:00
Vitaliy Filippov
9f58f01425
Mirror afr.js from /vitalif/ceph-afr-calc
2021-02-25 23:55:33 +03:00
Vitaliy Filippov
4cb79a3bf8
Allow to calculate simple-offsets for files
2021-02-01 19:43:10 +03:00
Vitaliy Filippov
915d04c446
Allow empty global configuration, report OSD statistics faster
2021-02-01 19:31:10 +03:00
Vitaliy Filippov
bcc8e697f9
Delete PGs when deleting pools
...
(All OSD crash with "Online PG count change not allowed" if you try to delete an active pool though)
2020-12-04 11:47:27 +03:00
Vitaliy Filippov
a4c46ba745
Add jerasure EC support (reed_sol_van, others are slower) (not tested yet)
2020-12-04 11:47:27 +03:00
Vitaliy Filippov
959089b919
Enable progress_notify=true for etcd watches
2020-11-17 16:29:42 +03:00
Vitaliy Filippov
e41bee72a5
Lower node.js requirement to 10.x
2020-11-08 01:54:12 +03:00
Vitaliy Filippov
64d471cf53
Add simple Debian packaging
2020-11-06 18:40:42 +03:00
Vitaliy Filippov
1e8f0328e0
Cancel outbound operations after re-peering PGs
2020-10-22 22:54:38 +00:00
Vitaliy Filippov
d2b901aa09
Fix default auto-created failure domains
2020-10-20 21:07:40 +00:00
Vitaliy Filippov
7cb561f95a
Add etcd to the example service generator
2020-10-20 01:50:56 +03:00
Vitaliy Filippov
9350656af6
Fix osd tags
2020-10-16 23:28:48 +00:00
Vitaliy Filippov
94efb54feb
Implement OSD tags (device classes), fix pool failure_domain configuration
2020-10-04 17:31:50 +03:00
Vitaliy Filippov
9f2a948712
Make pg_stripe_size a per-pool config
2020-10-01 18:51:49 +03:00
Vitaliy Filippov
ba74eece4a
More fixes to the failure model (why am I doing this?..)
2020-10-01 18:38:30 +03:00
Vitaliy Filippov
2fdd8a1b38
More correct failure model (I hope so)
2020-10-01 02:33:48 +03:00
Vitaliy Filippov
8e36f04482
One more experiment with cluster AFR%
2020-09-27 19:42:42 +03:00
Vitaliy Filippov
6852f299ae
Add functions to calculate AFR for a cluster
2020-09-24 23:15:26 +03:00
Vitaliy Filippov
0471b09b9c
Add license notices to all source code files
2020-09-17 23:07:06 +03:00
Vitaliy Filippov
28a0f08ce7
Add a very simple tool for calculating device offsets
2020-09-17 00:45:59 +03:00
Vitaliy Filippov
9b4e5b64ae
Move monitor to mon/
2020-09-16 02:15:26 +03:00
Vitaliy Filippov