Compare commits

base: vitalif:v1.4.0

Branches Tags

vitalif:test

vitalif:osd-cluster-network

vitalif:master

vitalif:qemu-fix-crash-idea

vitalif:eventloop-auto-init-trigger

vitalif:cow-meta

vitalif:hugo-docs

vitalif:node-mutex-and-postpone

vitalif:check-writeback

vitalif:msgr-iothreads-v2

vitalif:antietcd

vitalif:rel-1.4

vitalif:test-fix-ec-unknown-state-51

vitalif:kv-update

vitalif:kv-debug

vitalif:hotfix-1.2.0

vitalif:rdma-simple-nodp

vitalif:zerocopy-tcp-send

vitalif:csi-staging

vitalif:hotfix-1.1.0

vitalif:hotfix-1.0.0

vitalif:fsync-feedback

vitalif:cached-reads

vitalif:qemu-send-loop

vitalif:openssl

vitalif:test-double-alloc

vitalif:hier-failure-domains

vitalif:mon-self-restart

vitalif:epoch-deletions

vitalif:rdma-flow-control

vitalif:csi-use-vitastor-cli

vitalif:rdma-v2

vitalif:etcd-hide-base64

vitalif:lrc-matrix

vitalif:rm-left-on-dead

vitalif:nfs-proxy-old

vitalif:non-odp-rdma

vitalif:test-zctr

vitalif:separate-data-connections

vitalif:sec_osd_msgr

vitalif:test-assert

vitalif:nbd-vmsplice

vitalif:allow-etcd-address-option

vitalif:rdma-zerocopy

vitalif:rel-0.5

vitalif:openonload

vitalif:sync-io-test

vitalif:trace-sync

vitalif:trace-sqes

vitalif:test-submit-and-wait

vitalif:test-sq-poll

vitalif:blocking-uring-test

vitalif:v2.0.0

vitalif:v1.11.0

vitalif:v1.10.1

vitalif:v1.10.0

vitalif:v1.9.3

vitalif:v1.9.2

vitalif:v1.9.1

vitalif:v1.9.0

vitalif:v1.8.0

vitalif:v1.7.1

vitalif:v1.7.0

vitalif:v1.6.1

vitalif:v1.6.0

vitalif:v1.5.0

vitalif:v1.4.8

vitalif:v1.4.7

vitalif:v1.4.6

vitalif:v1.4.5

vitalif:v1.4.4

vitalif:v1.4.3

vitalif:v1.4.2

vitalif:v1.4.1

vitalif:v1.4.0

vitalif:v1.3.1

vitalif:v1.3.0

vitalif:v1.2.0

vitalif:v1.1.0

vitalif:v1.0.0

vitalif:v0.9.6

vitalif:v0.9.5

vitalif:v0.9.4

vitalif:v0.9.3

vitalif:v0.9.2

vitalif:v0.9.1

vitalif:v0.9.0

vitalif:v0.8.9

vitalif:v0.8.8

vitalif:v0.8.7

vitalif:v0.8.6

vitalif:v0.8.5

vitalif:v0.8.4

vitalif:v0.8.3

vitalif:v0.8.2

vitalif:v0.8.1

vitalif:v0.8.0

vitalif:v0.7.1

vitalif:v0.7.0

vitalif:v0.6.17

vitalif:v0.6.16

vitalif:v0.6.15

vitalif:v0.6.14

vitalif:v0.6.13

vitalif:v0.6.12

vitalif:v0.6.11

vitalif:v0.6.10

vitalif:v0.6.9

vitalif:v0.6.8

vitalif:v0.6.7

vitalif:v0.6.6

vitalif:v0.6.5

vitalif:v0.6.4

vitalif:v0.6.3

vitalif:v0.6.2

vitalif:v0.6.1

vitalif:v0.6.0

vitalif:v0.5.13

vitalif:v0.5.12

vitalif:v0.5.11

vitalif:v0.5.10

vitalif:v0.5.9

vitalif:v0.5.7

vitalif:v0.5.6

vitalif:v0.5.5

vitalif:v0.5.4

vitalif:v0.5.3

vitalif:v0.5.2

vitalif:v0.5.1

vitalif:v0.5.0

vitalif:v0.4.0

vitalif:v0.5.8

..

compare: vitalif:master

Branches Tags

vitalif:test

vitalif:osd-cluster-network

vitalif:master

vitalif:qemu-fix-crash-idea

vitalif:eventloop-auto-init-trigger

vitalif:cow-meta

vitalif:hugo-docs

vitalif:node-mutex-and-postpone

vitalif:check-writeback

vitalif:msgr-iothreads-v2

vitalif:antietcd

vitalif:rel-1.4

vitalif:test-fix-ec-unknown-state-51

vitalif:kv-update

vitalif:kv-debug

vitalif:hotfix-1.2.0

vitalif:rdma-simple-nodp

vitalif:zerocopy-tcp-send

vitalif:csi-staging

vitalif:hotfix-1.1.0

vitalif:hotfix-1.0.0

vitalif:fsync-feedback

vitalif:cached-reads

vitalif:qemu-send-loop

vitalif:openssl

vitalif:test-double-alloc

vitalif:hier-failure-domains

vitalif:mon-self-restart

vitalif:epoch-deletions

vitalif:rdma-flow-control

vitalif:csi-use-vitastor-cli

vitalif:rdma-v2

vitalif:etcd-hide-base64

vitalif:lrc-matrix

vitalif:rm-left-on-dead

vitalif:nfs-proxy-old

vitalif:non-odp-rdma

vitalif:test-zctr

vitalif:separate-data-connections

vitalif:sec_osd_msgr

vitalif:test-assert

vitalif:nbd-vmsplice

vitalif:allow-etcd-address-option

vitalif:rdma-zerocopy

vitalif:rel-0.5

vitalif:openonload

vitalif:sync-io-test

vitalif:trace-sync

vitalif:trace-sqes

vitalif:test-submit-and-wait

vitalif:test-sq-poll

vitalif:blocking-uring-test

vitalif:v2.0.0

vitalif:v1.11.0

vitalif:v1.10.1

vitalif:v1.10.0

vitalif:v1.9.3

vitalif:v1.9.2

vitalif:v1.9.1

vitalif:v1.9.0

vitalif:v1.8.0

vitalif:v1.7.1

vitalif:v1.7.0

vitalif:v1.6.1

vitalif:v1.6.0

vitalif:v1.5.0

vitalif:v1.4.8

vitalif:v1.4.7

vitalif:v1.4.6

vitalif:v1.4.5

vitalif:v1.4.4

vitalif:v1.4.3

vitalif:v1.4.2

vitalif:v1.4.1

vitalif:v1.4.0

vitalif:v1.3.1

vitalif:v1.3.0

vitalif:v1.2.0

vitalif:v1.1.0

vitalif:v1.0.0

vitalif:v0.9.6

vitalif:v0.9.5

vitalif:v0.9.4

vitalif:v0.9.3

vitalif:v0.9.2

vitalif:v0.9.1

vitalif:v0.9.0

vitalif:v0.8.9

vitalif:v0.8.8

vitalif:v0.8.7

vitalif:v0.8.6

vitalif:v0.8.5

vitalif:v0.8.4

vitalif:v0.8.3

vitalif:v0.8.2

vitalif:v0.8.1

vitalif:v0.8.0

vitalif:v0.7.1

vitalif:v0.7.0

vitalif:v0.6.17

vitalif:v0.6.16

vitalif:v0.6.15

vitalif:v0.6.14

vitalif:v0.6.13

vitalif:v0.6.12

vitalif:v0.6.11

vitalif:v0.6.10

vitalif:v0.6.9

vitalif:v0.6.8

vitalif:v0.6.7

vitalif:v0.6.6

vitalif:v0.6.5

vitalif:v0.6.4

vitalif:v0.6.3

vitalif:v0.6.2

vitalif:v0.6.1

vitalif:v0.6.0

vitalif:v0.5.13

vitalif:v0.5.12

vitalif:v0.5.11

vitalif:v0.5.10

vitalif:v0.5.9

vitalif:v0.5.7

vitalif:v0.5.6

vitalif:v0.5.5

vitalif:v0.5.4

vitalif:v0.5.3

vitalif:v0.5.2

vitalif:v0.5.1

vitalif:v0.5.0

vitalif:v0.4.0

vitalif:v0.5.8

530 Commits

v1.4.0 ... master

Author	SHA1	Message	Date
Vitaliy Filippov	ccc32b9e68	Use TCP on RDMA connection failure	2025-03-23 12:04:23 +03:00
Vitaliy Filippov	ebaf3fee79	Add an assertion to prevent sending message to TCP channel when switched to RDMA	2025-03-23 12:04:09 +03:00
Vitaliy Filippov	196d28e987	Fix typo	2025-03-23 12:00:20 +03:00
Vitaliy Filippov	8f243b2328	Fix qemu buster build and bullseye version	2025-03-23 02:46:52 +03:00
Vitaliy Filippov	7a835fcd8f	Add allow_net_split parameter	2025-03-23 02:12:32 +03:00
Vitaliy Filippov	8b0389b4e8	Log RDMA ibv_modify_qp() errors	2025-03-22 15:58:13 +03:00
Vitaliy Filippov	f544c350ba	%l* -> %j*	2025-03-22 15:32:07 +03:00
Vitaliy Filippov	4eafb55b5c	Add a patch for QEMU 9.2, fix debian bookworm QEMU build	2025-03-22 15:30:52 +03:00
Vitaliy Filippov	5030396f71	Clear QEMU eventfd handler on vitastor block driver destruction	2025-03-21 20:47:17 +03:00
Vitaliy Filippov	be22c363ca	Do not skip client_retry_interval on reconnecting OSDs to prevent OSD spam	2025-03-20 00:12:38 +03:00
Vitaliy Filippov	0f80c87b43	Add a minimum interval for etcd_state_client to reload state ... (To prevent excessive load on etcd during outages)	2025-03-19 02:36:09 +03:00
Vitaliy Filippov	e0953fd502	Wait for all "up" OSDs to be connected before starting PG	2025-03-19 02:36:09 +03:00
Vitaliy Filippov	6e0ae47938	Add Proxmox QEMU 9.2 patch	2025-03-19 02:36:02 +03:00
Vitaliy Filippov	b8f19e85ad	Fix pg state formatting in ls-pgs	2025-03-17 01:37:58 +03:00
Vitaliy Filippov	b7636e595f	Update version in docker docs	2025-03-16 16:53:57 +03:00
Vitaliy Filippov	48c026bfa0	Release 2.0.0 ... No breaking features, it's 2.0.0 just because it includes S3 and because there are already too many 1.x releases :). New features: - S3 is finally available: https://vitastor.io/docs/installation/s3.html - node.js addon is now packaged as a Debian package - Support listing PGs by OSDs in `vitastor-cli ls-pgs` - Implement offline TRIM support: [vitastor-disk trim](https://vitastor.io/docs/usage/disk.html#trim), [discard_on_start](https://vitastor.io/docs/config/osd.html#discard_on_start) - Change used_for_fs pool option to used_for_app Bug fixes: - Fix several bugs in the node.js addon (a memory leak, an incorrectly triggered event loop) - Fix a client crash (vitastor-cli rm) during deletion when writeback is enabled - Fix PG object count statistics on deletion of non-existing objects - Fix vitastor-nbd crash when mapping by ID instead of inode name - Fix a client memory leak with enabled immediate_commit and write-back cache - Add seccomp=unconfined for vitastor docker OSDs to not break io_uring - Add udev and systemd to vitastor docker image - Fix upgrading from pre-0.7.1 (very old) systemd units O_o - Fix total object count calculation in rm_data	2025-03-16 14:34:31 +03:00
Vitaliy Filippov	a73b2a26b6	Fix blockstore initialization after moving clean_dyn_size calc to calc_lengths	2025-03-16 13:44:02 +03:00
Vitaliy Filippov	f3192b610d	Fix vitastor-disk in Docker installations	2025-03-16 13:44:01 +03:00
Vitaliy Filippov	a950889976	Add missing docs for discard_on_start	2025-03-16 12:29:22 +03:00
Vitaliy Filippov	ef5194d93c	Add S3 installation docs	2025-03-16 01:17:09 +03:00
Vitaliy Filippov	f904576ab1	Fix total calculation in rm_data	2025-03-15 17:01:10 +03:00
Vitaliy Filippov	4f9b1f2f62	Support listing PGs by OSDs	2025-03-15 16:42:57 +03:00
Vitaliy Filippov	1d94afbd51	Implement offline TRIM support	2025-03-14 01:37:16 +03:00
Vitaliy Filippov	3634f005f1	Fix upgrading from pre-0.7.1 systemd units O_o	2025-03-14 01:37:16 +03:00
Vitaliy Filippov	263a3b5ad6	Rename allocator to allocator_t	2025-03-13 00:53:34 +03:00
Vitaliy Filippov	b760951aa7	Add seccomp=unconfined for vitastor docker OSDs to not break io_uring	2025-03-11 00:42:10 +03:00
Vitaliy Filippov	c8321b8ed1	Add udev and systemd to vitastor docker image	2025-03-11 00:40:39 +03:00
Vitaliy Filippov	21066a095b	Fix a memory leak with enabled immediate_commit and write-back cache ... Remove dirty buffers after writing when immediate_commit is on instead of saving them for repeating later	2025-03-11 00:40:18 +03:00
Vitaliy Filippov	a96900b696	Explicitly destroy Nan::Persistents, otherwise it leaks memory	2025-03-09 16:45:10 +03:00
Vitaliy Filippov	8a6e461322	Fix license (VNPL 1.1, not 2.0)	2025-03-08 17:17:23 +03:00
Vitaliy Filippov	0b6a0463a4	Save a reference to the buffer during write	2025-03-08 16:00:26 +03:00
Vitaliy Filippov	35d4047f46	Fix vitastor-nbd crash when mapping by ID instead of inode name	2025-03-08 15:52:57 +03:00
Vitaliy Filippov	819f1125ae	Support used_for_app instead of used_for_fs	2025-03-07 01:03:43 +03:00
Vitaliy Filippov	108df7329f	Fix PG object count statistics on deletion of non-existing objects	2025-03-04 00:40:56 +03:00
Vitaliy Filippov	d32edf6cdf	Fix deletion writeback	2025-03-04 00:40:35 +03:00
Vitaliy Filippov	dca436d7e6	Trigger event loop automatically in libvitastor_c	2025-03-03 00:57:09 +03:00
Vitaliy Filippov	8129a0b4e3	Loop once after registering eventfd to prevent skipping previous events	2025-03-03 00:57:00 +03:00
Vitaliy Filippov	704c87d512	Trigger initial epoll when adding an FD	2025-03-03 00:56:17 +03:00
Vitaliy Filippov	10216a5fb5	Build node.js addon as a Debian package	2025-03-02 18:04:56 +03:00
Vitaliy Filippov	3932eb7ff6	Trigger event loop once after each vitastor_c_* call	2025-03-02 01:23:41 +03:00
Vitaliy Filippov	69cbe7bbb2	Release 1.11.0 ... New features: - Support containerized Vitastor installations: http://vitastor.io/docs/installation/docker.html - Add new functions to the node.js binding: delete(), get_immediate_commit(), on_ready(), get_min_io_size(), get_max_atomic_write_size() - S3 (Zenko Cloudserver with Vitastor support) is coming shortly and will be released separately Bug fixes: - Use IP-derived etcd node names in make-etcd - Set short name of the OSD process to display in `top` - Fix snap-create without pool_id failing when there are multiple pools - Several bugs are fixed in the write-back cache, it should now be stable: - Fix incorrect snapshot reads from dirty write-back cache - Do not try to repeat pending writebacks on OSD reconnections - Fix client hangs with multiple SYNCs in the writeback queue - Fix client hangs do to incorrect calculation of the writeback queue size - Several improvements for NBD mapping/unmapping: - Add a workaround for race condition in the Linux kernel NBD driver leading to vitastor-nbd sometimes breaking a previously mapped device instead of setting up a new one - Check if the device is actually mapped in vitastor-nbd unmap - Fix device name/number validation in vitastor-nbd - Fix OSD crashes after starting with corrupted metadata - from now it will skip corrupted metadata entries and heal itself - Fix scrubbing of misplaced objects and object state recalculation after vitastor-cli fix - previously, an OSD restart could be required to fix object states - Make primary OSD distribution more stable by using murmur3 hash instead of the old pseudo-rng - Fix monitor sometimes racing with itself - do not touch /pool/stats from stats aggregation if PG recheck is active - Sort vitastor-cli ls output by name by default - Update antietcd to 1.1.2	2025-03-01 13:39:42 +03:00
Vitaliy Filippov	4950a1636c	Allow "infinite" startup for clients if explicitly requested	2025-03-01 13:39:42 +03:00
Vitaliy Filippov	2eb20dff28	Do not crash on io_uring initialization failure in node-vitastor	2025-03-01 13:29:48 +03:00
Vitaliy Filippov	59f0b0427c	Support containerized Vitastor installations	2025-02-27 20:06:15 +03:00
Vitaliy Filippov	124162ad38	Use IP-derived etcd node names in make-etcd	2025-02-26 11:54:37 +03:00
Vitaliy Filippov	391c92af1a	Set OSD process name	2025-02-26 11:54:37 +03:00
Vitaliy Filippov	c3d8fdd855	Fix snap-create without pool_id ID generation with multiple pools	2025-02-26 11:54:28 +03:00
Vitaliy Filippov	9ccf3af97b	Add qemu-block-extra and qemu-utils	2025-02-23 15:08:16 +03:00
Vitaliy Filippov	568a209f0d	Update docker image to debian bookworm	2025-02-23 13:27:32 +03:00
Vitaliy Filippov	b151013201	Fix snapshot reads from a dirty write-back cache	2025-02-23 02:31:19 +03:00
Vitaliy Filippov	4a763725fe	Add free() to bindiff.c	2025-02-22 16:52:19 +03:00
Vitaliy Filippov	b8d83cd7f4	No, it's not a good idea to destroy client in the child nbd process ... Should probably have been an obvious side effect :-) Child process gets open file descriptors to parent's epoll/timerfd, and it's totally OK to just close() all of them, but it's absolutely NOT OK to run destructors - they modify the kernel state of epoll/timerfd before destroying. So, basically, when we destroy the client in the child process, we break it in the parent too. This also means that cluster_client_t doesn't support fork(). :-)	2025-02-22 15:10:27 +03:00
Vitaliy Filippov	2e9ee2fe20	Do not try to repeat pending writebacks	2025-02-22 14:16:44 +03:00
Vitaliy Filippov	508ae852e4	Fix trap in test_rebalance_verify	2025-02-22 02:18:41 +03:00
Vitaliy Filippov	97ee400505	Add a workaround for race condition in the Linux kernel NBD driver ... Do all NBD configuration in the child process, after the last fork. Why? It's needed because there is a race condition in the Linux kernel nbd driver in nbd_add_socket() - it saves `current` task pointer as `nbd->task_setup` and then rechecks if the new `current` is the same. Problem is that if that process is already dead, `current` may be freed and then replaced by another process with the same pointer value. So the check passes and NBD allows a different process to set up a device which is already set up. Proper fix would have to be done in the kernel code, but the workaround is obviously to perform NBD setup from the process which will then actually call NBD_DO_IT. That process stays alive during the whole time of NBD device execution and the (nbd->task_setup != current) check always works correctly, and we don't accidentally break previous NBD devices while setting up a new device. Forking to check every device is of course rather slow, so we also do an additional check by calling list_mapped() before searching for a free NBD device.	2025-02-21 13:17:37 +03:00
Vitaliy Filippov	5ee4894fab	Check if mapped in vitastor-nbd unmap	2025-02-21 01:28:06 +03:00
Vitaliy Filippov	125dcafb11	Prevent OSD crashes when metadata is corrupted	2025-02-20 02:19:32 +03:00
Vitaliy Filippov	9f44cf71df	Fix device name/number validation in vitastor-nbd	2025-02-20 01:33:11 +03:00
Vitaliy Filippov	df3c63ca7f	Sort vitastor-cli ls by name by default	2025-02-20 01:32:49 +03:00
Vitaliy Filippov	be66edd09f	Prevent infinite loops on syncs in writeback_overflow	2025-02-19 01:44:12 +03:00
Vitaliy Filippov	ccbc0c5928	Add assert !writeback_bytes	2025-02-19 01:15:46 +03:00
Vitaliy Filippov	78ca4538bf	Fix qemu docker build for ubuntu	2025-02-18 23:44:16 +03:00
Vitaliy Filippov	86b5760ec1	Fix writeback incorrectly calculating queue size which was leading to client hangs	2025-02-18 23:42:55 +03:00
Vitaliy Filippov	27f3803d2f	Add vitastor_c_delete() and delete() to the node.js binding	2025-02-15 18:27:17 +03:00
Vitaliy Filippov	2ead06e126	Add ubuntu jammy to docs	2025-02-12 15:32:35 +03:00
Vitaliy Filippov	a5d5559f8e	Add get_immediate_commit() to the node.js binding	2025-02-06 01:35:48 +03:00
Vitaliy Filippov	e8e7ba8fde	Add FIXME for CAS in non-immediate_commit mode	2025-02-06 01:35:48 +03:00
Vitaliy Filippov	6fd831a299	Add on_ready(), get_min_io_size(), get_max_atomic_write_size() to the node.js binding	2025-02-06 01:35:48 +03:00
Vitaliy Filippov	069808dfce	Fix --config_path option in docs	2025-01-24 17:21:11 +03:00
Vitaliy Filippov	bcefa42bc0	Scrub all chunks, not just 1 chunk per position	2025-01-23 02:02:55 +03:00
Vitaliy Filippov	4636e02d43	Remove scheme, pg_size, pg_data_size from op_data	2025-01-23 01:20:31 +03:00
Vitaliy Filippov	e4c7d1c147	s/3/4/	2025-01-23 01:20:31 +03:00
Vitaliy Filippov	a4677f3e69	Mention P5530	2025-01-23 01:20:31 +03:00
Vitaliy Filippov	7cbf207d65	Use murmur3 to select primary OSD instead of old pseudo-rng	2025-01-18 12:28:54 +03:00
Vitaliy Filippov	7c9711af20	Do not touch /pool/stats from stats aggregation if PG recheck is active	2025-01-16 20:41:16 +03:00
Vitaliy Filippov	33ef701464	Update antietcd to 1.1.2	2025-01-04 02:13:36 +03:00
Vitaliy Filippov	61ededa230	Release 1.10.1 ... New features: - Add "deleted" image flag which is set when vitastor-cli rm starts to delete an image, but can't delete it fully due to inactive PGs or stopped OSDs - Support JSON output in vitastor-disk prepare and purge - Show backfillfull pools in vitastor-cli status - Make object listings consistent (used in vitastor-cli rm/rm-data/merge/etc). This means that there is now a guarantee that if a data block is present when you invoke rm, rm will attempt to delete it, even if rm is invoked when the PG switches state. Previously in such cases rm could skip and leave some objects behind as garbage, and merge probably could incorrectly move data between snapshots. - Make deletions (rm/rm-data) consistent. This means that rm/rm-data will either complete successfully and delete all requested image data or complete with an error if some objects could not be deleted or if there is a possibility that some data is left on stopped OSDs. Previously, when some PGs or OSDs were inactive at the moment of deletion, rm-data was behaving incorrectly: it wasn't retrying deletions failed due to dropped OSD connections, it could hang waiting for PGs to activate, and it could return with a successful error code while some garbage was still possibly left on some OSDs. Deletions are not fully atomic cluster-wide yet, which means that you still have to repeat the deletion request after you return stopped OSDs back, but now you always know for sure if you have to repeat it. Bug fixes: - Fix vitastor-cli rm --exact / --matching command not working - Finally fix "Unexpected status" in the Proxmox plugin - Fix vitastor-cli create-snap incorrectly linking multiple snapshots in a different pool - Fix incomplete image parent_id loop check in OSD - Fix reads from snapshots in a different pool not working if there are more than 2 snapshots - Fix append of VITASTOR_CONF to cmdline in the opennebula prebackup script - Fix OSDs crashing again when the cluster is full with EC (was meant to work since 1.6.0 but didn't) - Improve logging of subop failures	2025-01-03 16:22:09 +03:00
Vitaliy Filippov	d9d90d3183	Fix build for debian buster	2025-01-03 16:21:56 +03:00
Vitaliy Filippov	9dbcdbcec9	Return left_on_dead OSD list in DELETE replies and use it in rm-data	2025-01-03 15:57:09 +03:00
Vitaliy Filippov	a147f7e7dc	Copy & repeat deletions too	2025-01-03 00:21:52 +03:00
Vitaliy Filippov	0e6bf66734	Add bindiff for tests	2025-01-02 19:59:04 +03:00
Vitaliy Filippov	ab822d3050	Support consistent listings in client (rm-data, merge and etc)	2025-01-02 18:07:12 +03:00
Vitaliy Filippov	d5366a0767	Support listings from primary OSDs (for consistent deletions)	2025-01-02 11:07:24 +03:00
Vitaliy Filippov	40b8a8b0da	Add wait_up_timeout support to cluster_client and use it in vitastor-cli rm-data & merge	2025-01-01 17:57:58 +03:00
Vitaliy Filippov	5c5119aba4	Pass min_offset/max_offset to list_inode()	2025-01-01 15:40:12 +03:00
Vitaliy Filippov	4edda88903	Wait for OSDs to either connect or stop infinitely during listing, not for peer_connect_timeout	2025-01-01 15:29:42 +03:00
Vitaliy Filippov	80dda3ca94	Remove separate list_inode_next()	2025-01-01 14:19:18 +03:00
Vitaliy Filippov	c8decb32e8	Rename to client_wait_up_timeout	2025-01-01 11:26:57 +03:00
Vitaliy Filippov	4995592e61	Retry listings on broken OSD connections	2025-01-01 11:14:36 +03:00
Vitaliy Filippov	d9f9b0bca5	Start listings consistently with the current PG state, add wait_up_timeout ... This still doesn't make listings 100% consistent yet; for 100% consistent listings we have to receive listings only from the primary OSD, not from all peer OSDs, but this issue will be fixed separately.	2025-01-01 10:58:22 +03:00
Vitaliy Filippov	d0396267d0	Clear retry_timeout when the client is destroyed	2025-01-01 10:58:22 +03:00
Vitaliy Filippov	b46d5db115	Support JSON output in vitastor-disk prepare and purge	2024-12-29 15:19:44 +03:00
Vitaliy Filippov	ecd92655fe	Fix rm --exact / --matching not removing one uppermost image in each chain	2024-12-28 21:53:49 +03:00
Vitaliy Filippov	383712148b	Fix rm --exact / --matching not being invoked at all O_o	2024-12-28 21:47:00 +03:00
Vitaliy Filippov	42d40153ff	Do not intercept STDERR in Proxmox plugin (finally fixes "unexpected status"!)	2024-12-28 21:18:49 +03:00
Vitaliy Filippov	561b36a4c1	Use revision from txn response header, not from put subresponse	2024-12-28 21:01:15 +03:00
Vitaliy Filippov	685af019f5	Allow :: and 0.0.0.0 as local IPs in antietcd_adapter	2024-12-28 20:52:27 +03:00
Vitaliy Filippov	a31592d131	Print sizes in "Auto-selecting" as "4K", not "4 K"	2024-12-28 19:15:23 +03:00
Vitaliy Filippov	28b0a2597d	Add a test for multiple snapshots in a second pool	2024-12-28 18:57:30 +03:00
Vitaliy Filippov	de6b345473	Fix create-snap taking parent_pool from incorrect key parent_pool_id	2024-12-28 18:53:29 +03:00
Vitaliy Filippov	8bf52d6e96	Fix inode parent_id loop check	2024-12-28 18:40:17 +03:00
Vitaliy Filippov	5623dca02c	Fix vitastor client passing incorrect mod_revision for snapshotted images ... This was leading to reads only working for the image itself and for its latest snapshot	2024-12-28 16:01:35 +03:00
Vitaliy Filippov	abdc207297	Fix append of VITASTOR_CONF to cmdline in the opennebula prebackup script	2024-12-28 13:33:24 +03:00
Vitaliy Filippov	044e621b62	Add test_rm_degraded to CI	2024-12-27 18:31:58 +03:00
Vitaliy Filippov	ba9aabf187	Return listing errors from list_inode_start(), abort merging and fail deletion on unsuccessfull listings	2024-12-27 18:31:21 +03:00
Vitaliy Filippov	5c890e4a12	Fix rm-data hanging when some OSDs are inactive, add a test for it ... There's also another case which also needs to be fixed - we shouldn't retry deletions for indefinite time if an OSD is stopped during deletion	2024-12-27 16:29:33 +03:00
Vitaliy Filippov	0b0c2afbce	Implement "deleted" flag	2024-12-27 01:18:55 +03:00
Vitaliy Filippov	651c055bd9	Show backfillfull pools in vitastor-cli status	2024-12-26 12:17:47 +03:00
Vitaliy Filippov	42eebfc1bd	Fix OSDs still crashing when the cluster is full with EC ... ENOSPC handling was introduced in 1.6.0 but it was not complete; now it is P.S: See also client_retry_enospc (true by default)	2024-12-26 01:56:33 +03:00
Vitaliy Filippov	cef98052f5	Improve logging of subop failures	2024-12-26 01:54:40 +03:00
Vitaliy Filippov	7fbb04fdfa	Release 1.10.0 ... New features: - Implement basic VitastorFS support in [CSI](https://vitastor.io/docs/installation/kubernetes.html) - Implement [NFS RDMA](https://vitastor.io/docs/usage/nfs.html#rdma) support - Pause pool rebalance when monitor detects that it can lead to any OSD becoming full ([osd_backfillfull_ratio](https://vitastor.io/docs/config/monitor.html#osd_backfillfull_ratio)) - Auto-select correct [RDMA device and GID](https://vitastor.io/docs/config/network.html#rdma_device) based on osd_network and RoCEv2 priority - Report slow ops in OSD stats in etcd and show them in vitastor-cli status Bug fixes: - Fix possibly incorrect linked list deserialization in NFS - Fix possible crash in vitastor-nfs --block READDIR operation - Map netlink after forking to show correct PID in vitastor-nbd ls - Simplify and fix create-pool OSD count checks for the case of hosts split into sub-nodes - Make monitor print "Waiting to become master" just once, not every 5s - Take out_size from oimg if not specified in vitastor-cli dd - Do not report OSDs with empty statistics as "full" in status - Trigger double autosync when switching PG state to prevent leaving garbage in non-immediate_commit clusters - Fix a lack of connection timeout for etcd websockets in OSD leading to slower etcd failover (~70s instead of ~10s) - Fix a rare OSD crash during client disconnect - Fix PGs sometimes sticking until OSD restart in the "has_unclean" state with EC pools - Fix metadata partition zeroing in vitastor-disk prepare - Add patches for qemu 9.1 and pve-qemu 9.0 and 9.1 - Fix libvirt 8 patch	2024-12-19 15:49:19 +03:00
Vitaliy Filippov	63b85b6bfb	Fix clang warnings/errors	2024-12-19 15:30:31 +03:00
Vitaliy Filippov	2f5959e3fa	Add pve-qemu 9.1 patch	2024-12-19 14:05:12 +03:00
Vitaliy Filippov	a4a286ed95	Document NFS-RDMA	2024-12-19 14:05:12 +03:00
Vitaliy Filippov	b8009bad5e	Add librdmacm-dev to build dockerfile	2024-12-19 14:05:12 +03:00
Vitaliy Filippov	9be3d27dc9	Document VitastorFS-based CSI	2024-12-19 13:06:47 +03:00
Vitaliy Filippov	a19d2066c2	Document osd_backfillfull_ratio	2024-12-19 02:15:02 +03:00
Vitaliy Filippov	2a8780b4b5	Add a note about slow ops	2024-12-19 02:02:37 +03:00
Vitaliy Filippov	109f51a015	Implement basic VitastorFS support in CSI	2024-12-17 02:26:23 +03:00
Vitaliy Filippov	8a86c123c3	Allow to auto-select and print the port	2024-12-14 16:55:13 +03:00
Vitaliy Filippov	b856524e0c	Workaround for Linux bug: return post_op_attr for NFS-RDMA READ3 ... Linux NFS RDMA transport has a stupid bug - when the reply doesn't contain post_op_attr, the data gets offsetted by 84 bytes (size of attributes) and first 84 bytes are filled with probably random data.	2024-12-11 21:09:36 +03:00
Vitaliy Filippov	ae3ca7451f	Use per-connection RDMA device contexts	2024-12-11 21:09:36 +03:00
Vitaliy Filippov	1dbbb0c3f8	Implement NFS RDMA support	2024-12-11 21:09:36 +03:00
Vitaliy Filippov	64db31ec10	Fix slow op warning format	2024-12-11 21:09:36 +03:00
Vitaliy Filippov	76470686b3	Fix possibly incorrect linked list deserialization in NFS	2024-12-08 02:54:13 +03:00
Vitaliy Filippov	652ca631bb	Fix possible crash in nfs_block readdir	2024-12-01 18:04:49 +03:00
Vitaliy Filippov	2105f4b654	Add lost netlink daemonize	2024-11-27 17:13:30 +03:00
Vitaliy Filippov	0d01573da3	Fix typos	2024-11-26 14:31:47 +03:00
Vitaliy Filippov	d84b84f58d	Fix new backfillfull feature, add more logs	2024-11-23 01:08:13 +03:00
Vitaliy Filippov	8cfe705d7a	Map netlink after forking to show correct PID in vitastor-nbd ls	2024-11-23 00:46:44 +03:00
Vitaliy Filippov	66c9271cbd	Radically simplify create-pool pg_size check	2024-11-22 01:44:14 +03:00
Vitaliy Filippov	7b37ba921d	Pause pool rebalance when monitor detects that it can lead to any OSD becoming full	2024-11-22 01:01:07 +03:00
Vitaliy Filippov	262c581400	Fix create-pool for the case of hosts split into sub-nodes	2024-11-22 01:01:07 +03:00
Vitaliy Filippov	ad3b6b7267	Add a note about GID and RDMA device auto-selection	2024-11-21 23:54:05 +03:00
Vitaliy Filippov	1f6a061283	Move ibv_query_gid under #ifdef to only build it with libibverbs 32+	2024-11-21 23:47:57 +03:00
Vitaliy Filippov	fc4d97da10	Print "Waiting to become master" just once	2024-11-21 00:55:22 +03:00
Vitaliy Filippov	c7a4ce7341	Take out_size from dd oimg if not specified	2024-11-19 02:13:34 +03:00
Vitaliy Filippov	ddea31d86d	Auto-select first RDMA device only if RoCE is not found, add rocev2->rocev1->ib priority	2024-11-19 01:54:00 +03:00
Vitaliy Filippov	156d005412	Add serialize_overlap to test_heal	2024-11-17 01:26:35 +03:00
Vitaliy Filippov	7e076c7049	Do not report OSDs with empty statistics as full	2024-11-16 23:36:16 +03:00
Vitaliy Filippov	7de38250ad	Auto-select RDMA device based on osd_network	2024-11-16 18:38:57 +03:00
Vitaliy Filippov	9c59d30e83	Report slow ops in OSD stats in etcd and show them in vitastor-cli status	2024-11-16 15:11:16 +03:00
Vitaliy Filippov	5db02cdf6e	Add pve-qemu 9.0 patch	2024-11-16 11:20:47 +03:00
Vitaliy Filippov	8202ee9d74	Trigger double autosync when switching PG state to prevent leaving garbage in non-immediate_commit clusters	2024-11-15 01:26:36 +03:00
Vitaliy Filippov	5864bd067c	Add missing connection timeout for etcd websockets in OSD	2024-11-12 02:28:07 +03:00
Vitaliy Filippov	c312557ace	Do not execute remaining operations if the client is stopped during read	2024-11-10 16:44:13 +03:00
Vitaliy Filippov	5ce20116d8	Postpone trigger_nearest to prevent timer callbacks called from setTimer/clearTimer	2024-11-10 15:51:16 +03:00
Vitaliy Filippov	be66791e59	Add another note about 1.8 upgrade	2024-11-09 00:57:58 +03:00
Vitaliy Filippov	141cec2383	Add missing refcounting for flush_batch errors	2024-11-09 00:46:38 +03:00
Vitaliy Filippov	1ce4b1b417	Fix stop condition in osd_flush ... Could probably lead to PGs hung in peering states on OSD restart in EC pools, fixable by primary OSD restart	2024-11-08 00:30:40 +03:00
Vitaliy Filippov	ebf24bac9a	Fix partition zeroing during prepare ... Previously it zeroed area beginning with 0 instead of actual metadata offset which was leading to non-zeroed metadata when the disk is very small	2024-11-08 00:14:37 +03:00
Vitaliy Filippov	edd9051f81	Fix arch.en toc	2024-11-08 00:14:18 +03:00
Vitaliy Filippov	662ca86dc0	Fix libvirt 8 patch	2024-11-07 12:21:32 +03:00
Vitaliy Filippov	a1ca573168	Support QEMU 9.1	2024-11-07 12:21:13 +03:00
Vitaliy Filippov	f69f801ffb	Release 1.9.3 ... - Support custom hybrid OSD creation (`vitastor-disk prepare --hybrid --fast-devices /dev/xxx,/dev/yyy`) - Auto-change partition paths to /dev/disk/by-partuuid/ in `vitastor-disk prepare` - Allow to select cached I/O in vitastor-disk commands - Fix multiple bugs in vitastor-disk resize & add tests for them - Fix vitastor-disk write-meta/write-journal in superblock-based mode writing it to an incorrect device - Fix vitastor-disk prepare sometimes again not seeing new partitions - Cleanup PG history and stats of deleted pools - Fix "is already mounted" checks in CSI	2024-11-07 01:28:31 +03:00
Vitaliy Filippov	af92cbdfcc	Dynamic device size in test	2024-11-06 14:16:58 +03:00
Vitaliy Filippov	a775db10cc	Also allow cached I/O in dsk.open_*() in disk_tool	2024-11-06 13:52:25 +03:00
Vitaliy Filippov	eafce26049	Add resize and resize-auto tests	2024-11-06 13:30:51 +03:00
Vitaliy Filippov	625c74294f	Support direct I/O	2024-11-06 13:30:12 +03:00
Vitaliy Filippov	ef8c21ad6f	Change %lu to %ju	2024-11-06 02:58:51 +03:00
Vitaliy Filippov	2bb8e8999e	Do not check length in "data alignment mismatch"	2024-11-06 02:58:26 +03:00
Vitaliy Filippov	c2e7c28672	Fix calc_lengths data size recalc during auto-resize	2024-11-06 02:27:17 +03:00
Vitaliy Filippov	bd22beefb5	Auto-extend new_data_len if new_data_offset is changed too	2024-11-06 02:13:30 +03:00
Vitaliy Filippov	e7038ab99c	Auto-change partition paths to /dev/disk/by-partuuid/	2024-11-06 01:04:05 +03:00
Vitaliy Filippov	b6f75ebcfd	Add missing I/O path description in english	2024-11-06 00:43:17 +03:00
Vitaliy Filippov	9def199981	Auto-reduce new_data_len in resize	2024-11-05 02:57:11 +03:00
Vitaliy Filippov	c72e8e649e	Support test mode for vitastor-disk	2024-11-05 02:43:55 +03:00
Vitaliy Filippov	8bdb3e8786	Write meta/journal to correct device when used in superblock mode	2024-11-05 02:43:55 +03:00
Vitaliy Filippov	a87e236c70	Fix resize --data-size, particularly when expanding the device	2024-11-04 18:55:03 +03:00
Vitaliy Filippov	16f67cf6f1	Fix missing metadata checksums after resize	2024-11-04 18:36:35 +03:00
Vitaliy Filippov	56de4a520d	Support custom hybrid OSD creation (--hybrid --fast-devices /dev/xxx,/dev/yyy)	2024-11-04 17:52:29 +03:00
Vitaliy Filippov	adca162278	Note that osd_per_disk is also incompatible	2024-11-04 15:20:01 +03:00
Vitaliy Filippov	490b314d72	Rework & fix new partition waiting code	2024-11-04 15:16:30 +03:00
Vitaliy Filippov	9f52074e1e	Delete PG history and stats of deleted pools	2024-11-01 02:38:31 +03:00
Vitaliy Filippov	2b3e877546	Add notes about vitastor-disk in disable_data_fsync	2024-11-01 02:38:18 +03:00
Vitaliy Filippov	01d55e5420	Merge pull request #64 from 0x00ace/fio_version_fix ... use fio 3.35-1 for AlmaLinux 9	2024-10-31 11:55:40 +03:00
Vitaliy Filippov	f5aa5cfdfe	Fix "is already mounted" checks in CSI	2024-10-26 14:06:21 +03:00
Vitaliy Filippov	2826bb9e7e	Add more logging to CSI	2024-10-24 02:07:55 +03:00
Vitaliy Filippov	30d1ad0f66	Add Intel D5-P4320	2024-10-22 23:22:48 +03:00
Vitaliy Filippov	79719e44ac	Release 1.9.2 ... New features: - Support resizing normal vitastor-disk partitions and moving journal/metadata: [vitastor-disk resize](https://vitastor.io/docs/usage/disk.html#resize) - Support simple forms of vitastor-disk {dump,write}-{meta,journal} for OSD partitions Bug fixes: - Fix block RWX volumes broken after introducing stage/unstage support - Do not allow to create non-block RWX volumes in CSI - Fix vitastor-disk prepare not seeing the newly created partition in rare cases - Fix non-array tags not showing up in ls-osd/osd-tree - Make OpenNebula oned.conf patching during installation smarter - Fix iseek option in vitastor-cli dd not working - Validate conv=, iflag=, oflag= options in vitastor-cli dd - Fix vitastor-disk write-meta not writing header checksum to the disk - Fix JSON format in vitastor-disk dump-meta - Fix read_chain_bitmap not working for snapshot in another pool - Fix a possible OSD crash during parallel read & write to an image with snapshots - Several followups to the READ_CHAIN_BITMAP fix: avoid data reads, fix possible overflow in is_zero(), fix bitmap size	2024-10-20 01:49:13 +03:00
Vitaliy Filippov	f5626655df	Add new disk command docs	2024-10-20 01:47:46 +03:00
Vitaliy Filippov	7e2dde2702	Fix block RWX volumes broken after introducing stage/unstage support	2024-10-19 11:56:56 +03:00
Vitaliy Filippov	3b0ab317cf	Validate non-block RWX in CSI	2024-10-18 01:55:38 +03:00
Vitaliy Filippov	18eb99c494	Implement resizing partitions created with vitastor-disk	2024-10-18 01:55:19 +03:00
Vitaliy Filippov	4e8a1a8895	Run partprobe in add_partition() if /dev/disk/by-partuuid symlink is not present	2024-10-12 18:07:53 +03:00
Vitaliy Filippov	d27a8bdabc	Make get_parent_device return full path	2024-10-12 13:44:52 +03:00
Vitaliy Filippov	ebd616e42f	Extract clear_osd_superblock()	2024-10-12 13:44:52 +03:00
Vitaliy Filippov	b18d296e01	Extract check_existing_partition(), get_device_size()	2024-10-12 13:44:52 +03:00
Vitaliy Filippov	a03508320e	Move json_is_true/json_is_false to json_util.cpp	2024-10-12 00:40:39 +03:00
Vitaliy Filippov	c9ccc790ec	Fix non-array tags not showing up in ls-osd/osd-tree	2024-10-11 18:33:35 +03:00
Vitaliy Filippov	db2d9c5b3d	Fix tables in NFS doc	2024-10-08 00:20:10 +03:00
Vitaliy Filippov	09f15f44c9	Fix Toshiba MG and VDUSE Debian kernel note in docs	2024-10-08 00:17:14 +03:00
Vitaliy Filippov	c5a58c2e81	Support reading parameters automatically from the superblock in vitastor-disk {dump,write}-{meta,journal}	2024-10-07 02:21:58 +03:00
Vitaliy Filippov	30e7c2ad1e	Add custom OpenNebula oned.conf patcher (it uses a SHITTY configuration file format)	2024-10-06 13:46:05 +03:00
Vitaliy Filippov	2e76ceabbe	Fix iseek option in vitastor-cli dd	2024-10-05 18:25:38 +03:00
Vitaliy Filippov	3df088c207	Validate conv=, iflag=, oflag= options in vitastor-cli dd	2024-10-05 18:02:36 +03:00
Vitaliy Filippov	d882a19eab	Fix vitastor-disk write-meta not writing header checksum to the disk...	2024-10-05 17:32:55 +03:00
Vitaliy Filippov	702be3da7a	Fix JSON format in vitastor-disk dump-meta	2024-10-05 16:08:34 +03:00
Vitaliy Filippov	99533e1c2f	Fix .yml links	2024-10-02 00:38:07 +03:00
Vitaliy Filippov	a6cceb43bf	Fix read_chain_bitmap not working for snapshot in another pool	2024-10-02 00:24:48 +03:00
Vitaliy Filippov	745d89459a	Fix link, add title	2024-09-29 22:05:56 +03:00
Vitaliy Filippov	48f023292d	Fix extra data reads on read_chain	2024-09-21 17:05:42 +03:00
Vitaliy Filippov	b58bf3ada5	Fix possible OSD crash during parallel read & write to an image with snapshots ... OSDs could crash with the following "assertion failed" message (crash didn't affect data and was caused by OSD thinking upper blocks are full while they weren't). Reproduction without introducing artificial delays is hard because you have to force OSD to read an object with enqueued but not handled write which fills previously non-full bitmap. O_o. ``` vitastor-osd: ./src/osd/osd_primary_chain.cpp:613: void osd_t::send_chained_read_results(pg_t&, osd_op_t*): Assertion `stripes[role].read_buf' failed. ```	2024-09-21 13:44:36 +03:00
Vitaliy Filippov	f18a749324	READ_CHAIN fix was incomplete :-)	2024-09-21 13:40:31 +03:00
Vitaliy Filippov	6e9307c522	Fix possible overflow in is_zero()	2024-09-21 13:40:10 +03:00
Vitaliy Filippov	99adbb9483	Release 1.9.1 ... Hotfixes for OpenNebula and upgrade hotfix for 1.7 - Fix deploy.vitastor, save.vitastor, restore.vitastor scripts not working for nodes other than master oned - Fix deploy.vitastor not working for VMs without Vitastor disks - Disable clearing old PG configuration when upgrading from 1.7 or older versions (it was breaking old clients)	2024-09-14 19:17:30 +03:00
Vitaliy Filippov	b489a611a9	Add 1.8 upgrade note	2024-09-14 19:17:30 +03:00
Vitaliy Filippov	c6c0b8957a	Stop updating old PG configuration when the user manually deletes it	2024-09-14 19:15:40 +03:00
Vitaliy Filippov	5d40d2a459	Fix oned.conf patch	2024-09-14 19:08:44 +03:00
Vitaliy Filippov	f449c28c3b	Always write decoded base64 deployment file (otherwise it breaks VMs without Vitastor disks)	2024-09-14 15:25:02 +03:00
Vitaliy Filippov	a6274f58cc	Same fix for save/restore: they also need to ssh to target node	2024-09-14 02:46:48 +03:00
Vitaliy Filippov	ac29ffea6a	Add ssh to target node to deploy.vitastor - without it it always tried to deploy VMs on oned host	2024-09-14 02:15:24 +03:00
Vitaliy Filippov	bc06acc153	Disable clearing old PG configuration - we can not be sure that old clients do not need it	2024-09-13 19:00:12 +03:00
Vitaliy Filippov	fe8e611e23	Release 1.9.0 ... - OpenNebula support! [Installation instructions](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/installation/opennebula.en.md) - Added [vitastor-cli rm --exact|--matching](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/cli.en.md#rm) command - Added [vitastor-cli dd](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/cli.en.md#dd) command - copy files between Vitastor images, files and pipes - Add a startup timeout to vitastor-cli to not wait for etcd infinitely - Fix non-working OSD_OP_READ_CHAIN_BITMAP O_o - Autodetect block_size/bitmap_granularity/immediate_commit when creating pools - Do not allow to create multiple pools with the same name from vitastor-cli - Fix skip_cache_check option not applied due to type issue (see github issue #70)	2024-09-06 01:46:16 +03:00
Vitaliy Filippov	7636f9c726	Turn off brp-python-bytecompile in RPM specs	2024-09-06 01:44:44 +03:00
Vitaliy Filippov	d5f7005ddd	Add dd and rm --exact|--matching documentation	2024-09-05 02:22:05 +03:00
Vitaliy Filippov	70d6fcd32a	Add OpenNebula to README	2024-09-05 02:00:14 +03:00
Vitaliy Filippov	17caaa59af	vitastor-opennebula is probably more correct than opennebula-vitastor	2024-09-05 01:44:16 +03:00
Vitaliy Filippov	2dac6ee38b	Fix OpenNebula reinstall	2024-09-04 11:05:56 +03:00
Vitaliy Filippov	8be67a2d5b	Fix OpenNebula save/restore	2024-09-04 11:05:56 +03:00
Vitaliy Filippov	9c2132882c	Fix unaligned last block read/write in cli_dd	2024-09-04 11:05:56 +03:00
Vitaliy Filippov	9f25bb059b	Use just IMAGE_PREFIX, not IMAGE_PREFIX+"one"	2024-09-04 01:23:00 +03:00
Vitaliy Filippov	ee3094c5e5	Add OpenNebula plugin docs	2024-09-04 01:22:39 +03:00
Vitaliy Filippov	ba9f263b75	Add wildcard removal command	2024-08-31 14:13:09 +03:00
Vitaliy Filippov	30eaa1a8e6	Add vitastor-cli ls --exact	2024-08-31 02:36:25 +03:00
Vitaliy Filippov	6a8daedbe2	rm --wildcard	2024-08-31 02:36:25 +03:00
Vitaliy Filippov	2b96ac0b44	Implement OpenNebula driver	2024-08-30 23:46:37 +03:00
Vitaliy Filippov	986cd11705	Implement CLI "dd" command - copy data between Vitastor images, files and pipes	2024-08-30 02:31:06 +03:00
Vitaliy Filippov	b804051eaf	Remove debug print in nbd-proxy	2024-08-30 02:31:06 +03:00
Vitaliy Filippov	3cc326500e	Fix non-working OSD_OP_READ_CHAIN_BITMAP O_o	2024-08-30 01:25:05 +03:00
Vitaliy Filippov	f848c450a4	Clients should not wait infinitely for etcd to start if it's unavailable	2024-08-28 02:03:35 +03:00
Vitaliy Filippov	4121c66281	Autodetect block_size/bitmap_granularity/immediate_commit when creating pools	2024-08-28 02:03:35 +03:00
Vitaliy Filippov	b3716fbe23	Validate pool name when creating a pool	2024-08-28 02:03:35 +03:00
Vitaliy Filippov	97f49d7d94	Fix #70 from github - skip_cache_check type issue	2024-08-14 01:35:43 +03:00
Vitaliy Filippov	131de4b790	Disable trace in header	2024-08-13 11:21:35 +03:00
Vitaliy Filippov	ce359c5a69	Release 1.8.0 ... Bugfix release, would be 1.7.2, but etcd layout changes mandate it to be 1.8.0. :-) - Change etcd layout: /config/pgs is now /pg/config, /pg/stats/* is now /pgstats/*. This is required to fix a rare PG history tracking issue caused by non-atomic delivery of etcd events sometimes resulting in `incomplete` objects in EC pools after mass OSD restarts. Upgrading can be performed freely, downgrade requires additional action: [1.8.0 to 1.7.1](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/admin.en.md#1-8-0-to-1-7-1) - Fix a rare client hang on PG primary OSD switch - Fix vitastor-nfs started using mount command sometimes not stopping automatically after unmount - Fix vitastor-nfs mounts started using mount command sometimes hanging after daemonizing - Fix merge/flatten into a pool with different object size (image migration between pools case) - Do not print extra "PG disappeared after reload" verbose log messages for non-existing PGs - Fix clustered Antietcd support and persistence filter - Do not try to purge the same OSD multiple times if its multiple devices are passed to purge - Various node.js binding fixes	2024-08-11 14:28:31 +03:00
Vitaliy Filippov	521e867b10	Run check_exit also on deferred stop. Now vitastor-nfs should finally always stop on umount	2024-08-11 00:05:20 +03:00
Vitaliy Filippov	333c54ebbf	Cleanup clients correctly during stop(). Was also affecting #67, but could also reproduce during normal operation	2024-08-11 00:00:13 +03:00
Vitaliy Filippov	58d3da95c8	Fix github issue #67 by closing active NFS sockets before daemonize()	2024-08-10 20:13:37 +03:00
Vitaliy Filippov	4e90e752eb	Fix merge/flatten into a pool with different object size	2024-08-10 19:23:26 +03:00
Vitaliy Filippov	09342d7189	node.js binding fixes	2024-08-05 00:10:37 +03:00
Vitaliy Filippov	eb3e8b8c19	Do not print "PG disappeared after reload" verbose log messages when *it* was not present	2024-08-04 01:42:05 +03:00
Vitaliy Filippov	e2ca3ad99e	Add a note about storage ID in proxmox storage config doc	2024-07-31 01:19:44 +03:00
Vitaliy Filippov	dd4b0aed2b	Support scattered write in node.js binding	2024-07-31 01:17:06 +03:00
Vitaliy Filippov	42851a061c	Always continue operations to not miss resuming after the lack of PG primary ... Should fix spurious client hangs during PG primary switchover	2024-07-31 01:17:03 +03:00
Vitaliy Filippov	8e0f242d30	Add downgrade docs	2024-07-31 01:15:37 +03:00
Vitaliy Filippov	0daa8ea39b	Support seamless upgrade to new PG config and stats etcd key names	2024-07-31 01:15:37 +03:00
Vitaliy Filippov	b263d311ef	Use separate watch revisions for different watchers	2024-07-31 01:15:37 +03:00
Vitaliy Filippov	8720185780	Run tests in CI in memory (in tmpfs)	2024-07-31 01:15:37 +03:00
Vitaliy Filippov	20584414d8	Report OSD version in /osd/state/ and /osd/stats/ (for the future)	2024-07-31 01:15:37 +03:00
Vitaliy Filippov	306a3db7f3	Rename VERSION define to VITASTOR_VERSION	2024-07-31 01:15:37 +03:00
Vitaliy Filippov	5b0aebada4	Rename /config/pgs to /pg/config and /pg/stats/* to /pgstats/*	2024-07-31 01:15:37 +03:00
Vitaliy Filippov	d6f0b480c8	Fix broken link	2024-07-22 14:01:53 +03:00
Vitaliy Filippov	f1f8531fd4	Make tests compatible with antietcd, add 2 antietcd tests to CI	2024-07-20 02:16:38 +03:00
Vitaliy Filippov	8d79d59964	Update antietcd to 1.1.0	2024-07-20 02:15:48 +03:00
Vitaliy Filippov	551a209a50	Fix persistence filter initialization	2024-07-20 02:15:48 +03:00
Vitaliy Filippov	06cafd7702	Do not merge config an extra unneeded time	2024-07-20 02:15:48 +03:00
Vitaliy Filippov	3018352443	Fix clustered Antietcd support	2024-07-19 18:58:58 +03:00
Vitaliy Filippov	f8edfb4a71	No need to check for PG intersection if a history set is smaller than EC data part count	2024-07-18 19:29:05 +03:00
Vitaliy Filippov	8239ea2356	Do not try to purge the same OSD multiple times if its multiple devices are passed to purge	2024-07-16 16:48:16 +03:00
Vitaliy Filippov	e898335b8d	Release 1.7.1 ... Some stupid hotfixes for 1.7.0 :) - Fix NFS mount - Fix modify-osd - Fix use_antietcd not taken from /etc	2024-07-16 00:07:03 +03:00
Vitaliy Filippov	e7869611fa	Another stupid fix for NFS (no idea how it worked for me)	2024-07-16 00:05:51 +03:00
Vitaliy Filippov	e1c2500b60	Use modify-osd in the disk removal instruction	2024-07-16 00:01:42 +03:00
Vitaliy Filippov	42cf3a11df	Oops, fix reweight :)	2024-07-16 00:01:11 +03:00
Vitaliy Filippov	4d9293f0e9	Fix QEMU 8.2 and 9.0 patches (add @location comments)	2024-07-15 16:30:14 +03:00
Vitaliy Filippov	7a13f85ae2	Fix mon config merge	2024-07-15 16:25:22 +03:00
Vitaliy Filippov	fc219b8602	Add pg-list to docs	2024-07-15 13:29:22 +03:00
Vitaliy Filippov	989d73f874	Release 1.7.0 ... Omnidirectional release New features: - Support handling TCP I/O in simple separate io_uring-based [I/O threads](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/config/client.en.md#client_iothread_count) - may increase linear performance to 7-8 GB/s - Experimental internal etcd replacement - [antietcd](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/config/monitor.en.md#use_antietcd) - Monitor now has a [built-in Prometheus exporter](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/config/monitor.en.md#enable_prometheus) - Added a reference [Grafana dashboard](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/mon/scripts/Vitastor-Grafana-6+.json) - Implement vitastor-cli [osd-tree](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/cli.en.md#osd-tree) and [ls-osd](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/cli.en.md#ls-osd) commands - Implement vitastor-cli [modify-osd](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/cli.en.md#modify-osd) command - Implement vitastor-cli [pg-list](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/cli.en.md#pg-list) command - Implement [VitastorFS defragmentation](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/usage/nfs.en.md#defrag) - Implement basic node.js binding (not published on npm yet) Changes: - Make immediate_commit=all the default everywhere to match default vitastor-disk behaviour - Make pool-create error message more obvious and add details to it - Set default etcd_ws_keepalive_interval to 5 seconds (speedup client etcd failover) - Support OpenStack 2023.2 in Nova and Cinder drivers/patches - Add patches for libvirt 10.x - Add patches for QEMU 8.2 and 9.0 - Implement internal restart / run_forever in monitor - Some source tree refactoring - sources are now moved into subdirectories, monitor is now split into multiple files - Add vitastor_c_inode_get_immediate_commit in vitastor_c client library - Make vitastor_kv.h header public Bug fixes: - Fix total statistics usec/count/bytes not being reported when delta (bps/iops/lat) is zero - Prevent infinite loop in NFS on files with incorrect metadata pointing to an empty volume - Fix READDIR offsets (cookies) in VitastorFS sometimes leading to client infinite loops when reading a directory - Fix a rare infinite loop during OSD journal flushing (OSD hanging and eating 100 % CPU) - Fix several bugs which could lead to lost writes in setups without immediate_commit: - Client library treated writes as completed before actually completing them, thus missing them in a subsequent fsync - Client library didn't repeat writes on the new PG primary when it changed - OSDs didn't drop peer connections with dirty writes when stopping PG - Fix Block Pseudo-FS initialization leading to ENOENTs some time after start - Fix vitastor-cli merge-based commands (merge/flatten/rm snapshot) slowing down and finally failing when using CAS optimistic locks - Fix pool create/modify --block_size validation - Fix TTL comparison for determining failed lease/keepalive requests in OSD - Add support for size suffixes in pool-create --block_size and --immediate_commit values	2024-07-15 11:48:35 +03:00
Vitaliy Filippov	f0630722ce	Make pool-create error message more obvious, add details	2024-07-15 11:47:49 +03:00
Vitaliy Filippov	93b0947720	Support size suffixes in pool-create --block_size / --bitmap_granularity	2024-07-15 11:47:05 +03:00
Vitaliy Filippov	9c628646fa	Remove bullseye-backports from build, remove buster-backports from docs	2024-07-15 11:47:05 +03:00
Vitaliy Filippov	cf476a3b95	Add mkdir /var/lib/vitastor	2024-07-15 11:47:05 +03:00
Vitaliy Filippov	23f9273ba3	Take use_antietcd setting from /etc/vitastor/vitastor.conf too	2024-07-15 02:02:56 +03:00
Vitaliy Filippov	74b88bf8ba	Use own repo instead of buster-backports as it is EOL	2024-07-14 20:25:44 +03:00
Vitaliy Filippov	1254d5a0de	Fix delta stats when counters may be hypothetically reset	2024-07-14 13:11:00 +03:00
Vitaliy Filippov	f87bece253	Fix build with antietcd & tinyraft, remove some version hardcode	2024-07-14 13:04:25 +03:00
Vitaliy Filippov	ba85d0ef16	Add vitastor_kv.h to RPM specs	2024-07-14 11:20:37 +03:00
Vitaliy Filippov	17a909ea3a	Stop metrics/future API HTTP server when closing Monitor instance	2024-07-14 11:16:41 +03:00
Vitaliy Filippov	a4dfc220ab	Implement basic node.js binding (not published on npm yet)	2024-07-14 10:58:38 +03:00
Vitaliy Filippov	26426dd95e	Return it back, but fix stats in another way	2024-07-13 19:14:34 +03:00
Vitaliy Filippov	9f38b7e5c1	Fix osd_ping_time_remaining reset from 990c3ba7eb, leading to osd disconnections	2024-07-13 16:09:56 +03:00
Vitaliy Filippov	20057defbe	Revert 8ad63465cd	2024-07-13 15:34:34 +03:00
Vitaliy Filippov	b4e9140755	Add defrag docs, fix trace message	2024-07-13 00:45:53 +03:00
Vitaliy Filippov	413959e75a	Prevent infinite loop in NFS - return EIO when an inode points to an incorrect volume position	2024-07-12 20:53:54 +03:00
Vitaliy Filippov	8973982570	Delete keys from internal state instead of setting them to null on DELETE event in mon	2024-07-12 16:42:21 +03:00
Vitaliy Filippov	990c3ba7eb	Implement FS defragmentation	2024-07-12 16:11:35 +03:00
Vitaliy Filippov	1771d2ef36	Fix READDIR cookie/offset bug	2024-07-12 16:11:35 +03:00
Vitaliy Filippov	d88ab76636	Fix active mon stat	2024-07-11 01:34:59 +03:00
Vitaliy Filippov	c010a0aa54	Fix OSD "local write" latency sum	2024-07-11 01:30:03 +03:00
Vitaliy Filippov	0d42712d29	Fix refresh in dashboard variable	2024-07-11 01:13:02 +03:00
Vitaliy Filippov	66b438106a	Add vitastor-cli pg-list command	2024-07-10 02:27:41 +03:00
Vitaliy Filippov	3aef6682fb	Add vitastor-cli modify-osd command	2024-07-09 16:52:19 +03:00
Vitaliy Filippov	8535bccf4c	Add a note about antietcd dump/load	2024-07-09 15:58:03 +03:00
Vitaliy Filippov	0487b3b239	Add clusterid to Grafana dashboard	2024-07-09 15:58:03 +03:00
Vitaliy Filippov	a54ef97f5d	Add Grafana dashboard link	2024-07-09 15:37:25 +03:00
Vitaliy Filippov	10434a9b2b	Add notes about antietcd to documentation	2024-07-09 15:01:41 +03:00
Vitaliy Filippov	c6be194508	Implement experimental antietcd-based version of monitor	2024-07-09 13:54:58 +03:00
Vitaliy Filippov	df668286fb	Add Grafana dashboard	2024-07-09 02:39:36 +03:00
Vitaliy Filippov	667c5999c9	Report all PG states	2024-07-08 19:52:56 +03:00
Vitaliy Filippov	8ad63465cd	Do not wipe previous metrics at moments when difference is 0	2024-07-08 02:20:12 +03:00
Vitaliy Filippov	976290e6a9	Implement built-in Prometheus exporter in monitor	2024-07-08 02:20:12 +03:00
Vitaliy Filippov	79f1d1969b	Make immediate_commit=all the default	2024-07-07 11:45:18 +03:00
Vitaliy Filippov	918e1f83b0	Add JSON output for ls-osd	2024-07-07 02:24:36 +03:00
Vitaliy Filippov	abbba6ade4	Support handling TCP I/O in simple separate io_uring-based I/O threads ... Required mainly for clients, allows to scale parallel client I/O with TCP from 100-150k iops to ~400k iops and from 2-3 GB/s to at least 7-8 GB/s with 4 I/O threads, at the same time increasing Q=1 latency by 2x thread switching delay, which is ~10 us when CPU powersaving is disabled and may be as high as 200 us when it's enabled.	2024-07-04 13:29:20 +03:00
Vitaliy Filippov	21d1171ba4	Fix parsing after "slightly decopypasting" :)	2024-06-29 00:09:30 +03:00
ace	8f83086889	Nova and cinder driver patches for OpenStack 2023.2	2024-06-28 00:04:57 +03:00
Vitaliy Filippov	ceb18f25db	Add libvirt 10.0 patch (same as 9.10 and 10.4 actually)	2024-06-28 00:03:46 +03:00
Vitaliy Filippov	ed51a89f70	Add QEMU 8.2 and 9.0 patches	2024-06-27 12:33:16 +03:00
Vitaliy Filippov	f59456f22d	Add libvirt 10.4 patch (same as 9.10 actually)	2024-06-27 01:35:29 +03:00
Vitaliy Filippov	ca63cd507d	Fix possible infinite loop in flusher (surprisingly reproduced in test_write.sh with iothreads)	2024-06-27 00:38:01 +03:00
Vitaliy Filippov	ea0d72289c	Treat copied buffers as written only after completing the write in client ... SYNC operation fsyncs only completed operations, so treating writes as "eligible for fsync" before actually completing them is incorrect It affected SCHEME=ec test_heal.sh (with immediate_commit=none) test - it was flapping with lost writes - some non-fsynced writes were legitimately lost by the OSD, but weren't repeated by the client	2024-06-20 02:11:53 +03:00
Vitaliy Filippov	e400a851f4	Repeat dirty buffer flushes on any PG primary change because the new primary may not know about unfinished operations of the old primary	2024-06-19 00:28:26 +03:00
Vitaliy Filippov	0fec7a9fea	Drop dirty peer connections also when stopping PG to guarantee that clients do not miss fsync	2024-06-19 00:28:26 +03:00
Vitaliy Filippov	b9de2a92a9	Print OSD performance stats	2024-06-17 13:02:58 +03:00
Vitaliy Filippov	5360a70853	Make OSD also report derived stats	2024-06-17 13:02:52 +03:00
Vitaliy Filippov	4c2328eb13	Implement ls-osd command	2024-06-17 02:22:14 +03:00
Vitaliy Filippov	313daef12d	Slightly decopypaste etcd key parsing	2024-06-17 01:38:42 +03:00
Vitaliy Filippov	ad9c12e1b9	Fix Pseudo-FS initialization leading to ENOENTs some time after start	2024-06-16 23:43:09 +03:00
Vitaliy Filippov	4473eb5512	Fix slow & failing CAS layer merge	2024-06-14 02:15:49 +03:00
Vitaliy Filippov	6501abc060	Set default etcd_ws_keepalive_interval to 5	2024-06-08 00:38:48 +03:00
Vitaliy Filippov	1228403e74	Implement internal restart / run_forever in monitor	2024-06-08 00:35:18 +03:00
Vitaliy Filippov	4eabebd245	Put all configuration to Mon.config	2024-06-07 00:20:38 +03:00
Vitaliy Filippov	cf60b6818c	Extract PG generation into pg_gen.js	2024-06-05 11:22:06 +03:00
Vitaliy Filippov	1a4a7cdc37	Extract OSD Tree generation functions to osd_tree.js	2024-06-05 11:19:35 +03:00
Vitaliy Filippov	1b48085e21	Extract remote etcd interaction to etcd_adapter.js	2024-06-05 11:19:35 +03:00
Vitaliy Filippov	a71847244e	Rename PGUtil.js to pg_utils.js	2024-06-05 10:51:20 +03:00
Vitaliy Filippov	848c2d2722	Move LPOptimizer, DSL and tests to lp_optimizer/	2024-06-05 10:51:20 +03:00
Vitaliy Filippov	86832dc43f	Add eslint import/no-unresolved	2024-06-05 10:51:20 +03:00
Vitaliy Filippov	1f6da79463	Extract stats calculation into a separate file	2024-06-05 10:51:20 +03:00
Vitaliy Filippov	9bf57c3760	Mention generic Toshiba MG instead of specific MGxx, fix russian vitastorfs link	2024-06-05 02:08:09 +03:00
Vitaliy Filippov	a0305b5b4a	Extract pool configuration validation into a separate file	2024-06-05 02:08:08 +03:00
Vitaliy Filippov	1546f8e447	Extract etcd data "schema" into a separate file	2024-06-05 02:07:53 +03:00
Vitaliy Filippov	8ce962b312	Move scripts	2024-06-05 02:07:53 +03:00
Vitaliy Filippov	50e56b3b92	Add vitastor_c_inode_get_immediate_commit	2024-05-19 01:57:18 +03:00
ace	b85dab8583	use fio 3.35-1 for AlmaLinux 9	2024-05-18 21:17:16 +03:00
Vitaliy Filippov	a12d328793	Rename cli/ to cmd/, fix cmake install	2024-05-15 23:04:50 +03:00
Vitaliy Filippov	c79b38bd26	Move all sources to subdirs	2024-05-15 11:06:01 +03:00
Vitaliy Filippov	44692d148a	Make vitastor_kv.h header public	2024-05-15 01:49:38 +03:00
Vitaliy Filippov	ba52359611	Fix last master commit	2024-05-15 01:49:31 +03:00
Vitaliy Filippov	23a9aa93b5	Fix pool create/modify --block_size validation	2024-05-04 16:33:22 +03:00
Vitaliy Filippov	2412d9e239	Fix TTL comparison for lease/keepalive	2024-04-30 01:53:05 +03:00
Vitaliy Filippov	9301c857b1	Release 1.6.1 ... A bunch of monitor fixes - Add noout flag for OSDs (/vitastor/config/osd/xx) - Fix "effective" size of degraded PGs (and thus "used space") calculation in monitor - Fix monitor not clearing PGs of deleted pools - Fix incorrect PG generation with hosts with 0 OSDs - Fix monitor crashing during primary OSD recheck when pool has no PGs - Fix monitor crashing when node_placement included non-existing OSDs - Fix possible data movement after removing OSDs reweighted to 0 - Remove extra empty keys from pool configurations created by vitastor-cli create-pool - Fix 32-bit build	2024-04-22 02:01:29 +03:00
Vitaliy Filippov	3094358ec2	Fix autovivification leading to extra empty keys in pool-create	2024-04-20 02:04:09 +03:00
Vitaliy Filippov	87f666d2a2	Filter out OSDs reweighted to 0	2024-04-20 02:03:53 +03:00
Vitaliy Filippov	bd7fe4ef8f	Filter out non-existing OSDs added in node_placement	2024-04-20 02:03:36 +03:00
Vitaliy Filippov	1b3f9a1416	Do not set non-existing OSD weight to 0, we'll remove them instead	2024-04-20 02:03:11 +03:00
Vitaliy Filippov	a7b7354f38	Do not recheck primary distribution when pool has no PGs	2024-04-20 02:02:47 +03:00
Vitaliy Filippov	765befa22f	Remove empty nodes from tree because PG DSL expects that all leaf nodes are OSDs	2024-04-20 02:02:28 +03:00
Vitaliy Filippov	87b3ab94fe	Do not disable require-atomic-updates and no-unused-vars	2024-04-20 02:02:13 +03:00
Vitaliy Filippov	2c0801f6e4	Configure ESLint and add it to CI	2024-04-16 02:39:31 +03:00
Vitaliy Filippov	fd83fef1d9	Fix pool deletion	2024-04-16 02:20:26 +03:00
Vitaliy Filippov	8d1067971b	Fix pg_effsize (and thus "used space") calculation in monitor	2024-04-16 02:20:18 +03:00
Vitaliy Filippov	ae5af04fde	Add noout flag for OSDs	2024-04-16 02:19:55 +03:00
Vitaliy Filippov	266d038b11	Fix 32-bit build warnings and one error again :-)	2024-04-11 22:49:33 +03:00
Vitaliy Filippov	ff4414d37e	Release 1.6.0 ... New features: - Implement "hierarchical failure domains" and other complex distribution rules, for example EC 4+2 over 3 DC, with 2 chunks per each DC ([documentation](docs/config/pool.en.md#level_placement)) - Make OSDs handle ENOSPC - now cluster stays online even if some OSDs fill up to 100 %, only writes requiring free space hang - Implement Stage/Unstage & volume locking for CSI to prevent parallel mounting and/or modifications of the same volume - Warn about full and almost full OSDs in vitastor-cli status - Add an experimental NBD netlink map mode as an option ([documentation](docs/usage/nbd.en.md)) - Add --pg parameter to vitastor-cli describe, print objects with 0x in human-readable format too - Add [administration docs](docs/usage/admin.en.md) Bug fixes: - Fix client operation retry timeout - previously the timeout wasn't applied and writes were retries almost instantly - Fix monitors crashing on invalid pool configurations - Fix journaling - make each journal write wait for all previous journal writes - Fix monitor thinking that OSD weight is 0 after deleting /osd/config/ key online - Fix a write stall caused by flusher possibly not trimming journal on rollback - Set 32k csum_block_size for HDD by default in vitastor-disk	2024-04-09 16:57:59 +03:00
Vitaliy Filippov	0fa7ecc03f	Add also a test for OSD tags	2024-04-09 16:57:59 +03:00
Vitaliy Filippov	c29bfe12eb	Oops - fix filter_by_root_node, add a test for it	2024-04-09 15:48:44 +03:00
Vitaliy Filippov	57bf84ddb2	Fix filtering in mon	2024-04-09 14:51:05 +03:00
Vitaliy Filippov	dff4879c8c	Check if NBD_ATTR_BACKEND_IDENTIFIER is defined	2024-04-09 13:16:58 +03:00
Vitaliy Filippov	af9a853db6	Move NBD netlink map&unmap to separate commands, add "netlink-revive" command	2024-04-08 16:34:41 +03:00
Vitaliy Filippov	b7a3275af3	Make netlink optional	2024-04-08 01:51:28 +03:00
Vitaliy Filippov	64c5c4ca26	Fix code style	2024-04-08 01:35:03 +03:00
idelson	442a9d838d	nbd-proxy: add configuration via netlink to support kinds of timeouts. ... PR #58 - https://github.com/vitalif/vitastor/pull/58/commits By MIND Software LLC By submitting this pull request, I accept Vitastor CLA	2024-04-08 00:50:08 +03:00
Vitaliy Filippov	6366972fe8	Warn about full and almost full OSDs in status	2024-04-07 19:39:51 +03:00
Vitaliy Filippov	2b863fb715	Add ENOSPC handling tests	2024-04-07 19:39:33 +03:00
Vitaliy Filippov	3bf4dd5abd	Fix client op retry timeout - do not retry immediately	2024-04-07 19:08:36 +03:00
Vitaliy Filippov	3b84dcaedd	Handle ENOSPC during write - rollback partial EC writes, remember partial replica writes	2024-04-07 18:02:05 +03:00
Vitaliy Filippov	20fbc4a745	Add --pg parameter to vitastor-cli describe, print objects with 0x in human-readable format too	2024-04-07 12:39:46 +03:00
Vitaliy Filippov	02993ee1dd	Implement Stage/Unstage & volume locking for CSI to prevent parallel modifications of the same volume	2024-04-07 11:48:19 +03:00
Vitaliy Filippov	3629dbc54d	Plug the new PG combinator into monitor	2024-04-07 02:44:17 +03:00
Vitaliy Filippov	29284bef40	Implement new DSL/rule-based PG generation algorithm	2024-04-07 00:36:20 +03:00
Vitaliy Filippov	6a924d6066	Extract PG combinator into a separate module	2024-04-07 00:36:20 +03:00
Vitaliy Filippov	9fe779a691	Do not die on invalid pool configurations	2024-04-07 00:36:20 +03:00
Vitaliy Filippov	31c2751b9b	Move NBD/VDUSE map/unmap functions to a separate file	2024-04-07 00:36:09 +03:00
Vitaliy Filippov	c5195666cd	Fix journal sequencing: make each journal write wait for all previous journal writes	2024-04-06 23:53:12 +03:00
Vitaliy Filippov	f36d7eb76c	Fix monitor thinking that OSD weight is 0 after deleting /osd/config/ key	2024-04-05 23:14:46 +03:00
Vitaliy Filippov	dd7f651de1	Add --max-request-bytes=104857600 to etcd params in tests	2024-04-05 23:14:46 +03:00
Vitaliy Filippov	a2994ecd0d	Fix flusher possibly not trimming journal on rollback	2024-04-05 23:14:39 +03:00
Vitaliy Filippov	5d3aaf016b	Add administration docs	2024-03-31 01:54:52 +03:00
Vitaliy Filippov	0b097ca3f2	Set 32k csum_block_size for HDD by default	2024-03-30 16:16:49 +03:00
Vitaliy Filippov	989675a780	s/etcd_ws_keepalive_timeout/etcd_ws_keepalive_interval/ in docs	2024-03-26 01:56:08 +03:00
Vitaliy Filippov	f8c403ec9e	Add newer benchmark results	2024-03-23 18:28:48 +03:00
Vitaliy Filippov	bfbb85e653	Replace -Oanything with -O3, not just -O/-O1/-O2	2024-03-18 02:03:44 +03:00
Vitaliy Filippov	9ad6822353	Release 1.5.0 ... After half a year of hard work, VitastorFS is finally here ! :-) New features: - VitastorFS, a full-featured clustered (read-write-many) file system. Documentation: [VitastorFS](docs/usage/nfs.en.md) - Embedded key-value database implementation based on Parallel Optimistic B-Tree algorithm and used for the metadata of VitastorFS - Pool management commands in vitastor-cli (create-pool, list-pools, rm-pool, modify-pool). Thanks MIND Software (https://mindsw.io) for their contribution! [Documentation](docs/usage/cli.en.md#create-pool) Bug fixes: - Fix a very rare "infinite loop" in the client library - Fix a rare OSD hang on during start when zeroing out bad metadata entries left from the previous run	2024-03-16 15:35:10 +03:00
Vitaliy Filippov	2043b4e374	Fix build errors for gcc 8	2024-03-16 15:35:10 +03:00
Vitaliy Filippov	de840e6fe3	Reduce kv-cli loadjson load parallelism to 16	2024-03-16 15:35:10 +03:00
Vitaliy Filippov	b5e04bf809	Fix build warning	2024-03-16 15:35:10 +03:00
Vitaliy Filippov	8807a1623b	Fix markdown tables	2024-03-16 15:35:10 +03:00
Vitaliy Filippov	f12855c31b	Add vitastor-kv to packages	2024-03-16 15:35:10 +03:00
Vitaliy Filippov	e75dcc9a71	Add documentation for VitastorFS	2024-03-16 15:16:43 +03:00
Vitaliy Filippov	88516ab4bd	Remove extra log	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	6221126b4f	Allow to print simple-offsets just given the device size	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	6783d4a13c	Implement fool protection for FS pools	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	dcbe1afac3	Store pool ID in inode metadata	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	0bde28c24a	Make nfs_do_rmw a library function	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	bb8ca6184e	Support setattr guard	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	87310ef7bb	Support ctime	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	4f4b2dab80	Log NFS liveness checks	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	f70da82317	Add loadjson command to vitastor-kv	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	e42148f347	Allow to specify KV commands on command line	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	c289584469	Add JSON dump format	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	018e89f867	Erase verf key left from creation from ientries on every modification	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	603dc68f11	Implement async mtime change	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	7b12342933	Allow to specify additional NFS mount options	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	44bf0f16ee	Fix malloc/free in nfs_kv_read/write	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	8840c84572	Fix "bad key in etcd" in mon for FS pools	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	5b747c12ec	Check if already mounted before mounting	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	05f5f46162	Fix zero used space, update mtime when moving/changing inode	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	b5604191c8	Ignore ECANCELED in nfs-proxy (happens in io_uring on fork)	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	e871de27de	Support unaligned shared_offsets, align shared file data instead of header	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	f600ce98e2	Implement auto-unmount local NFS server mode for vitastor-nfs	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	57605a5c13	Return error on failed shrink	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	29bd4561bb	Implement rename over an existing file/directory	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	7142460ec8	Support --logfile in nfs-proxy	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	d03f19ebe5	Fix shared file overlap, add FIXMEs	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	88f9d18be3	Create inode, then direntry, not direntry, then inode; retry ID collisions	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	6213fbd8c6	Fix NFS shared/aligned write FIXMEs	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	3aee37eadd	Allow to disable per-inode stats for VitastorFS pools	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	ecfc753e93	Add basic NFS tests, fix bugs	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	a574f9ad71	Return block NFS implementation back as an option too	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	7c235c9103	Move KV FS header into a separate file	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	e5bb986164	Implement packing small files into shared inodes	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	181795d748	Split new NFS proxy implementation into multiple files	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	8cdc38805b	WIP VitastorFS with metadata storage in VitastorKV	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	0cd455d17f	First just recheck version without actually re-reading block in vitastor-kv	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	32ba653ba6	Fix vitastor-kv hang on reopen & unfinished closed listing	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	231d4b15fc	Add loadable dump format to vitastor-kv (dump)	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	9dc4d5fd7b	Fix freeing r/w buffers on errors in kv_db	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	e58538fa47	Fix eviction when random_pos selects the end	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	11ac9e7024	Implement min/max list_count to make listings during performance test reasonable	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	511bc3df1c	Fix and improve parallel allocation ... - Do not try to allocate more DB blocks in an inode block until it's "confirmed" and "locked" by the first write - Do not recheck for new zero DB blocks on first write into an inode block - a CAS failure means someone else is already writing into it - Throw new allocation blocks away regardless of whether the known_version is 0 on a CAS failure	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	a64f0d1f73	Implement key_prefix for K/V stress test	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	ec5f7c6b87	More fixes ... - do not overwrite a block with older version if known version is newer (read may start before update and end after update) - invalidated block versions can't be remembered and trusted - right boundary for split blocks is right_half when diving down, not key_lt - restart update also when block is "invalidated", not just on version mismatch - copy callback in listings to avoid closure destruction bugs too	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	3ebed9a749	Add logging and one more assert	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	eab67a6e8f	Make get_block() wait for updating when unrelated block is found along the path	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	20993d9b7a	Fix a race condition where changed blocks were parsed over existing cached blocks and getting a mix of data	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	5cf9b343c0	Simplify code by removing an unneeded "optimisation"	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	79ae0aadcd	Add kv_log_level, print warnings on level 1, trace ops on level 10	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	605afc3583	Fix duplicate keys in listings on parallel updates -- do not rewind key "iterator position"	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	c0681d8242	Implement key suffix to avoid collisions of multiple test workers	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	763e77b4f4	Do not complain on empty first block	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	19426aa4c5	Add JSON output for stress-tester	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	08f586bcec	Print total stats	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	f1cd87473a	Do not send more than op_count operations (fix segfault on finish)	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	1bd8d2da56	Add some more resiliency to serialize()	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	a7396d2baf	Invalidate blocks being updated too	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	e98a38810d	Change new block allocation method: make each writer choose multiple empty PG blocks and place blocks in them	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	28c4324c36	Remove blocks from cache on unsuccessful updates	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	31ec3fa8f5	Allow to track multiple updates per block (it should never happen though)	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	e4fa26f60a	Do not call stop_updating after failed write_new_block and after clear_block (both delete the item)	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	59ae27f9e5	Track versions of parent blocks and recheck if changed during update	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	2c6a301d9b	Fix resume_split condition (key_lt can also be "")	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	01558349f8	Experiment: transform offsets for better sharding	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	36f4717d0d	More post-stress-test fixes ... - Prevent _split types of new blocks - Stop updating new blocks only after the whole update, otherwise pointers may become invalid - Use recheck_none for updates initially - Use UINT64_MAX as initial block version when postponing ops, otherwise the check fails when the block is initially empty. This for example leads to writing both leaf items & block pointers (which is incorrect) into the root block when starting stress-test with --parallelism 32 - Fix -EINTR comparison	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	babaf2a0ce	Print operation statistics	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	5773f1a375	K/V fixes after stress-test :-) ... - track block versions correctly - per inode block (128kb) instead of tree block (4kb) - prevent multiple parallel CAS writes of the same inode block - add logging for EILSEQ which means invalid data in the tree - fix get_block updated flag which was true for blocks already in cache and was leading to infinite loops on "unrelated block" errors - apply changes to blocks in cache only after successful writes (using "virtual changes") - do not replace cached block with an older version from disk - recheck "unrelated blocks" (read/update collisions) until data stops changing - track tree path correctly - do not treat split block as parent of its right half - correctly move blocks when finding new empty place on disk - restart updates from the beginning when one of blocks is changed by a parallel update - fix delete using SET opcode and setting key to the empty value instead - prevent changing the same key more than 1 time in parallel - fix listing verification - resume continue_updates in update_find (required because it uses continue_update itself) - add allow_old_cached parameter to get()	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	57222a9f79	Implement K/V DB stress tester	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	61ef000c6e	Evict blocks based on memory limit & block usage	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	7d5e1cc393	Track blocks per level	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	5e7f27a02d	Track block level	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	fd1d8a8520	Experimental B-Tree Vitastor embedded K/V database implementation!	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	c364e14c40	Stop then retry, not retry then stop	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	3ebbfa0428	Fix another rare OSD hang on zeroing out entries on start	2024-03-16 13:24:36 +03:00
Vitaliy Filippov	aa79d1db1c	Fix incorrect "changing scheme" message in modify-pool	2024-03-06 00:41:35 +03:00
Vitaliy Filippov	a1fecb7eff	Move callback away when calling it in cluster_client	2024-03-06 00:41:35 +03:00
Vitaliy Filippov	ff74b19423	Fix rare OSD hang on zeroing out bad entries on start	2024-03-06 00:41:35 +03:00
Vitaliy Filippov	4cf6dceed7	Merge branch 'rel-1.4'	2024-02-29 09:59:01 +03:00
Vitaliy Filippov	38b8963330	Release 1.4.8 ... - Do not use \r if output is not a terminal (should fix unexpected job output in proxmox) - Fix rm/rm-data error return code, add --down-ok option to bypass the error - Add EIO retry timeout and allow to disable these retries, rename up_wait_retry_interval to client_retry_interval - Add ubuntu jammy build - Wait for blockstore initialisation before starting OSD (prevent timeouts when init takes time) - Fix a rare use-after-free in automatic sync after delete in blockstore	2024-02-29 09:58:34 +03:00
Vitaliy Filippov	77167e2920	Do not use \r if output is not a terminal	2024-02-29 00:21:17 +03:00
Vitaliy Filippov	5af23672d0	Fix rm/rm-data error return code, add --down-ok option to bypass the error	2024-02-29 00:20:10 +03:00
Vitaliy Filippov	6bf1f539a6	Add EIO retry timeout and allow to disable these retries, rename up_wait_retry_interval to client_retry_interval	2024-02-28 13:10:02 +03:00
Vitaliy Filippov	4eab26f968	Add documentation and a very basic test for pool management commands	2024-02-28 13:08:04 +03:00
Vitaliy Filippov	86243b7101	Rework & fix pool-create / pool-modify / pool-ls	2024-02-28 13:08:04 +03:00
idelson	dc92851322	vitastor-cli: add commands to control pools: pool-create, pool-ls, pool-modify, pool-rm ... PR #59 - https://github.com/vitalif/vitastor/pull/58/commits By MIND Software LLC By submitting this pull request, I accept Vitastor CLA	2024-02-28 13:08:04 +03:00
Zibort Cloud	02d1f16bbd	Add ubuntu jammy build ... PR #62 #62 I accept Vitastor CLA agreement: https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/CLA-en.md	2024-02-28 11:43:54 +03:00
Vitaliy Filippov	fc413038d1	Wait for blockstore initialisation before starting OSD	2024-02-27 02:20:04 +03:00
Vitaliy Filippov	1bc0b5aab3	Fix a rare use-after-free in automatic sync after delete in blockstore ... ASan report: [0] READ of size 16 at operator() /root/vitastor/src/blockstore_write.cpp:100 ...[5] blockstore_impl_t::ack_sync(blockstore_op_t*) /root/vitastor/src/blockstore_sync.cpp:232	2024-02-24 00:06:36 +03:00
Vitaliy Filippov	5e934264cf	Release 1.4.7 ... - Fix another old "BUG: Attempt to overwrite used offset" in a very simple case: bs=4k rw=write iodepth=16 from OSD start; add this case to tests - Fix a rare crash with "unexpected state during flush: 0x51" possible with EC since 1.4.2 during rebalance and OSD outages - Fix a rare write stall with EC & immediate_commit=none caused by sync operations reserving unneeded space in the journal - Fix 32-bit build warnings, most in printf/scanf format strings	2024-02-22 12:45:52 +03:00
Vitaliy Filippov	f20564b44b	Fix 32-bit build warnings (99.9% in printf)	2024-02-22 12:22:16 +03:00
Vitaliy Filippov	b3c15db331	32M journal by default in simple-offsets	2024-02-21 15:25:02 +03:00
Vitaliy Filippov	685bcd6ef9	Do not reserve extra space for big_writes during sync - sync itself is needed to commit and clear them	2024-02-21 13:00:14 +03:00
Vitaliy Filippov	3eb389b321	Supposed fix for "unexpected state during flush: 0x51" with EC	2024-02-21 01:32:06 +03:00
Vitaliy Filippov	3d16cde23c	Fix assertions, add small sequential write test	2024-02-20 19:41:48 +03:00
Vitaliy Filippov	c6406d67fc	Fix journal space_check incorrectly checking for space at the beginning	2024-02-20 19:40:56 +03:00
Vitaliy Filippov	f87964861d	Release 1.4.6 ... Unwavering stabilization of 1.4.x, continued :-) - Include the accidentally lost part of 1.4.5 journal trimming fix - Fix a possible OSD crash with "BUG: Attempt to overwrite used offset" which was probably present for long time, but became apparent after fixing flapping tests in CI - Fix remaining flapping tests in CI. It was the first time when tests actually passed without retries :-)	2024-02-20 17:01:26 +03:00
Vitaliy Filippov	62a4f45160	Raise test_scrub waiting timeout	2024-02-20 16:26:09 +03:00
Vitaliy Filippov	7048228678	Supposed fix for "BUG: Attempt to overwrite used offset"	2024-02-20 15:56:48 +03:00
Vitaliy Filippov	ea73857450	Add asserts to catch "BUG: Attempt to overwrite used offset"	2024-02-20 15:56:48 +03:00
Vitaliy Filippov	6cfe38ec04	Followup to empty cur.oid as stop condition for forced trim fix	2024-02-20 15:56:38 +03:00
Vitaliy Filippov	7ae5766fdb	Wait to clear has_degraded in test_heal - should fix flaps of test_heal_* in CI	2024-02-20 15:56:27 +03:00
Vitaliy Filippov	f882c7dd87	Release 1.4.5 ... - Fix a write stall caused by incorrect journal trimming introduced in 1.4.4 :) - Fix PGs sometimes hanging in "starting" state on mass OSD restarts - Fix a rare crash with "map::at" during OSD pings - Use new defaults for non-capacitor (desktop) SSDs - improves T1Q256 random write from ~6k iops to ~45k iops - Make journal_trim_interval configurable	2024-02-16 10:13:33 +03:00
Vitaliy Filippov	26dd863c8d	Fix sometimes possible crash on clients.at() during pings	2024-02-16 10:13:33 +03:00
Vitaliy Filippov	2ae859fbc6	Use min/max_flusher_count=32/256, 128M journal and autosync_writes=512 for non-capacitor SSDs by default	2024-02-16 10:13:33 +03:00
Vitaliy Filippov	f6cd9f9153	Add a note about pg_minsize	2024-02-15 23:38:52 +03:00
Vitaliy Filippov	8389c0f33b	Fix PGs sometimes hanging in "starting" state on mass OSD restarts	2024-02-15 23:38:52 +03:00
Vitaliy Filippov	9db2196aef	Make journal_trim_interval configurable	2024-02-15 23:38:51 +03:00
Vitaliy Filippov	8d6ae662fe	Use empty cur.oid as stop condition for forced trim, not journal_trim_counter	2024-02-15 23:27:17 +03:00
Vitaliy Filippov	c777a0041a	Release 1.4.4 ... A couple of fixes for EC pools - Fix a segfault possible on partial EC overwrite in 1234 -> 5030 rebalance scenario - Fix two problems leading to EC pools stalling on rebalance & parallel sudden stops of OSDs, for example during a sudden poweroff of a host: - Recovery auto-tuning (1.4.0 feature) could apply too large delays and stall the EC journal - fixed by limiting delays with a new recovery_tune_sleep_cutoff_us parameter (10 seconds by default) and applying recovery pauses before write operations, not after them, to not occupy space in the journal for long time - Dynamic journal space reservation (1.3.0 feature) wasn't accounting new writes when checking the limit so OSDs could still fill the journal fully and stall - fixed by including new writes into the limit - Print etcd dbSize instead of dbSizeInUse in status	2024-02-11 16:23:08 +03:00
Vitaliy Filippov	2947ea93e8	Raise test_snapshot_chain_ec timeout to 6 minutes	2024-02-11 16:13:52 +03:00
Vitaliy Filippov	978bdc128a	Apply recovery pause before writes, after commits, and do not apply it to syncs to not block EC pools from functioning	2024-02-11 16:13:52 +03:00
Vitaliy Filippov	bb2f395f1e	Add cutoff threshold for recovery auto-tuning	2024-02-11 16:13:52 +03:00
Vitaliy Filippov	b127da40f7	Add a FIXME about incomplete PGs	2024-02-11 13:42:51 +03:00
Vitaliy Filippov	ca34a6047a	Fix dynamic journal space reservation: include the new write itself, too	2024-02-11 13:42:51 +03:00
Vitaliy Filippov	38ba76e893	Fix flusher sometimes being unable to trim journal when the flush queue is empty	2024-02-11 13:42:51 +03:00
Vitaliy Filippov	1e3c4edea0	Print etcd dbSize instead of dbSizeInUse in status	2024-02-11 13:42:51 +03:00
Vitaliy Filippov	e7ac855b07	Fix that EC segfault (1234 -> 5030 partial overwrite)	2024-02-11 13:42:51 +03:00
Vitaliy Filippov	c53357ac45	Add a test for EC segfault with partial overwrite in 1234 -> 5030 rebalance scenario	2024-02-11 13:42:51 +03:00
Vitaliy Filippov	27e9f244ec	Release 1.4.3 ... Hotfix for hotfix O:-) - "Write stall fix" was incomplete and EC write stalls could continue even on 1.4.2. Now they're finally fixed O:-) - Make monitor ignore statistics of stopped OSDs. Previously if you stopped all OSDs the last total I/O numbers would remain the same indefinitely	2024-02-09 00:29:31 +03:00
Vitaliy Filippov	8e25a28a08	Ignore down OSDs in monitor statistics aggregation	2024-02-09 00:22:36 +03:00
Vitaliy Filippov	5d3317e4f2	Followup to 1.4.2 write stall fix - sadly, the previous version was not working correctly :)	2024-02-08 19:34:29 +03:00
Vitaliy Filippov	016115c0d4	Release 1.4.2 ... - Log to systemd by default - Fix excessive autosyncs after every operation with disabled immediate_commit (introduced in 1.1.0) - Fix a possible write stall with EC due to the lack of OSD wakeup after stabilizing previous writes - Change sync operation semantics as a final fix to possible write stalls with EC and disabled immediate_commit - Sync after deleting data in CLI rm / rm-data if immediate_commit is disabled - Fix OSDs ignoring syncs & autosyncs for delete operations - Fix OSD space reporting sometimes adding garbage zeros for deleted inodes (causing extra pool/stats etcd keys for deleted pools) - Speed up monitor failover - change default etcd_mon_ttl from 30 to 5 seconds - Speed up operation retries - change default up_wait_retry_interval to 50 ms - Add patch for libvirt 9.10	2024-02-04 02:23:49 +03:00
Vitaliy Filippov	e026de95d5	Log to systemd by default