87 Releases 87 Tags

RSS Feed

• v2.4.0 59f87c3e30

  Compare

  Release 2.4.0 Stable

  vitalif released this 2025-09-24 15:40:41 +03:00 | 2 commits to master since this release

  New features:

  • Support UBLK in CSI and make it the default
  • Add image tree output: vitastor-cli ls --tree

  Bug fixes:

  • Fix OSDs crashing on localized reads from snapshotted images
  • Several NFS fixes from github PR #95
  • Fix snapshotted bitmap reads not working in EC pools (fix #92)
  • Fix UBLK reporting incorrect device size (1/8 of actual size), report logical_block_size=4096
  • Do not repeer when stopping PEER_CONNECTING osd_clients (increase PG peering stability on failover)
  • Fix marking PGs as incomplete when peers are unavailable
  • Fix FS formatter (mkfs and fsck) error handling in CSI (it didn't report mounting errors)
  • Fix removal of block PVC devices not working in CSI
  • Do not attempt to free outbound ops in clear_immediate_ops (fix #88)
  • Add vitastor-disk prepare --dry-run option
  • Add librdmacm-dev to build-deps
  • Log has_invalid objects
  • Do not warn on incomplete+has_invalid PG states as unexpected
  • Fix OSD reweight values between 0 and 1 not working in monitor
  • Fix OSD assertion failed: n_subops != sent when all object copies are corrupted
  • Install ibverbs-providers in vitastor Docker builds
  • Fix #86 - base64_decode on arm64 O_o
  • Fix Proxmox 9.0 support (remove some kind of a whitelist added in upstream)

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)
• v2.3.0 0c89886374

  Compare

  Release 2.3.0 Stable

  vitalif released this 2025-08-25 21:39:19 +03:00 | 34 commits to master since this release

  New features:

  • Add a new kernel device mounting method: ublk.
    It's the fastest method for random IOPS, and it's on par with VDUSE for linear MB/s.
  • Disable io_uring waits being reported as iowait on kernels which support it (6.15+)
  • Allow to enforce permissions at the VitastorFS NFS server side
  • Add qemu_file_mirror_path option to the config to allow to trick Veeam and make it work
  • Speed up CRC32C calculation in OSD by fixing a bug and enabling AVX512 version
  • Support QEMU 10
  • Support Debian 13 Trixie and Proxmox 9.0
  • Remove the dependency on system liburing in package builds (build it statically)

  Bug fixes:

  • Fix checksums NEVER BEING ENABLED in vitastor-disk prepare, even when explicitly requested :-D
  • Use default uid and gid from NFS AUTH_SYS when creating files
  • Fix object bitmaps supposedly & possibly being corrupted in some rare cases with EC N+2+
  • Avoid multiple inflight overwrites to meta blocks - fixes possible data corruption with
    one specific SSD model: Memblaze PBlaze5 910 (github #79)
  • Fix a bug in antietcd which was leading to leases sometimes not expiring correctly
  • Fix a bug in NFS where ".." entry had its cookie equal to 0 instead of 1 (github #78)
  • Fix snapshots not being deleted during VM deletion in Proxmox plugin (github #85)
  • Fix monitor not filtering OSDs by block size correctly

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)
• v2.2.3 b529f77264

  Compare

  Release 2.2.3 Stable

  vitalif released this 2025-07-30 10:50:02 +03:00 | 69 commits to master since this release

  • Support Ubuntu 24.04 Noble
  • Fix clients hanging on online PG count change with in-flight operations
  • Fix volume_size_info in PVE VitastorPlugin
  • Implement bdrv_refresh_filename for the QEMU driver
  • Fix RDMA-CM broken in 2.2.0
  • Fix possible "invalid %N$ use detected" OSD crashes
  • Fix vitastor-nfs build without RDMA
  • Fix docker build by adding the forgotten apt/preferences
  • Fix libfio_blockstore.so
  • Fix "trigger event loop automatically" API version check in the QEMU driver
  • Add Content-Type header for prometheus metrics
  • Add a patch for libvirt 11.5

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)
• v2.2.2 97bb809b54

  Compare

  Release 2.2.2 Stable

  vitalif released this 2025-06-07 12:56:48 +03:00 | 92 commits to master since this release

  • Fix a bug introduced in 2.2.0 - pg_locks weren't disabled for pools without local_reads
    correctly which could lead to inactive pools during various operations
  • Fix an old bug where OSDs could send sub-operations to incorrect peer OSDs when their
    connections were stopped and reestablished quickly, in 2.2.0 it was usually leading
    to "sequencing broken" messages in OSD logs
  • Fix debug use_sync_send_recv mode

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)
• v2.2.1 957a4fce7e

  Compare

  Release 2.2.1 Stable

  vitalif released this 2025-05-25 01:04:48 +03:00 | 102 commits to master since this release

  • Fix vitastor-disk purge broken after adding the "OSD is still running" check
  • Fix iothreads hanging after adding zero-copy send support
  • Fix enabling localized reads online (without restarting OSDs) in the default PG lock mode

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)
• v2.2.0 95ec750b8c

  Compare

  Release 2.2.0 Stable

  vitalif released this 2025-05-11 00:26:08 +03:00 | 113 commits to master since this release

  New features:

  • Localized read support for multi-datacenter setups.
  • io_uring-based zero-copy send support - read the instruction carefully for optimal performance!
  • Improve and speedup data distribution, especially in cases of very large hosts (100 OSD+). Previously, PG optimization speed depended on the number of OSDs, now it only depends on the number of failure domains. Distribution over specific OSDs is now also more even and becomes strictly more even when you increase the number of PGs.
  • Add a very interesting instruction to change NFS_MAX_FILE_IO_SIZE
  • Check operation sequencing and stop connections when it breaks - should help catch some very rare RDMA packet loss problems.
  • vitastor-cli rm-osd now refuses to remove OSDs which are still up and suggests to use vitastor-disk purge.
  • Allow removal of direntries referring non-existent inodes in VitastorFS.
  • Change default vitastor-etcd data dir to /var/lib/etcd/vitastor.

  Bug fixes:

  • Fix compatibility with ISA-L 2.31+. ⚠️Very important: please upgrade Vitastor before upgrading ISA-L to 2.31+.
  • Fix in-memory state cleanup for incomplete PGs.
  • Fix monitor crash with non-existent node_placement nodes.
  • Slightly speedup vitastor-kv dump command by adding output buffering.
  • Fix theoretically possible slowdowns in OSD sub-operation failure handling code.
  • Fix very rare stack overflows in vitastor-kv.
  • Fix a possible crash in VitastorFS during handling of file creation race condition.
  • Fix modify-pool -s PG_SIZE which didn't work without --pg_minsize.
  • Fix marking peer OSDs as alive on receiving data from them via RDMA - in theory,
    the bug could result in instability with RDMA under high load with slow disks.
  • Fix a rare OSD crash due to double handle_primary_subop() call.
  • Fix latency aggregation in global stats (/vitastor/stats in etcd) - do not sum it.
  • Hide "Ran out of journal space" log messages by default.
  • Wait for RDMA-CM EVENT_ESTABLISHED after rdma_accept(), handle rdma_accept() before acking the event.
  • Fix VitastorFS total & free numbers multiplied by extra 2.
  • Fix systemd unit name in make-etcd.
  • Do not allow reweight > 1 in vitastor-cli modify-osd.
  • Fix docker build.

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)
• v2.1.0 891b2811c7

  Compare

  Release 2.1.0 Stable

  vitalif released this 2025-04-01 20:16:27 +03:00 | 151 commits to master since this release

  New features:

  • Support separate OSD cluster network - osd_cluster_network
    and, in general, multiple OSD networks, including RDMA
  • Add an alternative RDMA implementation via RDMA-CM - use_rdmacm,
    required for iWARP and, maybe, for some IB setups (but not for RoCE)
  • Change default PG behaviour to wait for all "up" OSDs to be connected before starting it.
    The old behaviour may be returned by enabling a new allow_net_split
    option.
  • Add a patch for QEMU 9.2

  Bug fixes:

  • Fix incorrect "has_xxx" PG state names in ls-pgs
  • Fix possible QEMU crashes after detaching of Vitastor disks (and update all QEMU builds in Vitastor repos)
  • Fix clients sometimes spamming OSDs with infinite reconnections when some PGs are offline
  • Fall back to TCP on RDMA connection failures
  • Add missing logging of RDMA ibv_modify_qp() errors
  • Add a minimum interval for etcd_state_client to reload state

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)
• v2.0.0 48c026bfa0

  Compare

  Release 2.0.0 Stable

  vitalif released this 2025-03-16 14:34:31 +03:00 | 176 commits to master since this release

  No breaking features, it's 2.0.0 just because it includes S3 and because there are already too many 1.x releases :).

  New features:

  • S3 is finally available: https://vitastor.io/docs/installation/s3.html
  • node.js addon is now packaged as a Debian package
  • Support listing PGs by OSDs in vitastor-cli ls-pgs
  • Implement offline TRIM support: vitastor-disk trim, discard_on_start
  • Change used_for_fs pool option to used_for_app

  Bug fixes:

  • Fix several bugs in the node.js addon (a memory leak, an incorrectly triggered event loop)
  • Fix a client crash (vitastor-cli rm) during deletion when writeback is enabled
  • Fix PG object count statistics on deletion of non-existing objects
  • Fix vitastor-nbd crash when mapping by ID instead of inode name
  • Fix a client memory leak with enabled immediate_commit and write-back cache
  • Add seccomp=unconfined for vitastor docker OSDs to not break io_uring
  • Add udev and systemd to vitastor docker image
  • Fix upgrading from pre-0.7.1 (very old) systemd units O_o
  • Fix total object count calculation in rm_data

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)
• v1.11.0 69cbe7bbb2

  Compare

  Release 1.11.0 Stable

  vitalif released this 2025-03-01 13:39:42 +03:00 | 201 commits to master since this release

  New features:

  • Support containerized Vitastor installations: https://vitastor.io/docs/installation/docker.html
  • Add new functions to the node.js binding: delete(), get_immediate_commit(), on_ready(),
    get_min_io_size(), get_max_atomic_write_size()
  • S3 (Zenko Cloudserver with Vitastor support) is coming shortly and will be released separately

  Bug fixes:

  • Use IP-derived etcd node names in make-etcd
  • Set short name of the OSD process to display in top
  • Fix snap-create without pool_id failing when there are multiple pools
  • Several bugs are fixed in the write-back cache, it should now be stable:
    • Fix incorrect snapshot reads from dirty write-back cache
    • Do not try to repeat pending writebacks on OSD reconnections
    • Fix client hangs with multiple SYNCs in the writeback queue
    • Fix client hangs do to incorrect calculation of the writeback queue size
  • Several improvements for NBD mapping/unmapping:
    • Add a workaround for race condition in the Linux kernel NBD driver leading
      to vitastor-nbd sometimes breaking a previously mapped device instead of
      setting up a new one
    • Check if the device is actually mapped in vitastor-nbd unmap
    • Fix device name/number validation in vitastor-nbd
  • Fix OSD crashes after starting with corrupted metadata - from now it will skip
    corrupted metadata entries and heal itself
  • Fix scrubbing of misplaced objects and object state recalculation after
    vitastor-cli fix - previously, an OSD restart could be required to fix object states
  • Make primary OSD distribution more stable by using murmur3 hash instead of the old pseudo-rng
  • Fix monitor sometimes racing with itself - do not touch /pool/stats from stats
    aggregation if PG recheck is active
  • Sort vitastor-cli ls output by name by default
  • Update antietcd to 1.1.2

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)
• v1.10.1 61ededa230

  Compare

  Release 1.10.1 Stable

  vitalif released this 2025-01-03 16:22:09 +03:00 | 237 commits to master since this release

  New features:

  • Add "deleted" image flag which is set when vitastor-cli rm starts to delete an image,
    but can't delete it fully due to inactive PGs or stopped OSDs
  • Support JSON output in vitastor-disk prepare and purge
  • Show backfillfull pools in vitastor-cli status
  • Make object listings consistent (used in vitastor-cli rm/rm-data/merge/etc).
    This means that there is now a guarantee that if a data block is present when you invoke rm,
    rm will attempt to delete it, even if rm is invoked when the PG switches state. Previously in
    such cases rm could skip and leave some objects behind as garbage, and merge probably could
    incorrectly move data between snapshots.
  • Make deletions (rm/rm-data) consistent. This means that rm/rm-data will either complete
    successfully and delete all requested image data or complete with an error if some objects
    could not be deleted or if there is a possibility that some data is left on stopped OSDs.
    Previously, when some PGs or OSDs were inactive at the moment of deletion, rm-data was
    behaving incorrectly: it wasn't retrying deletions failed due to dropped OSD connections,
    it could hang waiting for PGs to activate, and it could return with a successful error
    code while some garbage was still possibly left on some OSDs. Deletions are not fully atomic
    cluster-wide yet, which means that you still have to repeat the deletion request after you
    return stopped OSDs back, but now you always know for sure if you have to repeat it.

  Bug fixes:

  • Fix vitastor-cli rm --exact / --matching command not working
  • Finally fix "Unexpected status" in the Proxmox plugin
  • Fix vitastor-cli create-snap incorrectly linking multiple snapshots in a different pool
  • Fix incomplete image parent_id loop check in OSD
  • Fix reads from snapshots in a different pool not working if there are more than 2 snapshots
  • Fix append of VITASTOR_CONF to cmdline in the opennebula prebackup script
  • Fix OSDs crashing again when the cluster is full with EC (was meant to work since 1.6.0 but didn't)
  • Improve logging of subop failures

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)