antilles
/
vitastor
forked from vitalif/vitastor
1
0
Fork 0
Code Pull Requests Activity
1,605 Commits
44 Branches
57 Tags
3.7 MiB
Commit Graph

1605 Commits (d2b43cb118440841b72e94a5c4bfda262a33eebc)

Author SHA1 Message Date
Vitaliy Filippov 2e592a2f22 Fix undefined variable "timeout" 2023-10-29 01:30:55 +03:00
Vitaliy Filippov b92f644e3a Fix statistics aggregation, calculate inode stats by first deriving per-OSD stats, too 2023-10-29 01:30:55 +03:00
Vitaliy Filippov 890ea3dbc0 Forgot to add new parameter page to README 2023-10-28 13:39:53 +03:00
Vitaliy Filippov 06630369bf Plans++ 2023-10-28 13:38:04 +03:00
Vitaliy Filippov b4740acf62 Fix operations paused for 0.5-1 second when it happens that io_uring submit is not triggered 2023-10-28 13:18:21 +03:00
Vitaliy Filippov eae81bbda6 Fix typo 2023-10-28 01:09:20 +03:00
Vitaliy Filippov 8222e3c77d Release 1.1.0 
New features:

- Implement [client writeback cache](docs/config/client.en.md#client_enable_writeback)
- Add the third I/O mode: [O_DIRECT|O_SYNC](docs/config/osd.en.md#data_io) (good for Optane)
- Reduce load on etcd by splitting OSD lease and statistics reporting intervals:
  [etcd_stats_interval](docs/config/osd.en.md#etcd_stats_interval) (default 30 sec)
- Make MON automatically filter OSDs by layout (block_size/immediate_commit/bitmap_granularity)
  to prevent "refusing to start PGs of this pool" errors on misconfiguration
- Support running fio benchmarks on systems without io_uring
- Make QEMU driver compatible with QEMU 8.1
- Document usage of [vhost-user-blk](docs/usage/qemu.en.md#vhost-user-blk)

Bug fixes:

- Fix resizing disks in QEMU driver (for example, in Proxmox)
- Fix "unexpected result" in Proxmox driver by making CLI flush output on exit
- Remove unneeded block_size mismatch warnings on pools without matching PGs
- Fix possible segfault in vitastor-cli ls -l (usually with deleted pools)
- Fix QEMU driver compatibility with systems without io_uring
- Fix monitor eating 100% CPU when etcd is down (caused by infinite retries)
- Fix potential incorrect write processing with snapshots (not caught in tests
  but could probably lead to client hangs)
- Fix buffer insertion in cluster_client (not caught in tests but could
  probably lead to incorrect writes in rare cases)
- Fix rare OSD crash during sync operation processing
- Fix a reenterability issue in cluster_client not reproducible in QEMU/fio,
  but reproducible with the currently developed K/V database implementation
- Fix deletion of the first modified object - OSDs could crash if you modified
  the same object a lot of times, then deleted it, and then modified it again
- Fix the fio_sec_osd test tool
 2023-10-28 00:33:06 +03:00
Vitaliy Filippov 29cbe70e74 Bump qemu version to vitastor4 2023-10-28 00:33:06 +03:00
Vitaliy Filippov a883e79507 Make docs to add etcd_stats_interval 2023-10-27 14:09:26 +03:00
Vitaliy Filippov be7e76f849 Split etcd_stats_interval out of etcd_report_interval 2023-10-27 01:26:26 +03:00
Vitaliy Filippov 6fd2cf5df6 Add documentation for the write-back cache 2023-10-27 01:26:26 +03:00
Vitaliy Filippov 294a754c9e Allow write-back by default in NBD & NFS 2023-10-27 01:26:26 +03:00
Vitaliy Filippov 8bfea6e7de Support vitastor_c_create_epoll() in fio driver 2023-10-26 22:57:36 +03:00
Vitaliy Filippov bac9e34836 Allow to create vitastor_c with plain epoll without uring :-) 2023-10-26 22:57:36 +03:00
Vitaliy Filippov 8aa4d492c1 Allow to use epoll_manager without ringloop 2023-10-26 22:57:36 +03:00
Vitaliy Filippov 9336ee5476 Correctly free manual "small vector" in cluster_client %-) 2023-10-26 22:57:36 +03:00
Vitaliy Filippov ad30b11519 Add the missing ringloop creation check to vitastor_c_create_uring_json() 2023-10-26 18:07:23 +03:00
Vitaliy Filippov a061246997 Do not attempt to initialize QEMU driver via vitastor_c_create_qemu_uring() 
It doesn't add any compatibility because vitastor_c_uring_register_eventfd()
is added in the same VITASTOR_C_API_VERSION 2.
 2023-10-26 17:46:19 +03:00
Vitaliy Filippov 5066e35a49 Fix write-over-delete failing for the very first entry in dirty_db 2023-10-21 17:00:14 +03:00
Vitaliy Filippov 93dc31f3fc Fix possible segfault in vitastor-cli ls -l 2023-10-18 11:11:41 +03:00
Vitaliy Filippov f245b56176 Fix another possible reenterability issue in cluster_client 
Non-reproducible in QEMU/FIO, only caught during K/V DB debugging
 2023-10-08 11:02:53 +03:00
Vitaliy Filippov befca06f18 Support any OSD count in test_heal 2023-10-08 11:02:53 +03:00
Vitaliy Filippov fbf0263625 Add qemu-storage-daemon to documentation 2023-09-16 18:40:52 +03:00
Vitaliy Filippov 3bcf276d4d Run tests with writeback 2023-09-16 17:52:17 +03:00
Vitaliy Filippov 38db53f5ee Implement client writeback cache 
- Disabled by default, enable with client_enable_writeback=true
- Even then only enabled in FIO when -direct is disabled and in QEMU when
  block device cache is enabled in settings
- Can also be enabled in other clients like vitastor-cli using parameter
  client_writeback_allowed=true, but not recommended
 2023-09-16 17:52:17 +03:00
Vitaliy Filippov cd543a90bc Prevent stack overflows in cli_merge with CAS and writeback cache 2023-09-16 17:52:17 +03:00
Vitaliy Filippov f600cc07b0 Autosync in blockstore every autosync_writes, too 2023-09-16 17:52:17 +03:00
Vitaliy Filippov 6a8e530e6b Add FIXME to timerfd_manager 2023-09-16 17:52:17 +03:00
Vitaliy Filippov 5cadb170b9 Fix possible OSD crash during sync due to missing min_flushed_journal_sector reset 2023-09-16 17:52:17 +03:00
Vitaliy Filippov e72d4ed1d4 Remove unused bs_sync fields 2023-09-16 17:52:17 +03:00
Vitaliy Filippov ff479a102d Make MON filter OSDs by block layout to prevent "refusing to start PGs of this pool" errors on misconfiguration 2023-09-16 17:52:17 +03:00
Vitaliy Filippov 27d0d5b06a Reads do not have to wait for buffer flushes anymore 2023-09-16 17:52:17 +03:00
Vitaliy Filippov 33950c1ec8 Fix fio_sec_osd attr_len 2023-09-16 17:49:10 +03:00
Vitaliy Filippov eea7ef1f19 Remove debug osd_trace from test_write 2023-09-12 01:35:36 +03:00
Vitaliy Filippov cc0fdc6253 Remove erroneous block_size mismatch warnings on pools without matching PGs 2023-09-08 23:19:04 +03:00
Vitaliy Filippov 79ecd59b10 Flush STDOUT and STDERR before exiting from cli to fix Proxmox "Unexpected result" 2023-09-07 17:30:26 +03:00
Vitaliy Filippov 51081c9b45 Put etcd into tmpfs for tests 2023-09-07 02:35:09 +03:00
Vitaliy Filippov b7d398be5b Fix sscanf validation usage (field count instead of null_byte == 0) 2023-09-07 02:34:35 +03:00
Vitaliy Filippov 85e9f67d9d Add supported_truncate_flags 2023-09-06 17:37:52 +03:00
Vitaliy Filippov 79c6d6f323 Make QEMU driver compatible with QEMU 8.1 2023-08-24 02:23:55 +03:00
Vitaliy Filippov ae760dbc1d Fix co_truncate size division by BDRV_SECTOR_SIZE 2023-08-24 01:55:35 +03:00
Vitaliy Filippov 65487da4b1 Do not include msgr_rdma.h into messenger.h 2023-08-24 01:55:35 +03:00
Vitaliy Filippov 7862282938 Extract validation to check_rw(), remove duplicate code with OP_SYNC 2023-08-13 23:49:52 +03:00
Vitaliy Filippov 30ce2bd951 Fix buffer insert in cluster_client 2023-08-12 11:08:50 +03:00
Vitaliy Filippov b1a0afd10a Aggregate buffer flushes 2023-08-11 11:26:13 +03:00
Vitaliy Filippov 85b6134910 Return dirty buffers on read in client 
Required at least to return buffers when they need to be replayed, but until
they are actually replayed
 2023-08-09 00:57:08 +03:00
Vitaliy Filippov b1b07a393d Fix incorrect marking op parts as done with snapshots (could probably lead to client hangs) 2023-08-09 00:57:08 +03:00
Vitaliy Filippov 7333022adf Add a third I/O mode: O_DIRECT|O_SYNC, change parameters to data_io/meta_io/journal_io 2023-08-09 00:57:08 +03:00
Vitaliy Filippov ab8627c9fa Fix monitor retrying failed etcd connection in an infinite loop without pauses 2023-08-09 00:57:08 +03:00
Vitaliy Filippov 6acf562e01 Release 1.0.0 
New features:

- Data and metadata checksums!
  - Metadata checksums are always used with new disk format
  - Data checksums can be turned on with --data_csum_type crc32c for new OSDs
  - Checksum block size can be configured
  - inmemory_metadata now also affects keeping checksums in memory
- Linux page cache I/O caching support which can be enabled separately for
  data, metadata (including checksums) and journal (O_SYNC instead of O_DIRECT)
- Details [here](https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/docs/config/layout-osd.en.md#data_csum_type)
- Backwards compatibility is preserved, you can use new OSDs with old disks

Release also includes bug fixes from [0.9.6](https://git.yourcmc.ru/vitalif/vitastor/releases/tag/v0.9.6).

0.9.6 is moved to "-oldstable" repositories and will be available for some additional time.
 2023-07-29 18:57:19 +03:00