vitalif
/
vitastor
5
4
Fork 9
Code Issues 1 Pull Requests 2 Actions Releases 69 Wiki Activity
1,191 Commits
44 Branches
69 Tags
14 MiB
Commit Graph

451 Commits (lrc-matrix)

Author SHA1 Message Date
Vitaliy Filippov 238037ae31 Make journal trimmer wait until reads are completed when inmemory_journal is false 
Without this new writes may in theory overwrite journal data being read at that time
 2022-11-20 01:49:21 +03:00
Vitaliy Filippov 09a8864686 Fix degraded parity writes for EC N+K when K>1 
Fixes possible `calc_rmw_parity_ec(): Assertion `bufs[i][curbuf[i]].buf' failed` error
 2022-11-20 00:50:13 +03:00
Vitaliy Filippov 6e6f6ecbb0 Add missing journal space check for big_writes 
Fixes possible `prefill_single_journal_entry(): Assertion `!journal.sector_info[journal.cur_sector].flush_count' failed` error
 2022-11-20 00:50:13 +03:00
Vitaliy Filippov bf8a0581cd Fix possible "assertion failed: next->prev_wait >= 0" in client 2022-11-20 00:50:13 +03:00
Vitaliy Filippov 5953942042 Add crc32c test utility 2022-11-20 00:50:13 +03:00
Vitaliy Filippov a276a1f737 Do not copy journal data additional time when flushing 2022-11-20 00:50:13 +03:00
Vitaliy Filippov cc24e5796e Add a FIXME 2022-11-20 00:50:09 +03:00
Vitaliy Filippov 6e26732e6a Fix skipped "len" field in vitastor-disk write-journal big_writes 2022-11-12 12:01:40 +03:00
Vitaliy Filippov b4edc79449 Fix possible segfault on ENOSPC 2022-11-12 11:59:43 +03:00
Vitaliy Filippov 11ec9ad874 Release 0.8.0 
- Implement automatic OSD activation via udev and simple on-disk superblock storage
- Add a new `vitastor-disk` tool and merge all disk-related functionality there.
  Now it can prepare new OSD disks, upgrade plain old systemd units to the new scheme,
  resize OSD data area, manage OSD services by disk paths, manage superblocks,
  automatically check and disable disk cache, dump and write back journal and metadata.
- Add a documentation section about `vitastor-disk` (read it if you want details!)
- Install systemd services during package installation instead of the older method
  of manually creating them via separate shell scripts
- Add a new `make-etcd` script that reuses /etc/vitastor/vitastor.conf to configure etcd
- Allow to configure block_size, bitmap_granularity and immediate_commit per-pool
- Fix "fatal error: tried to overwrite non-zero metadata entry" which was possible
  in some cases after unclean OSD shutdown (caused by old metadata entries not being zeroed)
 2022-09-05 13:51:20 +03:00
Vitaliy Filippov 83bb6598dc Fix fsync autodetection for the single-device mode 2022-09-05 13:51:20 +03:00
Vitaliy Filippov 150f369346 Hotfixes for vitastor-disk prepare: max_other, get device size, older sfdisk 2022-09-05 12:48:27 +03:00
Vitaliy Filippov 8d9a5fde15 Fix docs (block_size vs object_size) 2022-09-04 14:47:04 +03:00
Vitaliy Filippov 9ccc607ab9 Fix parse_size 2022-09-04 14:20:56 +03:00
Vitaliy Filippov 9481456dfe Automatically check whether to disable cache during prepare 2022-09-03 02:04:21 +03:00
Vitaliy Filippov 68ebe5993a Fix partition reuse 2022-09-02 23:32:25 +03:00
Vitaliy Filippov a537db8909 Add documentation for the new "vitastor-disk" tool 2022-08-22 00:31:30 +03:00
Vitaliy Filippov 54ef2c389f Followup to the "tried to overwrite" fix: also handle it in case of inmemory_meta == false 2022-08-21 01:28:29 +03:00
Vitaliy Filippov 153c73574a Refactor blockstore_init_meta into slightly more obvious code 2022-08-21 01:21:13 +03:00
Vitaliy Filippov d83580bd68 Fix "tried to overwrite non-zero metadata entry" when during a previous metadata 
flush writing new entry is completed, but zeroing out an old one isn't
 2022-08-21 00:31:18 +03:00
Vitaliy Filippov 29b40aba93 Add write-meta command (for debug) 2022-08-20 23:56:57 +03:00
Vitaliy Filippov a52f2b0e8f Add write-journal command (for debug) 2022-08-20 14:05:53 +03:00
Vitaliy Filippov 1407db9c08 Fix vitastor-disk prepare bugs 2022-08-19 02:22:54 +03:00
Vitaliy Filippov c0d5e83fb8 Run partprobe when partitions do not appear 2022-08-18 02:05:16 +03:00
Vitaliy Filippov 40d8d65188 Rewrite upgrade-simple to C++ 2022-08-18 01:31:31 +03:00
Vitaliy Filippov a16263e88c Fix bugs in the upgrade script and in the udev startup script 2022-08-17 10:28:34 +03:00
Vitaliy Filippov cb4e3a118d Fix warning 2022-08-15 00:18:21 +03:00
Vitaliy Filippov b1e39b5dea Split disk_tool.cpp into separate files 2022-08-14 02:37:01 +03:00
Vitaliy Filippov 1170319431 Finish vitastor-disk prepare in theory 2022-08-14 02:13:24 +03:00
Vitaliy Filippov 2e0a2221eb vitastor-disk prepare: WIP second form command of the command 2022-08-12 01:58:28 +03:00
Vitaliy Filippov 5a10d135f3 Allow to configure block_size, bitmap_granularity and immediate_commit per-pool 2022-08-11 01:56:33 +03:00
Vitaliy Filippov 4c9aaa8a86 vitastor-disk prepare: implement first form of the command 2022-08-09 01:29:29 +03:00
Vitaliy Filippov ae99ee6266 Rename base64.{cpp.h} to str_util 2022-07-31 01:12:37 +03:00
Vitaliy Filippov 5af75f7d78 Implement vitastor-cli and vitastor-disk --help <command> 2022-07-31 01:10:05 +03:00
Vitaliy Filippov 7dc6f10ea1 Add read-sb command 2022-07-28 00:14:23 +03:00
Vitaliy Filippov 76dd0fdcea Implement pre-exec command with on-start OSD checks 2022-07-24 15:09:45 +03:00
Vitaliy Filippov 5acc19bbd5 Implement systemctl start/stop and other commands 2022-07-23 02:18:40 +03:00
Vitaliy Filippov d5ca4e1f90 Add exec-osd command 2022-07-22 02:17:24 +03:00
Vitaliy Filippov 67e04f789f Add write-sb (superblock) command 2022-07-19 01:14:31 +03:00
Vitaliy Filippov 837407a84c Add udev import command 2022-07-19 01:14:31 +03:00
Vitaliy Filippov 1fe5908899 WIP OSD activation from superblock 2022-07-17 02:14:50 +03:00
Vitaliy Filippov dcc6d546be Move simple-offsets into vitastor-disk, too 2022-07-15 02:19:35 +03:00
Vitaliy Filippov 85fa389557 Add a test for disk-tool resize 2022-07-15 01:38:30 +03:00
Vitaliy Filippov dfa433c63b Add JSON format to dump-journal 2022-07-15 01:38:30 +03:00
Vitaliy Filippov cf487c95aa Fix resizer 2022-07-15 01:38:30 +03:00
Vitaliy Filippov b10656ca09 Parse new disk params in disk_tool resizer 2022-07-15 01:38:30 +03:00
Vitaliy Filippov ea632367e9 Do not alter dsk.meta_offset/len to skip superblock 2022-07-15 01:38:30 +03:00
Vitaliy Filippov 4d777c6729 Set journal/meta devices to data device explicitly instead of "" 2022-07-15 01:38:30 +03:00
Vitaliy Filippov 0c404c5074 Use blockstore_disk in disk_tool 2022-07-15 01:38:30 +03:00
Vitaliy Filippov dfd80626bd Extract disk opening functions to separate module 2022-07-15 01:38:30 +03:00
Vitaliy Filippov 30907852c2 Use simple std::map for the config 2022-07-15 01:38:30 +03:00
Vitaliy Filippov 078ed5b116 WIP Data area resize tool 2022-07-15 01:38:30 +03:00
Vitaliy Filippov 73a363bf92 Rename some variables and constants 2022-07-15 01:38:30 +03:00
Vitaliy Filippov b0e86ca643 Merge dump-journal and dump-meta into the new "vitastor-disk" tool 2022-07-15 01:38:30 +03:00
Vitaliy Filippov 8800afb649 Fix void* arithmetic again 2022-07-15 01:38:30 +03:00
Vitaliy Filippov e20cdd13b6 Fix simple-offsets return value 2022-07-15 01:38:30 +03:00
Vitaliy Filippov bce357e2a5 Do not read all metadata into memory when dumping 2022-06-13 01:26:30 +03:00
Vitaliy Filippov 0876ca09cd Fix dumper includes and print format 2022-06-11 00:30:44 +03:00
Vitaliy Filippov dac12d8a4c Implement metadata dump tool 2022-06-10 18:50:09 +03:00
Vitaliy Filippov 1eec4407ab Fix inode creation when /index/maxid is out of sync 2022-06-06 16:35:51 +03:00
Vitaliy Filippov 675bc12a13 Add extern "C" for systems like Gentoo which miss it in jerasure includes 2022-06-05 00:33:38 +03:00
Vitaliy Filippov 101592bbff Release 0.7.1 
- Add ISA-L erasure code implementation, now used automatically instead of jerasure when available
- Fix listings sending too many parallel requests to OSDs
- Fix rm-data crashing with --wait-list
- Remove empty inodes from statistics and `ls` output, after <inode_vanish_time> seconds after deletion
- Make monitor delete pool statistics when the pool is deleted and thus remove them from `df` output
- Log multiple etcd addresses in OSD logs correctly
- Fix true/false parsing in json configs like no_recovery/no_rebalance
- Show no_recovery, no_rebalance, readonly flags in status
 2022-06-05 00:07:24 +03:00
Vitaliy Filippov 87613ed590 Add ISA-L into RPM specs 2022-06-04 13:27:06 +03:00
Vitaliy Filippov 2a2e914ef9 Show no_recovery, no_rebalance and readonly flags in status 2022-06-04 13:27:06 +03:00
Vitaliy Filippov 0cdc9292c8 Fix true/false parsing in json configs like no_recovery/no_rebalance 2022-06-04 13:27:06 +03:00
Vitaliy Filippov 3e1b03bb5c Show all etcd addresses in the "reporting to..." message 2022-06-04 13:27:06 +03:00
Vitaliy Filippov 1efbbb0c36 Make deleted inodes vanish from statistics after 60 seconds 2022-06-04 13:27:06 +03:00
Vitaliy Filippov 088dd15449 Exclude empty inodes from stats 2022-06-04 00:18:17 +03:00
Vitaliy Filippov 4a531d7b8b Fix listings sending too many parallel requests to OSDs, fix rm-data crashing with --wait-list 2022-06-03 23:36:37 +03:00
Vitaliy Filippov a0cae4c180 Rename "jerasure" to "ec" in pool configuration, function names, fix documentation and Debian build scripts 
Old pool configurations with "jerasure" also remain supported as an alias for "ec"
 2022-06-03 15:40:00 +03:00
Vitaliy Filippov 21b306e25f Add ISA-L support 2022-06-02 01:47:33 +03:00
Vitaliy Filippov d8313e939a Release 0.7.0 
- Add documentation! :-) in Russian and English
- Implement an NFS proxy for file-based access emulation to Vitastor
  images for non-QEMU based hypervisors like VMWare, as a better way
  than iSCSI
- Implement "primary affinity tags"
- Add a patch for libvirt 6.0
- Fix free_down_raw in cli status
- Fix a rare bug where OSDs could drop unrelated connections on errors
 2022-05-29 23:39:53 +03:00
Vitaliy Filippov 82b9f4c52d Add a test with OSD kills 2022-05-28 00:51:14 +03:00
Vitaliy Filippov 2bdf415eb3 Fix unknown OSD numbers on error 2022-05-28 00:51:14 +03:00
Vitaliy Filippov 5d47bbe04c Add documentation 2022-05-17 01:10:49 +03:00
Vitaliy Filippov 93a9f1ef89 Fix NFS socket read hangs 2022-05-11 21:06:56 +03:00
Vitaliy Filippov 2697aae909 Fix free_down_raw in cli status 2022-05-11 18:08:45 +03:00
Vitaliy Filippov 6b69db73ac Remove getrandom() usage 2022-05-11 11:25:20 +03:00
Vitaliy Filippov d48a824846 Fix some warnings 2022-05-10 12:42:58 +03:00
Vitaliy Filippov 40985282ff Fix build under GCC 8 2022-05-10 12:26:47 +03:00
Vitaliy Filippov acf403e886 Add install target for NFS proxy 2022-05-10 10:43:17 +03:00
Vitaliy Filippov 7c2379d458 Simplified NFS proxy based on own NFS/XDR implementation 2022-05-07 01:01:20 +03:00
Vitaliy Filippov a2189100dd Make CLI functions usable in library form 
Return results and errors in a variable instead of just printing them,
separate vitastor-cli main() from cli_tool_t, move positional argument
parsing to CLI main from command implementations.
 2022-05-06 02:18:32 +03:00
Vitaliy Filippov bb84379db6 Release 0.6.17 
- Fix incorrect reading of extra metadata block leading to extra unknown objects in stats
- Fix CSI driver volumeMode: Block support
- Add block PVC and pod examples
- Fix build under 32 bit architectures
- Fix slow connection ramp-up caused by up_wait_retry_interval
 2022-05-06 02:18:01 +03:00
Vitaliy Filippov 714dda8151 Fix slow connection ramp-up caused by up_wait_retry_interval pausing operations on first connection attempt 2022-05-06 02:12:08 +03:00
Vitaliy Filippov e718116f54 Fix incorrect reading of extra metadata block 2022-04-21 02:52:21 +03:00
Vitaliy Filippov caa2cc2e6c Fix 32bit build error 2022-04-16 01:48:24 +03:00
Vitaliy Filippov 842ba8b831 Use (uint64_t)1 instead of 1l / 1ul 2022-04-16 01:48:14 +03:00
Vitaliy Filippov 340a4b4f27 Release 0.6.16 
- Implement `vitastor-cli status` (print cluster status) command
- Add a new `make-osd-hybrid.js` script to quickly prepare a lot of hybrid (HDD+SSD) OSDs
- Implement snapshot deletion for Cinder driver (only works in a healthy cluster)
- Fix a huge :) bug causing reads to return all zeroes during rebalance. Add a test to prevent it in the future
- Disconnect NBD proxy correctly without leaving a zombie [vitastor-nbd] process in D state
- Fix a rare write hang appearing with small write throttling enabled
 2022-04-09 01:16:52 +03:00
Vitaliy Filippov d71cc174e3 Implement CLI status command 2022-04-09 00:25:51 +03:00
Vitaliy Filippov 83146fa3e2 Fix the same HUGE bug for regular reads during rebalance 2022-04-08 11:50:09 +03:00
Vitaliy Filippov cd18ef7323 Disconnect NBD proxy correctly without leaving a zombie [vitastor-nbd] process in D state 2022-04-07 16:03:35 +03:00
Vitaliy Filippov 39531ef1a6 Fix incorrect chained reads during rebalance (the bug detected by test_rebalance_verify.sh) 2022-04-07 15:56:58 +03:00
Vitaliy Filippov c373425562 Fix nbd log 2022-04-07 15:55:38 +03:00
Vitaliy Filippov 9c30df83e3 Fix a HUGE :) bug in NBD proxy 
The bug could result in corrupted data on large writes
 2022-04-03 10:42:06 +03:00
Vitaliy Filippov 4100d829c7 Allow to override log file for daemonized NBD proxy 2022-04-03 02:41:04 +03:00
Vitaliy Filippov 79ebda933e Fix a write hang with throttling due to timer reenterability / triggerability 2022-03-28 01:42:06 +03:00
Vitaliy Filippov 85298ddae2 Release 0.6.15 
- Make peering much faster in medium to large clusters
- Fix a reenterability issue which could rarely lead to peering process hangs
 2022-03-06 19:16:34 +03:00
Vitaliy Filippov e23296a327 Rename cli_rm -> cli_rm_data, cli_snap_rm -> cli_rm 2022-02-24 14:34:14 +03:00
Vitaliy Filippov 839ec9e6e0 Shard clean_db by PGs to speedup listings 2022-02-20 00:21:24 +03:00