shaba
/
vitastor
forked from vitalif/vitastor
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
767 Commits
12 Branches
0 Tags
13 MiB
Commit Graph

767 Commits (master)

Author SHA1 Message Date
Vitaliy Filippov cb282d25e0 Release 0.6.5 
- Basic support for OpenStack: Cinder driver, patches for Nova and libvirt
- Add missing "image" and "config_path" QEMU options
- Calculate aggregate per-pool statistics in monitor
- Implement writes with Check-And-Set semantics
- Add a C wrapper library with public header
 2021-07-10 11:01:21 +03:00
Vitaliy Filippov 8b2a4c9539 Fix centos builds (yum-builddep stopped working in el7, cmake in el8..) 2021-07-10 11:01:21 +03:00
Vitaliy Filippov b66a079892 State basic OpenStack support 2021-07-10 01:11:20 +03:00
Vitaliy Filippov e90bbe6385 Implement OpenStack Cinder driver for Vitastor 
It can't delete snapshots yet because Vitastor layer merge isn't
implemented yet. You can only delete volumes with all snapshots.
This will be fixed in the near future.
 2021-07-10 01:06:29 +03:00
Vitaliy Filippov 4be761254c Move patches to patches/ 2021-07-09 21:51:19 +03:00
Vitaliy Filippov 7a45c5f86c buster-backports has broken mesa 2021-07-09 12:29:39 +03:00
Vitaliy Filippov bff413584d Fix qemuBlockStorageSourceGetVitastorProps 2021-07-09 02:09:47 +03:00
Vitaliy Filippov bb31050ab5 Add missing image, config_path options to QEMU QAPI 2021-07-09 02:09:47 +03:00
Vitaliy Filippov b52dd6843a Rename qemu_rbd_unescape and qemu_rbd_next_tok to *_vitastor_* 2021-07-03 23:14:44 +03:00
Vitaliy Filippov b66160a7ad Aggregate per-pool statistics in mon 2021-07-03 23:14:44 +03:00
Vitaliy Filippov 30bb602681 Add _VITASTOR to missing switches in libvirt 7.0 patch 2021-06-28 22:00:23 +03:00
Vitaliy Filippov eb0a3adafc Patch libvirt schema, add an example to test libvirt 2021-06-28 01:20:55 +03:00
Vitaliy Filippov 24301b116c Add libvirt 5.0 patch 2021-06-27 18:43:29 +03:00
Vitaliy Filippov 1d00c17d68 Add libvirt 7.5 patch 2021-06-27 10:58:12 +03:00
Vitaliy Filippov 24f19c4b80 Add libvirt 7.0 patch 2021-06-27 00:58:56 +03:00
Vitaliy Filippov dfdf5c1f9c Fix comments in mon.js 2021-06-20 00:23:56 +03:00
Vitaliy Filippov aad7792d3f Check for loops in parent inode chains 2021-06-20 00:23:03 +03:00
Vitaliy Filippov 6ca8afffe5 Add CAS version parameter to the C wrapper 2021-06-19 01:00:52 +03:00
Vitaliy Filippov 511a89948b Rework qemu_proxy into a C wrapper library with public header 2021-06-19 00:39:11 +03:00
Vitaliy Filippov 3de553ecd7 Add a test for CAS write operation 2021-06-15 00:12:35 +03:00
Vitaliy Filippov 9c45d43e74 Extract common 3 OSD code from several test scripts 2021-06-15 00:12:35 +03:00
Vitaliy Filippov 891250d355 Implement CAS writes 
From now on, reads will return the server-side object version numbers
and writes and deletes will have an additional "version" parameter
which, if set to a non-zero value, will be atomically compared with
the current version of the object plus 1 and the modification will
fail if it doesn't match.

This feature opens the road to correct online flattening of snapshot
layers and other interesting things.
 2021-06-15 00:12:35 +03:00
Vitaliy Filippov f9fe72d40a Release 0.6.4 
- Implement a basic Kubernetes CSI driver
- Minor fixes for vitastor-nbd
- Fix build without RDMA broken in 0.6.3
 2021-05-16 01:38:01 +03:00
Vitaliy Filippov 10ee4f7c1d Add notes about CSI to README 2021-05-16 01:38:01 +03:00
Vitaliy Filippov fd8244699b Implement basic CSI driver 
Currently can create and remove volumes, but resizing and snapshots is not supported yet
 2021-05-16 01:15:43 +03:00
Vitaliy Filippov eaac1fc5d1 Log to stderr in etcd_state_client, too 2021-05-16 01:09:25 +03:00
Vitaliy Filippov 57be1923d3 Daemonize NBD_DO_IT process, correctly cleanup unmounted NBD clients 2021-05-16 01:09:25 +03:00
Vitaliy Filippov c467acc388 Fix /v3 appendage to etcd URLs without /v3 2021-05-15 19:22:24 +03:00
Vitaliy Filippov bf591ba3ee Fix nbd module load check 2021-05-15 19:22:24 +03:00
Vitaliy Filippov 699a0fbbc7 Log to stderr instead of stdout in client 2021-05-15 19:22:24 +03:00
Vitaliy Filippov 6b2dd50f27 Fix build without RDMA 2021-05-08 18:20:43 +03:00
Vitaliy Filippov caf2f3c56f Release 0.6.3 
- RDMA support
- Client performance optimisations (4k randread ~120k -> ~180k on 1 core)
- JSON configuration file (/etc/vitastor/vitastor.conf) support
- Bug fixes
 2021-05-02 17:47:43 +03:00
Vitaliy Filippov 9174f188b1 Build packages with libibverbs 
For CentOS 7 it also requires newer rdma-core as CentOS 7's native version doesn't have
implicit ODP support. The updated version is already uploaded into the vitastor repo.
 2021-05-02 17:47:16 +03:00
Vitaliy Filippov d3978c6d0e Do not print RDMA connection messages when log_level=0 
By the way, it's 1 by default in the OSD, so these messages will still be there in OSD logs
 2021-05-01 00:26:09 +03:00
Vitaliy Filippov 4a7365660d Do not wait for down OSDs during sync 
Fixes a hang introduced in 0.5.11 in the non-immediate_commit mode
 2021-05-01 00:26:07 +03:00
Vitaliy Filippov 818ae5d61d Some config parsing fixes 2021-05-01 00:20:01 +03:00
Vitaliy Filippov 6810e93c3f Add RDMA options to mon.js list 2021-04-30 01:23:22 +03:00
Vitaliy Filippov f6f35f4127 Pass options correctly to not override /etc/vitastor/vitastor.conf 2021-04-30 01:17:44 +03:00
Vitaliy Filippov 72aa2fd819 Make OSD and client read common configuration from /etc/vitastor/vitastor.conf 2021-04-30 01:11:27 +03:00
Vitaliy Filippov 5010b0dd75 Use json11 instead of blockstore_config_t 2021-04-30 00:52:46 +03:00
Vitaliy Filippov 483c5ab380 Negotiate max_msg instead of max_sge, make buffer settings more conservative :-) 2021-04-29 11:10:35 +03:00
Vitaliy Filippov 6a6fd6544d Add RDMA options to the QEMU driver 2021-04-29 11:02:49 +03:00
Vitaliy Filippov 971aa4ae4f Implement RDMA receive with memory copying (send remains zero-copy) 
This is the simplest and, as usual, the best implementation :)

100% zero-copy implementation is also possible (see rdma-zerocopy branch),
but it requires to create A LOT of queues (~128 per client) to use QPN as a 'tag'
because of the lack of receive tags and the server may simply run out of queues.
Hardware limit is 262144 on Mellanox ConnectX-4 which amounts to only 2048
'connections' per host. And even with that amount of queues it's still less optimal
than the non-zerocopy one.

In fact, newest hardware like Mellanox ConnectX-5 does have Tag Matching
support, but it's still unsuitable for us because it doesn't support scatter/gather
(tm_caps.max_sge=1).
 2021-04-29 02:34:45 +03:00
Vitaliy Filippov 9e6cbc6ebc Negotiate max_sge between RDMA client & server 2021-04-29 02:15:20 +03:00
Vitaliy Filippov ce777319c3 WIP RDMA support 
Basic naive implementation works, but it's highly non-optimal as
RNR retransmissions occur all the time. RDMA expects the receiver
to always have place for incoming WRs...
 2021-04-29 02:03:54 +03:00
Vitaliy Filippov f8ff39b0ab Rework continue_ops() to remove a CPU hot spot 
This rework increases fio -rw=randread -iodepth=128 result from ~120k to ~180k iops :)
 2021-04-29 01:50:13 +03:00
Vitaliy Filippov d749159585 Linked list experiment 
Rework client operation queue from a vector to a linked list.
This is required to rework continue_ops() as its current implementation
consumes ~25% of client process CPU.
 2021-04-29 01:47:33 +03:00
Vitaliy Filippov 9703773a63 Fix has_flushes setting 2021-04-28 23:40:44 +03:00
Vitaliy Filippov 5d8d486f7c Add SOVERSION 2021-04-20 01:01:32 +03:00
Vitaliy Filippov 2b546cdd55 Link vitastor_blk with vitastor_common for timerfd_manager_t 
Not really required to operate, but fixes a verify-elf error
 2021-04-20 00:51:53 +03:00