2022-01-29 23:43:22 +03:00
|
|
|
[Documentation](../../README.md#documentation) → Usage → Vitastor CLI
|
|
|
|
|
|
|
|
-----
|
|
|
|
|
|
|
|
[Читать на русском](cli.ru.md)
|
|
|
|
|
|
|
|
# Vitastor CLI
|
|
|
|
|
|
|
|
vitastor-cli is a command-line tool for administrative tasks like image management.
|
|
|
|
|
|
|
|
It supports the following commands:
|
|
|
|
|
|
|
|
- [status](#status)
|
|
|
|
- [df](#df)
|
|
|
|
- [ls](#ls)
|
|
|
|
- [create](#create)
|
2023-01-06 17:33:49 +03:00
|
|
|
- [snap-create](#create)
|
2022-01-29 23:43:22 +03:00
|
|
|
- [modify](#modify)
|
|
|
|
- [rm](#rm)
|
|
|
|
- [flatten](#flatten)
|
|
|
|
- [rm-data](#rm-data)
|
|
|
|
- [merge-data](#merge-data)
|
2023-04-22 02:44:10 +03:00
|
|
|
- [describe](#describe)
|
|
|
|
- [fix](#fix)
|
2022-01-29 23:43:22 +03:00
|
|
|
- [alloc-osd](#alloc-osd)
|
2022-12-24 16:07:02 +03:00
|
|
|
- [rm-osd](#rm-osd)
|
2024-02-26 23:47:12 +03:00
|
|
|
- [create-pool](#create-pool)
|
|
|
|
- [modify-pool](#modify-pool)
|
|
|
|
- [ls-pools](#ls-pools)
|
|
|
|
- [rm-pool](#rm-pool)
|
2022-01-29 23:43:22 +03:00
|
|
|
|
|
|
|
Global options:
|
|
|
|
|
|
|
|
```
|
2023-12-02 14:11:02 +03:00
|
|
|
--config_file FILE Path to Vitastor configuration file
|
|
|
|
--etcd_address URL Etcd connection address
|
2022-01-29 23:43:22 +03:00
|
|
|
--iodepth N Send N operations in parallel to each OSD when possible (default 32)
|
|
|
|
--parallel_osds M Work with M osds in parallel when possible (default 4)
|
|
|
|
--progress 1|0 Report progress (default 1)
|
|
|
|
--cas 1|0 Use CAS writes for flatten, merge, rm (default is decide automatically)
|
|
|
|
--no-color Disable colored output
|
|
|
|
--json JSON output
|
|
|
|
```
|
|
|
|
|
|
|
|
## status
|
|
|
|
|
|
|
|
`vitastor-cli status`
|
|
|
|
|
2022-07-14 02:32:43 +03:00
|
|
|
Show cluster status.
|
2022-01-29 23:43:22 +03:00
|
|
|
|
2022-07-14 02:32:43 +03:00
|
|
|
Example output:
|
2022-01-29 23:43:22 +03:00
|
|
|
|
|
|
|
```
|
|
|
|
cluster:
|
|
|
|
etcd: 1 / 1 up, 1.8 M database size
|
|
|
|
mon: 1 up, master stump
|
|
|
|
osd: 8 / 12 up
|
|
|
|
|
|
|
|
data:
|
|
|
|
raw: 498.5 G used, 301.2 G / 799.7 G available, 399.8 G down
|
|
|
|
state: 156.6 G clean, 97.6 G misplaced
|
|
|
|
pools: 2 / 3 active
|
|
|
|
pgs: 30 active
|
|
|
|
34 active+has_misplaced
|
|
|
|
32 offline
|
|
|
|
|
|
|
|
io:
|
|
|
|
client: 0 B/s rd, 0 op/s rd, 0 B/s wr, 0 op/s wr
|
|
|
|
rebalance: 989.8 M/s, 7.9 K op/s
|
|
|
|
```
|
|
|
|
|
|
|
|
## df
|
|
|
|
|
|
|
|
`vitastor-cli df`
|
|
|
|
|
2022-07-14 02:32:43 +03:00
|
|
|
Show pool space statistics.
|
2022-01-29 23:43:22 +03:00
|
|
|
|
2022-07-14 02:32:43 +03:00
|
|
|
Example output:
|
2022-01-29 23:43:22 +03:00
|
|
|
|
|
|
|
```
|
|
|
|
NAME SCHEME PGS TOTAL USED AVAILABLE USED% EFFICIENCY
|
|
|
|
testpool 2/1 32 100 G 34.2 G 60.7 G 39.23% 100%
|
|
|
|
size1 1/1 32 199.9 G 10 G 121.5 G 39.23% 100%
|
|
|
|
kaveri 2/1 32 0 B 10 G 0 B 100% 0%
|
|
|
|
```
|
|
|
|
|
2022-07-14 02:32:43 +03:00
|
|
|
In the example above, "kaveri" pool has "zero" efficiency because all its OSD are down.
|
2022-01-29 23:43:22 +03:00
|
|
|
|
|
|
|
## ls
|
|
|
|
|
|
|
|
`vitastor-cli ls [-l] [-p POOL] [--sort FIELD] [-r] [-n N] [<glob> ...]`
|
|
|
|
|
2022-07-14 02:32:43 +03:00
|
|
|
List images (only matching `<glob>` pattern(s) if passed).
|
2022-01-29 23:43:22 +03:00
|
|
|
|
2022-07-14 02:32:43 +03:00
|
|
|
Options:
|
2022-01-29 23:43:22 +03:00
|
|
|
|
|
|
|
```
|
2022-07-14 02:32:43 +03:00
|
|
|
-p|--pool POOL Filter images by pool ID or name
|
|
|
|
-l|--long Also report allocated size and I/O statistics
|
|
|
|
--del Also include delete operation statistics
|
|
|
|
--sort FIELD Sort by specified field (name, size, used_size, <read|write|delete>_<iops|bps|lat|queue>)
|
|
|
|
-r|--reverse Sort in descending order
|
|
|
|
-n|--count N Only list first N items
|
2022-01-29 23:43:22 +03:00
|
|
|
```
|
|
|
|
|
2022-07-14 02:32:43 +03:00
|
|
|
Example output:
|
2022-01-29 23:43:22 +03:00
|
|
|
|
|
|
|
```
|
|
|
|
NAME POOL SIZE USED READ IOPS QUEUE LAT WRITE IOPS QUEUE LAT FLAGS PARENT
|
|
|
|
debian9 testpool 20 G 12.3 G 0 B/s 0 0 0 us 0 B/s 0 0 0 us RO
|
|
|
|
pve/vm-100-disk-0 testpool 20 G 0 B 0 B/s 0 0 0 us 0 B/s 0 0 0 us - debian9
|
|
|
|
pve/base-101-disk-0 testpool 20 G 0 B 0 B/s 0 0 0 us 0 B/s 0 0 0 us RO debian9
|
|
|
|
pve/vm-102-disk-0 testpool 32 G 36.4 M 0 B/s 0 0 0 us 0 B/s 0 0 0 us - pve/base-101-disk-0
|
|
|
|
debian9-test testpool 20 G 36.6 M 0 B/s 0 0 0 us 0 B/s 0 0 0 us - debian9
|
|
|
|
bench testpool 10 G 10 G 0 B/s 0 0 0 us 0 B/s 0 0 0 us -
|
|
|
|
bench-kaveri kaveri 10 G 10 G 0 B/s 0 0 0 us 0 B/s 0 0 0 us -
|
|
|
|
```
|
|
|
|
|
|
|
|
## create
|
|
|
|
|
|
|
|
`vitastor-cli create -s|--size <size> [-p|--pool <id|name>] [--parent <parent_name>[@<snapshot>]] <name>`
|
|
|
|
|
2022-07-14 02:32:43 +03:00
|
|
|
Create an image. You may use K/M/G/T suffixes for `<size>`. If `--parent` is specified,
|
|
|
|
a copy-on-write image clone is created. Parent must be a snapshot (readonly image).
|
|
|
|
Pool must be specified if there is more than one pool.
|
2022-01-29 23:43:22 +03:00
|
|
|
|
|
|
|
```
|
|
|
|
vitastor-cli create --snapshot <snapshot> [-p|--pool <id|name>] <image>
|
|
|
|
vitastor-cli snap-create [-p|--pool <id|name>] <image>@<snapshot>
|
|
|
|
```
|
|
|
|
|
2022-07-14 02:32:43 +03:00
|
|
|
Create a snapshot of image `<name>` (either form can be used). May be used live if only a single writer is active.
|
2022-01-29 23:43:22 +03:00
|
|
|
|
2023-01-06 17:33:49 +03:00
|
|
|
See also about [how to export snapshots](qemu.en.md#exporting-snapshots).
|
|
|
|
|
2022-01-29 23:43:22 +03:00
|
|
|
## modify
|
|
|
|
|
2024-02-28 21:08:25 +03:00
|
|
|
`vitastor-cli modify <name> [--rename <new-name>] [--resize <size>] [--readonly | --readwrite] [-f|--force] [--down-ok]`
|
2022-01-29 23:43:22 +03:00
|
|
|
|
2022-07-14 02:32:43 +03:00
|
|
|
Rename, resize image or change its readonly status. Images with children can't be made read-write.
|
|
|
|
If the new size is smaller than the old size, extra data will be purged.
|
|
|
|
You should resize file system in the image, if present, before shrinking it.
|
2022-01-29 23:43:22 +03:00
|
|
|
|
2024-03-16 14:12:47 +03:00
|
|
|
* `-f|--force` - Proceed with shrinking or setting readwrite flag even if the image has children.
|
|
|
|
* `--down-ok` - Proceed with shrinking even if some data will be left on unavailable OSDs.
|
2022-01-29 23:43:22 +03:00
|
|
|
|
|
|
|
## rm
|
|
|
|
|
2024-02-28 21:08:25 +03:00
|
|
|
`vitastor-cli rm <from> [<to>] [--writers-stopped] [--down-ok]`
|
2022-01-29 23:43:22 +03:00
|
|
|
|
2022-07-14 02:32:43 +03:00
|
|
|
Remove `<from>` or all layers between `<from>` and `<to>` (`<to>` must be a child of `<from>`),
|
|
|
|
rebasing all their children accordingly. --writers-stopped allows merging to be a bit
|
|
|
|
more effective in case of a single 'slim' read-write child and 'fat' removed parent:
|
|
|
|
the child is merged into parent and parent is renamed to child in that case.
|
|
|
|
In other cases parent layers are always merged into children.
|
2022-01-29 23:43:22 +03:00
|
|
|
|
2024-02-28 21:08:25 +03:00
|
|
|
Other options:
|
|
|
|
|
2024-03-16 14:12:47 +03:00
|
|
|
* `--down-ok` - Continue deletion/merging even if some data will be left on unavailable OSDs.
|
2024-02-28 21:08:25 +03:00
|
|
|
|
2022-01-29 23:43:22 +03:00
|
|
|
## flatten
|
|
|
|
|
|
|
|
`vitastor-cli flatten <layer>`
|
|
|
|
|
2022-07-14 02:32:43 +03:00
|
|
|
Flatten a layer, i.e. merge data and detach it from parents.
|
2022-01-29 23:43:22 +03:00
|
|
|
|
|
|
|
## rm-data
|
|
|
|
|
|
|
|
`vitastor-cli rm-data --pool <pool> --inode <inode> [--wait-list] [--min-offset <offset>]`
|
|
|
|
|
2022-07-14 02:32:43 +03:00
|
|
|
Remove inode data without changing metadata.
|
2022-01-29 23:43:22 +03:00
|
|
|
|
|
|
|
```
|
2022-07-14 02:32:43 +03:00
|
|
|
--wait-list Retrieve full objects listings before starting to remove objects.
|
|
|
|
Requires more memory, but allows to show correct removal progress.
|
|
|
|
--min-offset Purge only data starting with specified offset.
|
2022-01-29 23:43:22 +03:00
|
|
|
```
|
|
|
|
|
|
|
|
## merge-data
|
|
|
|
|
|
|
|
`vitastor-cli merge-data <from> <to> [--target <target>]`
|
|
|
|
|
2022-07-14 02:32:43 +03:00
|
|
|
Merge layer data without changing metadata. Merge `<from>`..`<to>` to `<target>`.
|
|
|
|
`<to>` must be a child of `<from>` and `<target>` may be one of the layers between
|
|
|
|
`<from>` and `<to>`, including `<from>` and `<to>`.
|
2022-01-29 23:43:22 +03:00
|
|
|
|
2023-04-22 02:44:10 +03:00
|
|
|
## describe
|
|
|
|
|
2024-04-07 12:39:46 +03:00
|
|
|
`vitastor-cli describe [OPTIONS]`
|
2023-04-22 02:44:10 +03:00
|
|
|
|
2024-04-07 12:39:46 +03:00
|
|
|
Describe unclean object locations in the cluster. Options:
|
2023-04-22 02:44:10 +03:00
|
|
|
|
|
|
|
```
|
|
|
|
--osds <osds>
|
|
|
|
Only list objects from primary OSD(s) <osds>.
|
|
|
|
--object-state <states>
|
|
|
|
Only list objects in given state(s). State(s) may include:
|
|
|
|
degraded, misplaced, incomplete, corrupted, inconsistent.
|
|
|
|
--pool <pool name or number>
|
|
|
|
Only list objects in the given pool.
|
2024-04-07 12:39:46 +03:00
|
|
|
--pg <pg number>
|
|
|
|
Only list objects in the given PG of the pool.
|
2023-04-22 02:44:10 +03:00
|
|
|
--inode, --min-inode, --max-inode
|
|
|
|
Restrict listing to specific inode numbers.
|
|
|
|
--min-offset, --max-offset
|
|
|
|
Restrict listing to specific offsets inside inodes.
|
|
|
|
```
|
|
|
|
|
|
|
|
## fix
|
|
|
|
|
|
|
|
`vitastor-cli fix [--objects <objects>] [--bad-osds <osds>] [--part <part>] [--check no]`
|
|
|
|
|
|
|
|
Fix inconsistent objects in the cluster by deleting some copies.
|
|
|
|
|
|
|
|
```
|
|
|
|
--objects <objects>
|
|
|
|
Objects to fix, either in plain text or JSON format. If not specified,
|
|
|
|
object list will be read from STDIN in one of the same formats.
|
|
|
|
Plain text format: 0x<inode>:0x<stripe> <any delimiter> 0x<inode>:0x<stripe> ...
|
|
|
|
JSON format: [{"inode":"0x...","stripe":"0x..."},...]
|
|
|
|
--bad-osds <osds>
|
|
|
|
Remove inconsistent copies/parts of objects from these OSDs, effectively
|
|
|
|
marking them bad and allowing Vitastor to recover objects from other copies.
|
|
|
|
--part <number>
|
|
|
|
Only remove EC part <number> (from 0 to pg_size-1), required for extreme
|
|
|
|
edge cases where one OSD has multiple parts of a EC object.
|
|
|
|
--check no
|
|
|
|
Do not recheck that requested objects are actually inconsistent,
|
|
|
|
delete requested copies/parts anyway.
|
|
|
|
```
|
|
|
|
|
2022-01-29 23:43:22 +03:00
|
|
|
## alloc-osd
|
|
|
|
|
|
|
|
`vitastor-cli alloc-osd`
|
|
|
|
|
2022-07-14 02:32:43 +03:00
|
|
|
Allocate a new OSD number and reserve it by creating empty `/osd/stats/<n>` key.
|
2022-12-24 16:07:02 +03:00
|
|
|
|
|
|
|
## rm-osd
|
|
|
|
|
|
|
|
`vitastor-cli rm-osd [--force] [--allow-data-loss] [--dry-run] <osd_id> [osd_id...]`
|
|
|
|
|
|
|
|
Remove metadata and configuration for specified OSD(s) from etcd.
|
|
|
|
|
|
|
|
Refuses to remove OSDs with data without `--force` and `--allow-data-loss`.
|
|
|
|
|
|
|
|
With `--dry-run` only checks if deletion is possible without data loss and
|
|
|
|
redundancy degradation.
|
2024-02-26 23:47:12 +03:00
|
|
|
|
|
|
|
## create-pool
|
|
|
|
|
|
|
|
`vitastor-cli create-pool|pool-create <name> (-s <pg_size>|--ec <N>+<K>) -n <pg_count> [OPTIONS]`
|
|
|
|
|
|
|
|
Create a pool. Required parameters:
|
|
|
|
|
2024-03-16 14:12:47 +03:00
|
|
|
| <!-- --> | <!-- --> |
|
|
|
|
|--------------------------|---------------------------------------------------------------------------------------|
|
|
|
|
| `-s R` or `--pg_size R` | Number of replicas for replicated pools |
|
|
|
|
| `--ec N+K` | Number of data (N) and parity (K) chunks for erasure-coded pools |
|
|
|
|
| `-n N` or `--pg_count N` | PG count for the new pool (start with 10*<OSD count>/pg_size rounded to a power of 2) |
|
2024-02-26 23:47:12 +03:00
|
|
|
|
|
|
|
Optional parameters:
|
|
|
|
|
2024-03-16 14:12:47 +03:00
|
|
|
| <!-- --> | <!-- --> |
|
|
|
|
|--------------------------------|----------------------------------------------------------------------------|
|
2024-02-26 23:47:12 +03:00
|
|
|
| `--pg_minsize <number>` | R or N+K minus number of failures to tolerate without downtime ([details](../config/pool.en.md#pg_minsize)) |
|
|
|
|
| `--failure_domain host` | Failure domain: host, osd or a level from placement_levels. Default: host |
|
|
|
|
| `--root_node <node>` | Put pool only on child OSDs of this placement tree node |
|
|
|
|
| `--osd_tags <tag>[,<tag>]...` | Put pool only on OSDs tagged with all specified tags |
|
|
|
|
| `--block_size 128k` | Put pool only on OSDs with this data block size |
|
|
|
|
| `--bitmap_granularity 4k` | Put pool only on OSDs with this logical sector size |
|
|
|
|
| `--immediate_commit none` | Put pool only on OSDs with this or larger immediate_commit (none < small < all) |
|
2024-04-07 00:38:22 +03:00
|
|
|
| `--level_placement <rules>` | Use additional failure domain rules (example: "dc=112233") |
|
|
|
|
| `--raw_placement <rules>` | Specify raw PG generation rules ([details](../config/pool.en.md#raw_placement)) |
|
2024-02-26 23:47:12 +03:00
|
|
|
| `--primary_affinity_tags tags` | Prefer to put primary copies on OSDs with all specified tags |
|
|
|
|
| `--scrub_interval <time>` | Enable regular scrubbing for this pool. Format: number + unit s/m/h/d/M/y |
|
2024-03-10 18:08:57 +03:00
|
|
|
| `--used_for_fs <name>` | Mark pool as used for VitastorFS with metadata in image <name> |
|
2024-02-26 23:47:12 +03:00
|
|
|
| `--pg_stripe_size <number>` | Increase object grouping stripe |
|
|
|
|
| `--max_osd_combinations 10000` | Maximum number of random combinations for LP solver input |
|
|
|
|
| `--wait` | Wait for the new pool to come online |
|
2024-03-16 14:12:47 +03:00
|
|
|
| `-f` or `--force` | Do not check that cluster has enough OSDs to create the pool |
|
2024-02-26 23:47:12 +03:00
|
|
|
|
|
|
|
See also [Pool configuration](../config/pool.en.md) for detailed parameter descriptions.
|
|
|
|
|
|
|
|
Examples:
|
|
|
|
|
|
|
|
`vitastor-cli create-pool test_x4 -s 4 -n 32`
|
|
|
|
|
|
|
|
`vitastor-cli create-pool test_ec42 --ec 4+2 -n 32`
|
|
|
|
|
|
|
|
## modify-pool
|
|
|
|
|
|
|
|
`vitastor-cli modify-pool|pool-modify <id|name> [--name <new_name>] [PARAMETERS...]`
|
|
|
|
|
|
|
|
Modify an existing pool. Modifiable parameters:
|
|
|
|
|
|
|
|
```
|
|
|
|
[-s|--pg_size <number>] [--pg_minsize <number>] [-n|--pg_count <count>]
|
2024-03-01 01:42:46 +03:00
|
|
|
[--failure_domain <level>] [--root_node <node>] [--osd_tags <tags>] [--no_inode_stats 0|1]
|
2024-02-26 23:47:12 +03:00
|
|
|
[--max_osd_combinations <number>] [--primary_affinity_tags <tags>] [--scrub_interval <time>]
|
|
|
|
```
|
|
|
|
|
|
|
|
Non-modifiable parameters (changing them WILL lead to data loss):
|
|
|
|
|
|
|
|
```
|
|
|
|
[--block_size <size>] [--bitmap_granularity <size>]
|
|
|
|
[--immediate_commit <all|small|none>] [--pg_stripe_size <size>]
|
|
|
|
```
|
|
|
|
|
|
|
|
These, however, can still be modified with -f|--force.
|
|
|
|
|
|
|
|
See [create-pool](#create-pool) for parameter descriptions.
|
|
|
|
|
|
|
|
Examples:
|
|
|
|
|
|
|
|
`vitastor-cli modify-pool pool_A --name pool_B`
|
|
|
|
|
|
|
|
`vitastor-cli modify-pool 2 --pg_size 4 -n 128`
|
|
|
|
|
|
|
|
## rm-pool
|
|
|
|
|
|
|
|
`vitastor-cli rm-pool|pool-rm [--force] <id|name>`
|
|
|
|
|
|
|
|
Remove a pool. Refuses to remove pools with images without `--force`.
|
|
|
|
|
|
|
|
## ls-pools
|
|
|
|
|
|
|
|
`vitastor-cli ls-pools|pool-ls|ls-pool|pools [-l] [--detail] [--sort FIELD] [-r] [-n N] [--stats] [<glob> ...]`
|
|
|
|
|
|
|
|
List pools (only matching <glob> patterns if passed).
|
|
|
|
|
2024-03-16 14:12:47 +03:00
|
|
|
| <!-- --> | <!-- --> |
|
|
|
|
|----------------------|-------------------------------------------------------|
|
|
|
|
| `-l` or `--long` | Also report I/O statistics |
|
|
|
|
| `--detail` | Use list format (not table), show all details |
|
|
|
|
| `--sort FIELD` | Sort by specified field (see fields in --json output) |
|
|
|
|
| `-r` or `--reverse` | Sort in descending order |
|
|
|
|
| `-n` or `--count N` | Only list first N items |
|