Add a note about HDD, enable throttling only for hybrid OSDs
Test / test_add_osd (push) Successful in 2m21s Details
Test / buildenv (push) Successful in 10s Details
Test / build (push) Successful in 2m39s Details
Test / make_test (push) Successful in 34s Details
Test / test_cas (push) Successful in 7s Details
Test / test_change_pg_count (push) Successful in 43s Details
Test / test_change_pg_count_ec (push) Successful in 38s Details
Test / test_change_pg_size (push) Successful in 7s Details
Test / test_create_nomaxid (push) Successful in 7s Details
Test / test_etcd_fail (push) Successful in 1m30s Details
Test / test_interrupted_rebalance (push) Successful in 2m4s Details
Test / test_interrupted_rebalance_imm (push) Successful in 1m30s Details
Test / test_interrupted_rebalance_ec (push) Successful in 1m45s Details
Test / test_interrupted_rebalance_ec_imm (push) Successful in 1m23s Details
Test / test_failure_domain (push) Successful in 9s Details
Test / test_snapshot (push) Successful in 19s Details
Test / test_snapshot_ec (push) Successful in 19s Details
Test / test_minsize_1 (push) Successful in 12s Details
Test / test_move_reappear (push) Failing after 1m32s Details
Test / test_rm (push) Successful in 12s Details
Test / test_snapshot_chain (push) Successful in 1m8s Details
Test / test_snapshot_chain_ec (push) Successful in 2m2s Details
Test / test_snapshot_down (push) Successful in 22s Details
Test / test_snapshot_down_ec (push) Successful in 21s Details
Test / test_splitbrain (push) Successful in 13s Details
Test / test_rebalance_verify (push) Successful in 2m52s Details
Test / test_rebalance_verify_imm (push) Successful in 2m46s Details
Test / test_rebalance_verify_ec (push) Successful in 5m9s Details
Test / test_rebalance_verify_ec_imm (push) Successful in 5m5s Details
Test / test_write (push) Successful in 40s Details
Test / test_write_xor (push) Successful in 49s Details
Test / test_write_no_same (push) Successful in 15s Details
Test / test_heal_pg_size_2 (push) Successful in 3m43s Details
Test / test_heal_ec (push) Successful in 4m35s Details
Test / test_scrub (push) Successful in 37s Details
Test / test_scrub_zero_osd_2 (push) Successful in 40s Details
Test / test_scrub_xor (push) Successful in 31s Details
Test / test_scrub_pg_size_3 (push) Successful in 40s Details
Test / test_scrub_pg_size_6_pg_minsize_4_osd_count_6_ec (push) Successful in 26s Details
Test / test_scrub_ec (push) Successful in 22s Details

cached-reads
Vitaliy Filippov 2023-07-09 12:45:11 +03:00
parent b7e4d0c9bf
commit 57ad4c3636
5 changed files with 19 additions and 11 deletions

View File

@ -15,7 +15,7 @@ Vitastor архитектурно похож на Ceph, что означает
и автоматическое распределение данных по любому числу дисков любого размера с настраиваемыми схемами
избыточности - репликацией или с произвольными кодами коррекции ошибок.
Vitastor нацелен на SSD и SSD+HDD кластеры с как минимум 10 Гбит/с сетью, поддерживает
Vitastor нацелен в первую очередь на SSD и SSD+HDD кластеры с как минимум 10 Гбит/с сетью, поддерживает
TCP и RDMA и на хорошем железе может достигать задержки 4 КБ чтения и записи на уровне ~0.1 мс,
что примерно в 10 раз быстрее, чем Ceph и другие популярные программные СХД.

View File

@ -14,8 +14,8 @@ Vitastor is architecturally similar to Ceph which means strong consistency,
primary-replication, symmetric clustering and automatic data distribution over any
number of drives of any size with configurable redundancy (replication or erasure codes/XOR).
Vitastor targets SSD and SSD+HDD clusters with at least 10 Gbit/s network, supports
TCP and RDMA and may achieve 4 KB read and write latency as low as ~0.1 ms
Vitastor targets primarily SSD and SSD+HDD clusters with at least 10 Gbit/s network,
supports TCP and RDMA and may achieve 4 KB read and write latency as low as ~0.1 ms
with proper hardware which is ~10 times faster than other popular SDS's like Ceph
or internal systems of public clouds.

View File

@ -19,6 +19,9 @@
- Get some SATA or NVMe SSDs with capacitors (server-grade drives). You can use desktop SSDs
with lazy fsync, but prepare for inferior single-thread latency. Read more about capacitors
[here](../config/layout-cluster.en.md#immediate_commit).
- If you want to use HDDs, get modern HDDs with Media Cache or SSD Cache: HGST Ultrastar,
Toshiba MG08, Seagate EXOS or something similar. If your drives don't have such cache then
you also need small SSDs for journal and metadata (even 2 GB per 1 TB of HDD space is enough).
- Get a fast network (at least 10 Gbit/s). Something like Mellanox ConnectX-4 with RoCEv2 is ideal.
- Disable CPU powersaving: `cpupower idle-set -D 0 && cpupower frequency-set -g performance`.
- [Install Vitastor packages](../installation/packages.en.md).
@ -45,9 +48,10 @@ On the monitor hosts:
}
```
- Initialize OSDs:
- SSD-only: `vitastor-disk prepare /dev/sdXXX [/dev/sdYYY ...]`. You can add
`--disable_data_fsync off` to leave disk cache enabled if you use desktop
SSDs without capacitors.
- SSD-only or HDD-only: `vitastor-disk prepare /dev/sdXXX [/dev/sdYYY ...]`.
Add `--disable_data_fsync off` to leave disk write cache enabled if you use
desktop SSDs without capacitors. Do NOT add `--disable_data_fsync off` if you
use HDDs or SSD+HDD.
- Hybrid, SSD+HDD: `vitastor-disk prepare --hybrid /dev/sdXXX [/dev/sdYYY ...]`.
Pass all your devices (HDD and SSD) to this script — it will partition disks and initialize journals on its own.
This script skips HDDs which are already partitioned so if you want to use non-empty disks for

View File

@ -19,6 +19,9 @@
- Возьмите серверы с SSD (SATA или NVMe), желательно с конденсаторами (серверные SSD). Можно
использовать и десктопные SSD, включив режим отложенного fsync, но производительность будет хуже.
О конденсаторах читайте [здесь](../config/layout-cluster.ru.md#immediate_commit).
- Если хотите использовать HDD, берите современные модели с Media или SSD кэшем - HGST Ultrastar,
Toshiba MG08, Seagate EXOS или что-то похожее. Если такого кэша у ваших дисков нет,
обязательно возьмите SSD под метаданные и журнал (маленькие, буквально 2 ГБ на 1 ТБ HDD-места).
- Возьмите быструю сеть, минимум 10 гбит/с. Идеал - что-то вроде Mellanox ConnectX-4 с RoCEv2.
- Для лучшей производительности отключите энергосбережение CPU: `cpupower idle-set -D 0 && cpupower frequency-set -g performance`.
- [Установите пакеты Vitastor](../installation/packages.ru.md).
@ -45,9 +48,10 @@
}
```
- Инициализуйте OSD:
- SSD: `vitastor-disk prepare /dev/sdXXX [/dev/sdYYY ...]`. Если вы используете
десктопные SSD без конденсаторов, можете оставить кэш включённым, добавив
опцию `--disable_data_fsync off`.
- Только SSD или только HDD: `vitastor-disk prepare /dev/sdXXX [/dev/sdYYY ...]`.
Если вы используете десктопные SSD без конденсаторов, добавьте опцию `--disable_data_fsync off`,
чтобы оставить кэш записи диска включённым. НЕ добавляйте эту опцию, если используете
жёсткие диски (HDD).
- Гибридные, SSD+HDD: `vitastor-disk prepare --hybrid /dev/sdXXX [/dev/sdYYY ...]`.
Передайте все ваши SSD и HDD скрипту в командной строке подряд, скрипт автоматически выделит
разделы под журналы на SSD и данные на HDD. Скрипт пропускает HDD, на которых уже есть разделы

View File

@ -99,7 +99,7 @@ int disk_tool_t::prepare_one(std::map<std::string, std::string> options, int is_
if (options["journal_size"] == "")
{
if (options["journal_device"] == "")
options["journal_size"] = "32M";
options["journal_size"] = is_hdd ? "128M" : "32M";
else if (is_hdd)
options["journal_size"] = DEFAULT_HYBRID_JOURNAL;
}
@ -107,7 +107,7 @@ int disk_tool_t::prepare_one(std::map<std::string, std::string> options, int is_
{
if (options["block_size"] == "")
options["block_size"] = "1M";
if (options["throttle_small_writes"] == "")
if (options["journal_device"] != "" && options["throttle_small_writes"] == "")
options["throttle_small_writes"] = "1";
}
json11::Json::object sb;