Refuse to revive an NBD device with another image #155

Open
koolkhel wants to merge 1 commits from koolkhel/vitastor:nbd-backend-identifier into master
Contributor

Description

netlink-revive /dev/nbdN --image other silently attached a dead netlink device to another image: it exited 0, the device read the other image, and writes went there. A filesystem mounted from the device ends up corrupting both images.

The kernel already guards against this. A RECONFIGURE whose NBD_ATTR_BACKEND_IDENTIFIER differs from the one given on CONNECT fails with backend image doesn't match with ..., and one without it fails with must specify backend. But vitastor-nbd never sent the attribute. The code was under #ifdef NBD_ATTR_BACKEND_IDENTIFIER, which is an enum value in linux/nbd-netlink.h and never defined, and the caller passed NULL anyway.

This PR:

  • detects the attribute with check_c_source_compiles (HAVE_NBD_BACKEND_IDENTIFIER), since #ifdef can't see an enum value. Headers older than Linux 5.6 build without it, as before;
  • sends "<pool>:<inode>" on netlink-map and netlink-revive;
  • says so in the netlink-revive section of docs/usage/nbd.*.md;
  • adds tests/test_nbd_netlink_revive.sh: map an image, kill -9 the daemon, check that a revive with another image fails and one with the same image restores the data.

Choices I made, happy to change them:

  1. The identifier is <pool>:<inode>, not the image name. It survives a rename, and a device mapped with --image can be revived with --pool --inode and back. The name would read better in /sys/block/nbdN/backend, but after a rename the device could no longer be revived at all. Switching to the name is a one-line change.
  2. Compatibility, checked on kernel 6.8:
    • A device mapped by an older vitastor-nbd (no identifier) can be revived by the new one. It stays unprotected until it is mapped again, because the kernel doesn't set the identifier on RECONFIGURE.
    • A device mapped by the new vitastor-nbd can't be revived by an older one: the kernel says must specify backend. I don't see a way around that short of not sending the identifier.

Tested as root on kernel 6.8 against a local 3-OSD cluster (tests/run_3osds.sh):

  • The new test passes, and fails on master with netlink-revive attached /dev/nbd0 of testimg to testimg2.
  • Revive with another image: exit 1, backend image doesn't match with 1:1. Revive with the same image, by --image or by --pool 1 --inode 1 --size ...: exit 0, the device reads the image.
  • Old map with new revive, and new map with old revive, as described above.
  • map, netlink-map, busy devices, resize, revive after kill -9 and unmap work as before.

Contributor Name

Yury Luneff (MIND Software LLC, https://mindsw.io)

CLA

### Description `netlink-revive /dev/nbdN --image other` silently attached a dead netlink device to another image: it exited 0, the device read the other image, and writes went there. A filesystem mounted from the device ends up corrupting both images. The kernel already guards against this. A RECONFIGURE whose `NBD_ATTR_BACKEND_IDENTIFIER` differs from the one given on CONNECT fails with `backend image doesn't match with ...`, and one without it fails with `must specify backend`. But vitastor-nbd never sent the attribute. The code was under `#ifdef NBD_ATTR_BACKEND_IDENTIFIER`, which is an enum value in `linux/nbd-netlink.h` and never defined, and the caller passed `NULL` anyway. This PR: - detects the attribute with `check_c_source_compiles` (`HAVE_NBD_BACKEND_IDENTIFIER`), since `#ifdef` can't see an enum value. Headers older than Linux 5.6 build without it, as before; - sends `"<pool>:<inode>"` on `netlink-map` and `netlink-revive`; - says so in the `netlink-revive` section of `docs/usage/nbd.*.md`; - adds `tests/test_nbd_netlink_revive.sh`: map an image, `kill -9` the daemon, check that a revive with another image fails and one with the same image restores the data. Choices I made, happy to change them: 1. **The identifier is `<pool>:<inode>`, not the image name.** It survives a rename, and a device mapped with `--image` can be revived with `--pool --inode` and back. The name would read better in `/sys/block/nbdN/backend`, but after a rename the device could no longer be revived at all. Switching to the name is a one-line change. 2. **Compatibility, checked on kernel 6.8:** - A device mapped by an older vitastor-nbd (no identifier) can be revived by the new one. It stays unprotected until it is mapped again, because the kernel doesn't set the identifier on RECONFIGURE. - A device mapped by the new vitastor-nbd can't be revived by an older one: the kernel says `must specify backend`. I don't see a way around that short of not sending the identifier. Tested as root on kernel 6.8 against a local 3-OSD cluster (`tests/run_3osds.sh`): - The new test passes, and fails on master with `netlink-revive attached /dev/nbd0 of testimg to testimg2`. - Revive with another image: exit 1, `backend image doesn't match with 1:1`. Revive with the same image, by `--image` or by `--pool 1 --inode 1 --size ...`: exit 0, the device reads the image. - Old map with new revive, and new map with old revive, as described above. - `map`, `netlink-map`, busy devices, resize, revive after `kill -9` and `unmap` work as before. ### Contributor Name Yury Luneff (MIND Software LLC, https://mindsw.io) ### CLA - [x] I accept Vitastor CLA agreement: https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/CLA-en.md
koolkhel added 1 commit 2026-11-22 22:40:14 +03:00
netlink-revive /dev/nbdN --image other silently attached the device to
another image, so a filesystem mounted from it read and wrote that
image. The kernel refuses a RECONFIGURE whose backend identifier differs
from the one given on CONNECT, but vitastor-nbd never sent one: the code
sat under #ifdef NBD_ATTR_BACKEND_IDENTIFIER, which is an enum value and
never defined, and the caller passed NULL anyway.

Detect the attribute with CMake and send "<pool>:<inode>" on map and
revive. It survives an image rename and is the same for --image and
--pool/--inode. A device mapped without an identifier can still be
revived, unprotected. A device mapped with one can't be revived by an
older vitastor-nbd, which sends none.
This pull request can be merged automatically.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u nbd-backend-identifier:koolkhel-nbd-backend-identifier
git checkout koolkhel-nbd-backend-identifier
Sign in to join this conversation.
No Reviewers
No Label
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: vitalif/vitastor#155
No description provided.