cli/dd: Fix size=, status=, oflag=append, pipe and character device inputs #94

koolkhel · 2026-10-25T16:14:38+03:00

koolkhel commented

2026-10-25 16:14:38 +03:00

Description

Fixes for vitastor-cli dd, one commit each. They change the same functions and extend the same sections of tests/test_dd.sh, so they come as one series.

Detect input size also when size= is given. dd iimg=testimg oimg=testimg7 size=32M failed with "Input seek position is beyond end of input" and left the output image created; a skip= past the end also left it created.
Make status=none also suppress the progress output. With the default --progress 1, status=none still printed the progress line and the statistics.
Implement status=noxfer. It was listed in the help but ignored.
Fix oflag=append. O_APPEND replaced O_RDWR, so the output file was opened read-only and every write failed with EBADF. Also keeps the final fsync for an appending output and refuses conv=trunc together with oflag=append.
Copy a pipe input to its end when no count= or size= is given. cat 16M-file | vitastor-cli dd of=copy copied 1 MiB and exited 0, also with seek=.
Check write alignment before creating the output image. An unaligned seek= failed with EINVAL but left the output image created or grown.
Read a character device input like a pipe. A character device that allows lseek but reports size 0 (e.g. /dev/mem) failed with "Input seek position is beyond end of input".
Free the skip buffer when reading the input fails. skip_read() leaked its buffer on a read error or EOF.

Tested: tests/test_dd.sh, extended for each commit; every new check fails without its fix and passes with it. Checked by hand only: the fsync of an appending output (strace shows fsync() = 0 after the fix, and no fsync to a pipe); the character device fix, as root (dd if=/dev/mem bs=4096 count=1 read 0 bytes with rc 22 before, 4096 bytes with rc 0 after; the /dev/zero test added for it only guards against regressions); the skip buffer leak (found by static analysis).

Contributor Name

Yury Luneff (MIND Software LLC, https://mindsw.io)

CLA

I accept Vitastor CLA agreement: https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/CLA-en.md

### Description Fixes for `vitastor-cli dd`, one commit each. They change the same functions and extend the same sections of tests/test_dd.sh, so they come as one series. - **Detect input size also when size= is given.** `dd iimg=testimg oimg=testimg7 size=32M` failed with "Input seek position is beyond end of input" and left the output image created; a `skip=` past the end also left it created. - **Make status=none also suppress the progress output.** With the default `--progress 1`, `status=none` still printed the progress line and the statistics. - **Implement status=noxfer.** It was listed in the help but ignored. - **Fix oflag=append.** O_APPEND replaced O_RDWR, so the output file was opened read-only and every write failed with EBADF. Also keeps the final fsync for an appending output and refuses `conv=trunc` together with `oflag=append`. - **Copy a pipe input to its end when no count= or size= is given.** `cat 16M-file | vitastor-cli dd of=copy` copied 1 MiB and exited 0, also with `seek=`. - **Check write alignment before creating the output image.** An unaligned `seek=` failed with EINVAL but left the output image created or grown. - **Read a character device input like a pipe.** A character device that allows lseek but reports size 0 (e.g. /dev/mem) failed with "Input seek position is beyond end of input". - **Free the skip buffer when reading the input fails.** `skip_read()` leaked its buffer on a read error or EOF. Tested: tests/test_dd.sh, extended for each commit; every new check fails without its fix and passes with it. Checked by hand only: the fsync of an appending output (strace shows `fsync() = 0` after the fix, and no fsync to a pipe); the character device fix, as root (`dd if=/dev/mem bs=4096 count=1` read 0 bytes with rc 22 before, 4096 bytes with rc 0 after; the /dev/zero test added for it only guards against regressions); the skip buffer leak (found by static analysis). ### Contributor Name Yury Luneff (MIND Software LLC, https://mindsw.io) ### CLA - [x] I accept Vitastor CLA agreement: https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/CLA-en.md

koolkhel added 8 commits 2026-10-25 16:14:39 +03:00

cli/dd: Detect input size also when size= is given 6326a796ee

size= turned input size detection off, so dd from an image, a file or a
block device with size= always failed with "Input seek position is
beyond end of input", after the output image was already created or
resized. Always detect the input size: size= only sets the output size.
Also check the input seek position before opening the output.

cli/dd: Make status=none also suppress the progress output 1c0cc44e32

status=none only dropped the final statistics when --progress 0 was
also given; with the default --progress 1 dd still printed the progress
line and the statistics. Turn progress off with status=none, so only
errors are printed, like in GNU dd. --json still returns the result.

cli/dd: Implement status=noxfer c040b3f98b

status=noxfer is listed in the help, but dd ignored it. Make it drop
the final transfer statistics like in GNU dd. The progress line still
follows --progress and, if it was shown, is cleared at the end instead
of being replaced by the statistics. --json still returns them.

cli/dd: Fix oflag=append 0618b22013

oflag=append replaced O_RDWR with O_APPEND, so the output file was
opened read-only and every write failed with EBADF. Open it with
O_WRONLY|O_APPEND. An appending output is not seekable, which also
skipped the final fsync: skip it only for outputs that can't be synced
(anything but regular files and block devices). Refuse conv=trunc with
oflag=append, the two contradict each other.

cli/dd: Copy a pipe input to its end when no count= or size= is given d4d227a838

With a non-seekable input and no count= or size=, read_end stayed 0, or
was capped at zero bytes by out_size defaulting to seek=, so dd stopped
after the first block: "cat file | vitastor-cli dd of=copy" copied 1 MB
and exited 0. Leave read_end unlimited then and default out_size only
when the input length is known. A new or truncated image still needs
size=; conv=trunc on a file still truncates it at the seek offset.

cli/dd: Check write alignment before creating the output image 25ecb001aa

The "Unaligned write to Vitastor is not supported" check ran after the
output image was created or resized, so a seek= or bs= not aligned to
the pool's bitmap granularity failed with EINVAL but left a new or
resized image behind. Check it as soon as the granularity is known.

cli/dd: Read a character device input like a pipe 891f7552d1

A character device reports st_size 0, so one that supports lseek (e.g.
/dev/mem or an MTD device) was taken as an empty seekable input and dd
always failed with "Input seek position is beyond end of input". Treat
every character device input as non-seekable with unknown size and read
it up to count=, size= or its end. /dev/zero and /dev/urandom don't
seek and were already read this way.

cli/dd: Free the skip buffer when reading the input fails 34b09af052

skip_read() returned on a read error or EOF without freeing the buffer
it allocated, leaking it once per failed dd.

vitalif commented

2026-10-25 17:14:22 +03:00

Замержил всё отдельными коммитами

vitalif closed this pull request

2026-10-25 17:14:22 +03:00

Pull request closed

This pull request cannot be reopened because the branch was deleted.

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: vitalif/vitastor#94

Allow edits from maintainers