cli/dd: Respect O_APPEND and the current position of stdout #130

Open
koolkhel wants to merge 2 commits from koolkhel/vitastor:dd-stdout-position into master
Contributor

Description

With no of=, vitastor-cli dd treated a regular-file stdout as seekable and wrote it with positional writes from offset 0:

  • vitastor-cli dd iimg=X >> file: the kernel appends every positional write to an O_APPEND file, so with iodepth > 1 the blocks landed in their completion order. The file had the right size, scrambled content and exit code 0.
  • { printf hdr; vitastor-cli dd iimg=X; printf trailer; } > file: dd overwrote hdr and left the position at 0, so trailer overwrote the start of the copy.
  1. cli/dd: Respect O_APPEND and the current position of stdout: treat an O_APPEND stdout like oflag=append, write a seekable stdout from its current position (seek= counts from there), and after a successful copy leave the position at its end, an unallocated image tail included.
  2. cli/dd: Keep in-order output going after ignored errors: with conv=noerror, output written in order (a pipe, oflag=append, and now a >> stdout) waited forever for a block whose read or write had failed. A failed read from an image or a seekable input now gives zeros (what a seekable output already got as a hole); a failed stream read is retried into the same block; in-order writes advance by blocks taken, not bytes written; a write takes its block only once it has an io_uring sqe, so a full ring no longer drops it.

Tested: four new cases in tests/test_dd.sh (>>, a shared stdout, seek= on a shared stdout, conv=noerror with an unreadable block appended with >>, using /sys/class/net/lo/speed, whose read fails with EINVAL; skipped where it can be read). Each fails without the fix; the whole test passes with it.

Contributor Name

Yury Luneff (MIND Software LLC, https://mindsw.io)

CLA

### Description With no `of=`, `vitastor-cli dd` treated a regular-file stdout as seekable and wrote it with positional writes from offset 0: - `vitastor-cli dd iimg=X >> file`: the kernel appends every positional write to an `O_APPEND` file, so with `iodepth` > 1 the blocks landed in their completion order. The file had the right size, scrambled content and exit code 0. - `{ printf hdr; vitastor-cli dd iimg=X; printf trailer; } > file`: dd overwrote `hdr` and left the position at 0, so `trailer` overwrote the start of the copy. 1. `cli/dd: Respect O_APPEND and the current position of stdout`: treat an `O_APPEND` stdout like `oflag=append`, write a seekable stdout from its current position (`seek=` counts from there), and after a successful copy leave the position at its end, an unallocated image tail included. 2. `cli/dd: Keep in-order output going after ignored errors`: with `conv=noerror`, output written in order (a pipe, `oflag=append`, and now a `>>` stdout) waited forever for a block whose read or write had failed. A failed read from an image or a seekable input now gives zeros (what a seekable output already got as a hole); a failed stream read is retried into the same block; in-order writes advance by blocks taken, not bytes written; a write takes its block only once it has an io_uring sqe, so a full ring no longer drops it. Tested: four new cases in `tests/test_dd.sh` (`>>`, a shared stdout, `seek=` on a shared stdout, `conv=noerror` with an unreadable block appended with `>>`, using `/sys/class/net/lo/speed`, whose read fails with EINVAL; skipped where it can be read). Each fails without the fix; the whole test passes with it. ### Contributor Name Yury Luneff (MIND Software LLC, https://mindsw.io) ### CLA - [x] I accept Vitastor CLA agreement: https://git.yourcmc.ru/vitalif/vitastor/src/branch/master/CLA-en.md
koolkhel added 1 commit 2026-11-18 20:40:55 +03:00
With no of=, dd treated a regular-file stdout as seekable and wrote it
with positional writes from offset 0. On a shell's >> the kernel appends
every positional write, so with iodepth > 1 the blocks landed in their
completion order. Without O_APPEND, dd overwrote what the shell had
written before it and left the position at 0 for what came after.

Treat an O_APPEND stdout like oflag=append, write a seekable stdout from
its current position, and leave the position at the end of the copy.
koolkhel force-pushed dd-stdout-position from 04bd387433 to 45dee04172 2026-11-18 22:18:52 +03:00 Compare
koolkhel force-pushed dd-stdout-position from 45dee04172 to 0a78467afc 2026-11-19 02:12:01 +03:00 Compare
koolkhel force-pushed dd-stdout-position from 0a78467afc to ee7692750b 2026-11-19 05:14:44 +03:00 Compare
This pull request can be merged automatically.
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u dd-stdout-position:koolkhel-dd-stdout-position
git checkout koolkhel-dd-stdout-position
Sign in to join this conversation.
No Reviewers
No Label
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: vitalif/vitastor#130
No description provided.