cli/dd: Respect O_APPEND and the current position of stdout #130
Open
koolkhel
wants to merge 2 commits from
koolkhel/vitastor:dd-stdout-position into master
pull from: koolkhel/vitastor:dd-stdout-position
merge into: vitalif:master
vitalif:master
vitalif:tombstones
vitalif:primary-stickiness
vitalif:libvfn
vitalif:canceling
vitalif:immediate-repeering
vitalif:rdma-close-tcp
vitalif:use-degraded-writes
vitalif:fix-write-delete-flow
vitalif:openssl-3.0.12
vitalif:openssl-capture-secrets
vitalif:ignore-small-rdma-end
vitalif:encrypt-batch
vitalif:faster-peering-calc
vitalif:http-hello
vitalif:xxh3-bench
vitalif:create-user-etcd-based
vitalif:aesxts_streaming_nosupport
vitalif:refactor-heap-continue-write
vitalif:ston3lu-patch-0127
vitalif:rdma-credits
vitalif:heap-meta
vitalif:sendmsg-zc-no-more-error
vitalif:fix-clear-refs
vitalif:veeam-trick
vitalif:test-etcd-schizo
vitalif:test
vitalif:test1
vitalif:test2
vitalif:cow-meta
vitalif:kv-readahead
vitalif:qemu-fix-crash-idea
vitalif:eventloop-auto-init-trigger
vitalif:hugo-docs
vitalif:node-mutex-and-postpone
vitalif:check-writeback
vitalif:msgr-iothreads-v2
vitalif:antietcd
vitalif:rel-1.4
vitalif:test-fix-ec-unknown-state-51
vitalif:kv-update
vitalif:kv-debug
vitalif:hotfix-1.2.0
vitalif:rdma-simple-nodp
vitalif:zerocopy-tcp-send
vitalif:csi-staging
vitalif:hotfix-1.1.0
vitalif:hotfix-1.0.0
vitalif:fsync-feedback
vitalif:cached-reads
vitalif:qemu-send-loop
vitalif:test-double-alloc
vitalif:hier-failure-domains
vitalif:mon-self-restart
vitalif:epoch-deletions
vitalif:rdma-flow-control
vitalif:csi-use-vitastor-cli
vitalif:rdma-v2
vitalif:etcd-hide-base64
vitalif:lrc-matrix
vitalif:rm-left-on-dead
vitalif:nfs-proxy-old
vitalif:non-odp-rdma
vitalif:test-zctr
vitalif:separate-data-connections
vitalif:sec_osd_msgr
vitalif:test-assert
vitalif:nbd-vmsplice
vitalif:allow-etcd-address-option
vitalif:rdma-zerocopy
vitalif:rel-0.5
vitalif:openonload
vitalif:sync-io-test
vitalif:trace-sync
vitalif:trace-sqes
vitalif:test-submit-and-wait
vitalif:test-sq-poll
vitalif:blocking-uring-test
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
No description provided.
Delete Branch "koolkhel/vitastor:dd-stdout-position"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Description
With no
of=,vitastor-cli ddtreated a regular-file stdout as seekable and wrote it with positional writes from offset 0:vitastor-cli dd iimg=X >> file: the kernel appends every positional write to anO_APPENDfile, so withiodepth> 1 the blocks landed in their completion order. The file had the right size, scrambled content and exit code 0.{ printf hdr; vitastor-cli dd iimg=X; printf trailer; } > file: dd overwrotehdrand left the position at 0, sotraileroverwrote the start of the copy.cli/dd: Respect O_APPEND and the current position of stdout: treat anO_APPENDstdout likeoflag=append, write a seekable stdout from its current position (seek=counts from there), and after a successful copy leave the position at its end, an unallocated image tail included.cli/dd: Keep in-order output going after ignored errors: withconv=noerror, output written in order (a pipe,oflag=append, and now a>>stdout) waited forever for a block whose read or write had failed. A failed read from an image or a seekable input now gives zeros (what a seekable output already got as a hole); a failed stream read is retried into the same block; in-order writes advance by blocks taken, not bytes written; a write takes its block only once it has an io_uring sqe, so a full ring no longer drops it.Tested: four new cases in
tests/test_dd.sh(>>, a shared stdout,seek=on a shared stdout,conv=noerrorwith an unreadable block appended with>>, using/sys/class/net/lo/speed, whose read fails with EINVAL; skipped where it can be read). Each fails without the fix; the whole test passes with it.Contributor Name
Yury Luneff (MIND Software LLC, https://mindsw.io)
CLA
04bd387433to45dee0417245dee04172to0a78467afc0a78467afctoee7692750bView command line instructions
Checkout
From your project repository, check out a new branch and test the changes.