Compare commits

...

66 Commits

Author SHA1 Message Date
Vitaliy Filippov 7a12c251bb Expose Gid field in OpContext 2024-10-23 20:49:51 +03:00
Vitaliy Filippov e64d1db179 Remove default os.ModeDevice in ConvertFileMode 2024-10-23 20:49:51 +03:00
Vitaliy Filippov f403be49ac Add READDIRPLUS support 2024-10-23 20:49:51 +03:00
Vitaliy Filippov d9f86f936a Move convertChildInodeEntry, convertAttributes, convertExpirationTime to fuseops 2024-10-23 20:49:48 +03:00
Vitaliy Filippov e03805b2b5 Use fuse_kernel_linux for Windows builds too 2024-10-23 20:49:25 +03:00
Vitaliy Filippov 4a8b61d056 Move Convert(File|Golang)Mode to fuseops 2024-10-23 20:49:25 +03:00
Vitaliy Filippov e37465aa67 Do not use syscall.O_ACCMODE 2024-10-23 20:49:25 +03:00
Vitaliy Filippov 3f512fd68f Allow to use "zero-copy" writes 2024-10-23 20:48:25 +03:00
Vitaliy Filippov 299578a3b3 Add notification ops and conversions 2024-09-09 16:57:19 +03:00
Vitaliy Filippov 5231edee52 Add PollOp 2024-09-09 16:53:58 +03:00
Vitaliy Filippov 9afe460f00 Add missing poll and notify request structures 2024-09-09 16:52:33 +03:00
Michael Stapelberg a1c7c8268f run gofmt (not sure why CI didn’t run for PR #160) 2024-09-09 15:00:01 +02:00
Коренберг Марк 80d6127f10
Add SyncFS support (#160) 2024-09-09 14:58:51 +02:00
labulakalia bc4b1b58bb MountConfig: add FuseImpl field to select macOS FUSE implementation 2024-09-09 14:50:23 +02:00
Michael Stapelberg faebccf4c6 directmount: take fsname from opts, not cfg
This makes the directmount code path pick up this workaround,
which fixes mounting jacobsa/fuse based file systems (which
do not set the FSName explicitly, like the samples) in Docker:

dd426a02d6/mount_config.go (L208-L220)

fixes https://github.com/jacobsa/fuse/issues/166
2024-07-21 10:39:25 +02:00
Aaron Jacobs dd426a02d6 fuseops/ops.go: run gofmt. 2024-07-21 15:18:49 +10:00
Aaron Jacobs 5559e12766 Migrate goo.gl links to TinyURL.
Because Google can't ever commit to anything, they are
[shutting down](https://developers.googleblog.com/en/google-url-shortener-links-will-no-longer-be-available/)
`goo.gl` short link redirects after
[previously announcing](https://developers.googleblog.com/en/transitioning-google-url-shortener-to-firebase-dynamic-links/)
that they'll continue to work.

They also previously announced that you could export your links from "the
`goo.gl` console", but I can't find that anywhere (the site is now just a
redirect to their stupid blog post), so I had to do this by hand. There may be
some errors.
2024-07-21 15:00:45 +10:00
Prince Kumar 8a36813dc0
debug: print operation name, not just OK (#165) 2024-06-26 16:34:36 +02:00
pat-vish 7285af0d05
debug: add file handle to OpenFile and ReleaseFileHandle (#164) 2024-06-07 11:28:44 +02:00
Prince Kumar a2f23eec70
Expose config to cache ReadDir() response (#162) 2024-05-22 11:08:07 +02:00
Alex Fishman 39f95ce809 Ingore loopbackfs errors for special folders
Signed-off-by: Alex Fishman <alex@fuse-t.org>
2024-05-09 10:38:15 +02:00
Alex Fishman a385e05f09 Restrict the use of a write lock exclusively to the fuse-t platform.
Signed-off-by: Alex Fishman <alex@fuse-t.org>
2024-05-09 10:38:15 +02:00
dependabot[bot] edf18a690d
Bump golang.org/x/net from 0.17.0 to 0.23.0 (#159)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.17.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.17.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-19 19:18:48 +02:00
Ayush Sethi 84480b28de
Add PARALLEL_DIROPS FUSE option to jacobsa/fuse (#158) 2024-04-08 16:40:47 +02:00
Alex Fishman 2b0495a1d5 Use default settings for extended attributes support
Signed-off-by: Alex Fishman <alex@fuse-t.org>
2023-12-12 23:25:09 +01:00
Alex Fishman 6031ecfb44 Daemonize fuse-t
Signed-off-by: Alex Fishman <alex@fuse-t.org>
2023-12-12 23:25:09 +01:00
Alex Fishman 8e7cb84040 Replace ReadAll with io.ReadFull
Signed-off-by: Alex Fishman <alex@fuse-t.org>
2023-12-12 23:25:09 +01:00
Alex Fishman 729d9bae15 Make memfs compatible with FUSE-T 2023-12-12 23:25:09 +01:00
Alex Fishman d873f3245d Make loopbackfs compatible with FUSE-T 2023-12-12 23:25:09 +01:00
Alex Fishman 3b4e7be609 make hellofs sample compatible with FUSE-T 2023-12-12 23:25:09 +01:00
Alex Fishman 08e4247097 lower support protocol minor versionl 2023-12-12 23:25:09 +01:00
Alex Fishman 81500b6e41 Make message read compatible with FUSE-T 2023-12-12 23:25:09 +01:00
Alex Fishman 8d8cb12b5b Add code to mount FUSE-T 2023-12-12 23:25:09 +01:00
Scott Bauersfeld cefeec4a40
Add support for callbacks in ReadFile and WriteFile operations (#147)
Co-authored-by: Scott Bauersfeld <sbauersfeld@databricks.com>
2023-12-12 21:30:19 +01:00
dependabot[bot] 40a7e03ed1
Bump golang.org/x/net from 0.7.0 to 0.17.0 (#152)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.7.0 to 0.17.0.
- [Commits](https://github.com/golang/net/compare/v0.7.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-12 08:08:20 +02:00
Prince Kumar d0f3daf365
Return errors during initOp response to kernel (#151) 2023-10-03 15:28:04 +02:00
Tulsi Shah ab21db1af8
fusetesting: fix compilation on arm64 where syscall.Stat_t is a uint32 (#149) 2023-08-10 15:47:08 +02:00
Vaishali Kumanan b8484ee15d
Expose UID in the OpContext (#145) 2023-06-24 18:14:25 +02:00
Prince Kumar 7263f3a2b4
Suppress logging errors in case of function not implemented (#142) 2023-05-09 11:03:21 +02:00
Tulsishah b7182e0d0b
updating go version 1.20 (#141) 2023-04-25 14:01:56 +02:00
Vitaliy Filippov 28052ba41f Publish file mode conversion functions (useful for file systems implementing special files) 2023-04-02 19:15:23 +02:00
Vitaliy Filippov 043fbd1f54 Add device number support 2023-04-02 19:15:23 +02:00
dependabot[bot] 86031ac261
Bump golang.org/x/net from 0.0.0-20220526153639-5463443f8c37 to 0.7.0 (#138)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.0.0-20220526153639-5463443f8c37 to 0.7.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/commits/v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-25 16:52:27 +01:00
Doug Schaapveld 702f658418 Add FuseID/Unique Op ID to OpContext 2023-02-18 18:45:05 +01:00
Doug Schaapveld bb43c71d6f Fix typo 2023-02-18 18:45:05 +01:00
Ayush Sethi 5e0f2e6b43
Upgrading go from 1.18 to 1.19 (#136) 2023-01-24 17:41:09 +01:00
Lars Gohr c2e09611e9
Make tests work with macfuse, too (#135) 2022-12-20 08:53:24 +01:00
Eric Gouyer a4cd154343 add support for sticky bit on directories 2022-10-16 10:46:58 +02:00
Eric Gouyer 5e958a41f6 add support for retrieving UID GID PID for each fuseops, ala FUSE-C fuse_get_context() 2022-10-16 10:46:58 +02:00
Eric Gouyer 2681cd5156 add support for UID and GID in SetInodeAttributes 2022-10-16 10:46:58 +02:00
Doug Schaapveld 63437da750
Remove PID zero condition from sample memfs (#131) 2022-09-27 17:14:28 +02:00
Avi c62d7682a6
Add more debugging logs to the mounting process so we get more visibility for customer issues (#130) 2022-09-06 08:54:02 +02:00
Michael Stapelberg 4e67748df3 GitHub Actions: use Go 1.19 2022-09-06 08:43:12 +02:00
Michael Stapelberg 66d6bd9e7b gofmt with Go 1.19 2022-09-06 08:42:56 +02:00
Mei Gui 226fec2ce9
Pass OpenFlags for OpenFileOp followups (#129) 2022-07-26 09:34:00 +02:00
Mei Gui 9cc4ff0bc9
Pass OpenFlags for OpenFileOp (#127) 2022-07-18 13:15:27 +02:00
Vitaliy Filippov 13117049f3
Fallback from BatchForgetOp to a series of ForgetInodeOp if unsupported (#126)
Co-authored-by: Vitaliy Filippov <vitalif@yourcmc.ru>
2022-07-02 11:18:25 +02:00
Ben Linsay 21122235c7
allow passing open /dev/fuse file descriptors (#124)
allows passing open /dev/fuse file descriptors so that the FUSE process
can run fully unprivileged. uses the /dev/fd/N mountpoint format from
libfuse3.
2022-05-31 22:22:54 +02:00
Doychin Atanasov 37d63df227
Add support for the FUSE_BATCH_FORGET operation (#123)
There are certain Kernel versions which do not send individual
Forget operations after they receive "not implemented" for Batch Forget.
One example is "5.4.0-110-generic" on Ubuntu 20.04. I am sure there are
plenty of others.

This leads to inode "leaks". Where reference counts are never decreased
and the inodes were left hanging around long after they are not needed
any more.

The best way to fix that was adding support for batch operations to the
lib. This way all users will be able to benefit from the batching
optimization.

Co-authored-by: Doychin Atanasov <doychin.atanasov@chaosgroup.com>
2022-05-27 08:49:15 +02:00
Michael Stapelberg 468f285a46 update go.mod to pull in latest x/sys and x/net
Hopefully this fixes CI failures on mac
2022-05-27 08:33:30 +02:00
Michael Stapelberg 48612565d5 GitHub Actions: upgrade to latest stable version of Go 1.17 2022-03-03 09:31:36 +01:00
Michael Stapelberg 1c9fe7bc84 remove memclr/memmove entirely now that they are unused
This breaks with newer Go versions AFAICT, so it’s easier to just remove it entirely.
2022-03-03 09:28:15 +01:00
Vitaliy Filippov 84920d11dd Add vectored read to readbenchfs
You can now run `./readbenchfs --mount_point dir --vectored` and then
`dd if=dir/test of=/dev/null iflag=direct bs=1M status=progress` to test
vectored read speed.

Results from my laptop (Linux 5.10):

1 core (GOMAXPROCS=1):
- Before vectored read patch: 2.1 GB/s
- Non-vectored read after vectored read patch: 2.1 GB/s
- Vectored read: 2.8 GB/s

All cores:
- Before vectored read patch: 3.0 GB/s
- Non-vectored read after vectored read patch: 3.3 GB/s
- Vectored read: 5.9 GB/s
2022-03-03 09:25:18 +01:00
Vitaliy Filippov c818f6216b Implement vectored read support
Read requests can now take vectored responses from the filesystem
implementation and send them to FUSE device via the writev() system call.

This allows file systems to send data without copying it into the
library-provided buffer if the data is already in memory.
2022-03-03 09:25:18 +01:00
Vitaliy Filippov da71c70600 Add ReadBenchFS to test linear read speed 2022-03-03 09:25:18 +01:00
Jakob Waibel c4197873da
Create mount_memfs sample (#121) 2022-02-14 20:12:19 +01:00
50 changed files with 2696 additions and 830 deletions

View File

@ -17,7 +17,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v2.1.4
with:
go-version: 1.16
go-version: ^1.19
id: go
# Check the codebase has been formatted by `gofmt`.
# We run two `gofmt` commands for different purposes:
@ -38,7 +38,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v2.1.4
with:
go-version: 1.16
go-version: ^1.19
id: go
- name: Install fuse
run: sudo apt-get update && sudo apt-get install -y fuse3 libfuse-dev
@ -56,7 +56,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v2.1.4
with:
go-version: 1.16
go-version: ^1.19
id: go
- name: Install macfuse
run: HOMEBREW_NO_AUTO_UPDATE=1 brew install macfuse

View File

@ -39,17 +39,19 @@ var contextKey interface{} = contextKeyType(0)
//
// As of 2015-03-26, the behavior in the kernel is:
//
// * (http://goo.gl/bQ1f1i, http://goo.gl/HwBrR6) Set the local variable
// ra_pages to be init_response->max_readahead divided by the page size.
// - (https://tinyurl.com/2eakn5e9, https://tinyurl.com/mry9e33d) Set the
// local variable ra_pages to be init_response->max_readahead divided by
// the page size.
//
// * (http://goo.gl/gcIsSh, http://goo.gl/LKV2vA) Set
// backing_dev_info::ra_pages to the min of that value and what was sent
// in the request's max_readahead field.
// - (https://tinyurl.com/2eakn5e9, https://tinyurl.com/mbpshk8h) Set
// backing_dev_info::ra_pages to the min of that value and what was sent in
// the request's max_readahead field.
//
// * (http://goo.gl/u2SqzH) Use backing_dev_info::ra_pages when deciding
// how much to read ahead.
// - (https://tinyurl.com/57hpfu4x) Use backing_dev_info::ra_pages when
// deciding how much to read ahead.
//
// * (http://goo.gl/JnhbdL) Don't read ahead at all if that field is zero.
// - (https://tinyurl.com/ywhfcfte) Don't read ahead at all if that field is
// zero.
//
// Reading a page at a time is a drag. Ask for a larger size.
const maxReadahead = 1 << 20
@ -151,6 +153,7 @@ func (c *Connection) Init() error {
cacheSymlinks := initOp.Flags&fusekernel.InitCacheSymlinks > 0
noOpenSupport := initOp.Flags&fusekernel.InitNoOpenSupport > 0
noOpendirSupport := initOp.Flags&fusekernel.InitNoOpendirSupport > 0
readdirplusSupport := initOp.Flags&fusekernel.InitDoReaddirplus > 0
// Respond to the init op.
initOp.Library = c.protocol
@ -193,8 +196,17 @@ func (c *Connection) Init() error {
initOp.Flags |= fusekernel.InitNoOpendirSupport
}
c.Reply(ctx, nil)
return nil
// Tell the Kernel to allow sending parallel lookup and readdir operations.
if c.cfg.EnableParallelDirOps {
initOp.Flags |= fusekernel.InitParallelDirOps
}
// Tell the kernel to do readdirplus (readdir+lookup in one call)
if c.cfg.UseReadDirPlus && readdirplusSupport {
initOp.Flags |= fusekernel.InitDoReaddirplus
}
return c.Reply(ctx, nil)
}
// Log information for an operation with the given ID. calldepth is the depth
@ -309,13 +321,13 @@ func (c *Connection) handleInterrupt(fuseID uint64) {
defer c.mu.Unlock()
// NOTE(jacobsa): fuse.txt in the Linux kernel documentation
// (https://goo.gl/H55Dnr) defines the kernel <-> userspace protocol for
// interrupts.
// (https://tinyurl.com/2r4ajuwd) defines the kernel <-> userspace protocol
// for interrupts.
//
// In particular, my reading of it is that an interrupt request cannot be
// delivered to userspace before the original request. The part about the
// race and EAGAIN appears to be aimed at userspace programs that
// concurrently process requests (cf. http://goo.gl/BES2rs).
// concurrently process requests (https://tinyurl.com/3euehwfb).
//
// So in this method if we can't find the ID to be interrupted, it means that
// the request has already been replied to.
@ -338,7 +350,7 @@ func (c *Connection) readMessage() (*buffer.InMessage, error) {
// Loop past transient errors.
for {
// Attempt a reaed.
// Attempt a read.
err := m.Init(c.dev)
// Special cases:
@ -369,18 +381,33 @@ func (c *Connection) readMessage() (*buffer.InMessage, error) {
}
// Write the supplied message to the kernel.
func (c *Connection) writeMessage(msg []byte) error {
// Avoid the retry loop in os.File.Write.
n, err := syscall.Write(int(c.dev.Fd()), msg)
func (c *Connection) writeMessage(outMsg *buffer.OutMessage) error {
var err error
var n int
expectedLen := outMsg.Len()
if outMsg.Sglist != nil {
if fusekernel.IsPlatformFuseT {
// writev is not atomic on macos, restrict to fuse-t platform
writeLock.Lock()
defer writeLock.Unlock()
}
n, err = writev(int(c.dev.Fd()), outMsg.Sglist)
} else {
// Avoid the retry loop in os.File.Write.
n, err = syscall.Write(int(c.dev.Fd()), outMsg.OutHeaderBytes())
}
if err == nil && n != expectedLen {
err = fmt.Errorf("Wrote %d bytes; expected %d", n, expectedLen)
}
if err != nil {
return err
writeErrMsg := fmt.Sprintf("writeMessage: %v %v", err, outMsg.OutHeaderBytes())
if c.errorLogger != nil {
c.errorLogger.Print(writeErrMsg)
}
return fmt.Errorf(writeErrMsg)
}
if n != len(msg) {
return fmt.Errorf("Wrote %d bytes; expected %d", n, len(msg))
}
return nil
outMsg.Sglist = nil
return err
}
// ReadOp consumes the next op from the kernel process, returning the op and a
@ -405,7 +432,7 @@ func (c *Connection) ReadOp() (_ context.Context, op interface{}, _ error) {
// Convert the message to an op.
outMsg := c.getOutMessage()
op, err = convertInMessage(inMsg, outMsg, c.protocol)
op, err = convertInMessage(&c.cfg, inMsg, outMsg, c.protocol)
if err != nil {
c.putOutMessage(outMsg)
return nil, nil, fmt.Errorf("convertInMessage: %v", err)
@ -454,7 +481,7 @@ func (c *Connection) shouldLogError(
return false
}
case *fuseops.GetXattrOp, *fuseops.ListXattrOp:
if err == syscall.ENODATA || err == syscall.ERANGE {
if err == syscall.ENOSYS || err == syscall.ENODATA || err == syscall.ERANGE {
return false
}
case *unknownOp:
@ -467,11 +494,13 @@ func (c *Connection) shouldLogError(
return true
}
var writeLock sync.Mutex
// Reply replies to an op previously read using ReadOp, with the supplied error
// (or nil if successful). The context must be the context returned by ReadOp.
//
// LOCKS_EXCLUDED(c.mu)
func (c *Connection) Reply(ctx context.Context, opErr error) {
func (c *Connection) Reply(ctx context.Context, opErr error) error {
// Extract the state we stuffed in earlier.
var key interface{} = contextKey
foo := ctx.Value(key)
@ -485,9 +514,25 @@ func (c *Connection) Reply(ctx context.Context, opErr error) {
outMsg := state.outMsg
fuseID := inMsg.Header().Unique
// Make sure we destroy the messages when we're done.
defer c.putInMessage(inMsg)
defer c.putOutMessage(outMsg)
defer func() {
// Invoke any callbacks set by the FUSE server after the response to the kernel is
// complete and before the inMessage and outMessage memory buffers have been freed.
callback := c.callbackForOp(op)
if callback != nil {
callback()
}
// Make sure we destroy the messages when we're done.
suppressReuse := false
if wr, ok := op.(*fuseops.WriteFileOp); ok {
suppressReuse = wr.SuppressReuse
}
if !suppressReuse {
c.putInMessage(inMsg)
}
c.putOutMessage(outMsg)
}()
// Clean up state for this op.
c.finishOp(inMsg.Header().Opcode, inMsg.Header().Unique)
@ -495,7 +540,7 @@ func (c *Connection) Reply(ctx context.Context, opErr error) {
// Debug logging
if c.debugLogger != nil {
if opErr == nil {
c.debugLog(fuseID, 1, "-> OK (%s)", describeResponse(op))
c.debugLog(fuseID, 1, "-> %s", describeResponse(op))
} else {
c.debugLog(fuseID, 1, "-> Error: %q", opErr.Error())
}
@ -510,11 +555,31 @@ func (c *Connection) Reply(ctx context.Context, opErr error) {
noResponse := c.kernelResponse(outMsg, inMsg.Header().Unique, op, opErr)
if !noResponse {
err := c.writeMessage(outMsg.Bytes())
if err != nil && c.errorLogger != nil {
c.errorLogger.Printf("writeMessage: %v %v", err, outMsg.Bytes())
}
c.writeMessage(outMsg)
}
return nil
}
func (c *Connection) callbackForOp(op interface{}) func() {
switch o := op.(type) {
case *fuseops.ReadFileOp:
return o.Callback
case *fuseops.WriteFileOp:
return o.Callback
}
return nil
}
// Send a notification to the kernel
// notification must be a pointer to one of fuseops.NotifyXXX structures
// To avoid a deadlock notifications must not be called in the execution path of a related filesytem operation or within any code that could hold a lock that could be needed to execute such an operation. As of kernel 4.18, a "related operation" is a lookup(), symlink(), mknod(), mkdir(), unlink(), rename(), link() or create() request for the parent, and a setattr(), unlink(), rmdir(), rename(), setxattr(), removexattr(), readdir() or readdirplus() request for the inode itself.
func (c *Connection) Notify(notification interface{}) error {
outMsg := c.getOutMessage()
defer c.putOutMessage(outMsg)
c.kernelNotification(outMsg, notification)
outMsg.OutHeader().Len = uint32(outMsg.Len())
return c.writeMessage(outMsg)
}
// Close the connection. Must not be called until operations that were read

View File

@ -38,6 +38,7 @@ import (
//
// The caller is responsible for arranging for the message to be destroyed.
func convertInMessage(
config *MountConfig,
inMsg *buffer.InMessage,
outMsg *buffer.OutMessage,
protocol fusekernel.Protocol) (o interface{}, err error) {
@ -50,15 +51,25 @@ func convertInMessage(
}
o = &fuseops.LookUpInodeOp{
Parent: fuseops.InodeID(inMsg.Header().Nodeid),
Name: string(buf[:n-1]),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
Parent: fuseops.InodeID(inMsg.Header().Nodeid),
Name: string(buf[:n-1]),
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpGetattr:
o = &fuseops.GetInodeAttributesOp{
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpSetattr:
@ -69,18 +80,31 @@ func convertInMessage(
}
to := &fuseops.SetInodeAttributesOp{
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
o = to
valid := fusekernel.SetattrValid(in.Valid)
if valid&fusekernel.SetattrUid != 0 {
to.Uid = &in.Uid
}
if valid&fusekernel.SetattrGid != 0 {
to.Gid = &in.Gid
}
if valid&fusekernel.SetattrSize != 0 {
to.Size = &in.Size
}
if valid&fusekernel.SetattrMode != 0 {
mode := convertFileMode(in.Mode)
mode := fuseops.ConvertFileMode(in.Mode)
to.Mode = &mode
}
@ -107,9 +131,45 @@ func convertInMessage(
}
o = &fuseops.ForgetInodeOp{
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
N: in.Nlookup,
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
N: in.Nlookup,
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpBatchForget:
type input fusekernel.BatchForgetCountIn
in := (*input)(inMsg.Consume(unsafe.Sizeof(input{})))
if in == nil {
return nil, errors.New("Corrupt OpBatchForget")
}
entries := make([]fuseops.BatchForgetEntry, 0, in.Count)
for i := uint32(0); i < in.Count; i++ {
type entry fusekernel.BatchForgetEntryIn
ein := (*entry)(inMsg.Consume(unsafe.Sizeof(entry{})))
if ein == nil {
return nil, errors.New("Corrupt OpBatchForget")
}
entries = append(entries, fuseops.BatchForgetEntry{
Inode: fuseops.InodeID(ein.Inode),
N: ein.Nlookup,
})
}
o = &fuseops.BatchForgetOp{
Entries: entries,
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpMkdir:
@ -130,13 +190,18 @@ func convertInMessage(
Name: string(name),
// On Linux, vfs_mkdir calls through to the inode with at most
// permissions and sticky bits set (cf. https://goo.gl/WxgQXk), and fuse
// passes that on directly (cf. https://goo.gl/f31aMo). In other words,
// the fact that this is a directory is implicit in the fact that the
// opcode is mkdir. But we want the correct mode to go through, so ensure
// that os.ModeDir is set.
Mode: convertFileMode(in.Mode) | os.ModeDir,
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
// permissions and sticky bits set (https://tinyurl.com/3djx8498), and
// fuse passes that on directly (https://tinyurl.com/exezw647). In other
// words, the fact that this is a directory is implicit in the fact that
// the opcode is mkdir. But we want the correct mode to go through, so
// ensure that os.ModeDir is set.
Mode: fuseops.ConvertFileMode(in.Mode) | os.ModeDir,
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpMknod:
@ -153,10 +218,16 @@ func convertInMessage(
name = name[:i]
o = &fuseops.MkNodeOp{
Parent: fuseops.InodeID(inMsg.Header().Nodeid),
Name: string(name),
Mode: convertFileMode(in.Mode),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
Parent: fuseops.InodeID(inMsg.Header().Nodeid),
Name: string(name),
Mode: fuseops.ConvertFileMode(in.Mode),
Rdev: in.Rdev,
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpCreate:
@ -173,10 +244,15 @@ func convertInMessage(
name = name[:i]
o = &fuseops.CreateFileOp{
Parent: fuseops.InodeID(inMsg.Header().Nodeid),
Name: string(name),
Mode: convertFileMode(in.Mode),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
Parent: fuseops.InodeID(inMsg.Header().Nodeid),
Name: string(name),
Mode: fuseops.ConvertFileMode(in.Mode),
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpSymlink:
@ -192,10 +268,15 @@ func convertInMessage(
newName, target := names[0:i], names[i+1:len(names)-1]
o = &fuseops.CreateSymlinkOp{
Parent: fuseops.InodeID(inMsg.Header().Nodeid),
Name: string(newName),
Target: string(target),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
Parent: fuseops.InodeID(inMsg.Header().Nodeid),
Name: string(newName),
Target: string(target),
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpRename:
@ -237,7 +318,12 @@ func convertInMessage(
OldName: string(oldName),
NewParent: fuseops.InodeID(in.Newdir),
NewName: string(newName),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpUnlink:
@ -248,9 +334,14 @@ func convertInMessage(
}
o = &fuseops.UnlinkOp{
Parent: fuseops.InodeID(inMsg.Header().Nodeid),
Name: string(buf[:n-1]),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
Parent: fuseops.InodeID(inMsg.Header().Nodeid),
Name: string(buf[:n-1]),
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpRmdir:
@ -261,21 +352,43 @@ func convertInMessage(
}
o = &fuseops.RmDirOp{
Parent: fuseops.InodeID(inMsg.Header().Nodeid),
Name: string(buf[:n-1]),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
Parent: fuseops.InodeID(inMsg.Header().Nodeid),
Name: string(buf[:n-1]),
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpOpen:
type input fusekernel.OpenIn
in := (*input)(inMsg.Consume(unsafe.Sizeof(input{})))
if in == nil {
return nil, errors.New("Corrupt OpOpen")
}
o = &fuseops.OpenFileOp{
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
OpenFlags: fusekernel.OpenFlags(in.Flags),
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpOpendir:
o = &fuseops.OpenDirOp{
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpRead:
@ -285,24 +398,26 @@ func convertInMessage(
}
to := &fuseops.ReadFileOp{
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
Handle: fuseops.HandleID(in.Fh),
Offset: int64(in.Offset),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
Handle: fuseops.HandleID(in.Fh),
Offset: int64(in.Offset),
Size: int64(in.Size),
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
if !config.UseVectoredRead {
// Use part of the incoming message storage as the read buffer
// For vectored zero-copy reads, don't allocate any buffers
to.Dst = inMsg.GetFree(int(in.Size))
}
o = to
readSize := int(in.Size)
p := outMsg.GrowNoZero(readSize)
if p == nil {
return nil, fmt.Errorf("Can't grow for %d-byte read", readSize)
}
sh := (*reflect.SliceHeader)(unsafe.Pointer(&to.Dst))
sh.Data = uintptr(p)
sh.Len = readSize
sh.Cap = readSize
case fusekernel.OpReaddirplus:
fallthrough
case fusekernel.OpReaddir:
in := (*fusekernel.ReadIn)(inMsg.Consume(fusekernel.ReadInSize(protocol)))
if in == nil {
@ -310,15 +425,21 @@ func convertInMessage(
}
to := &fuseops.ReadDirOp{
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
Handle: fuseops.HandleID(in.Fh),
Offset: fuseops.DirOffset(in.Offset),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
Handle: fuseops.HandleID(in.Fh),
Offset: fuseops.DirOffset(in.Offset),
Plus: inMsg.Header().Opcode == fusekernel.OpReaddirplus,
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
o = to
readSize := int(in.Size)
p := outMsg.GrowNoZero(readSize)
p := outMsg.Grow(readSize)
if p == nil {
return nil, fmt.Errorf("Can't grow for %d-byte read", readSize)
}
@ -336,8 +457,13 @@ func convertInMessage(
}
o = &fuseops.ReleaseFileHandleOp{
Handle: fuseops.HandleID(in.Fh),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
Handle: fuseops.HandleID(in.Fh),
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpReleasedir:
@ -348,8 +474,13 @@ func convertInMessage(
}
o = &fuseops.ReleaseDirHandleOp{
Handle: fuseops.HandleID(in.Fh),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
Handle: fuseops.HandleID(in.Fh),
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpWrite:
@ -364,23 +495,45 @@ func convertInMessage(
}
o = &fuseops.WriteFileOp{
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
Handle: fuseops.HandleID(in.Fh),
Data: buf,
Offset: int64(in.Offset),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
Handle: fuseops.HandleID(in.Fh),
Data: buf,
Offset: int64(in.Offset),
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpFsync, fusekernel.OpFsyncdir:
type input fusekernel.FsyncIn
in := (*input)(inMsg.Consume(unsafe.Sizeof(input{})))
if in == nil {
return nil, errors.New("Corrupt OpFsync")
return nil, errors.New("Corrupt OpFsync/OpFsyncdir")
}
o = &fuseops.SyncFileOp{
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
Handle: fuseops.HandleID(in.Fh),
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpSyncFS:
type input fusekernel.SyncFSIn
in := (*input)(inMsg.Consume(unsafe.Sizeof(input{})))
if in == nil {
return nil, errors.New("Corrupt OpSyncFS")
}
o = &fuseops.SyncFSOp{
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
Handle: fuseops.HandleID(in.Fh),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
}
@ -392,15 +545,25 @@ func convertInMessage(
}
o = &fuseops.FlushFileOp{
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
Handle: fuseops.HandleID(in.Fh),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
Handle: fuseops.HandleID(in.Fh),
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpReadlink:
o = &fuseops.ReadSymlinkOp{
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpStatfs:
@ -448,10 +611,15 @@ func convertInMessage(
}
o = &fuseops.CreateLinkOp{
Parent: fuseops.InodeID(inMsg.Header().Nodeid),
Name: string(name),
Target: fuseops.InodeID(in.Oldnodeid),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
Parent: fuseops.InodeID(inMsg.Header().Nodeid),
Name: string(name),
Target: fuseops.InodeID(in.Oldnodeid),
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpRemovexattr:
@ -462,9 +630,14 @@ func convertInMessage(
}
o = &fuseops.RemoveXattrOp{
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
Name: string(buf[:n-1]),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
Name: string(buf[:n-1]),
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpGetxattr:
@ -482,22 +655,31 @@ func convertInMessage(
name = name[:i]
to := &fuseops.GetXattrOp{
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
Name: string(name),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
Name: string(name),
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
o = to
readSize := int(in.Size)
p := outMsg.GrowNoZero(readSize)
if p == nil {
return nil, fmt.Errorf("Can't grow for %d-byte read", readSize)
}
if readSize > 0 {
p := outMsg.Grow(readSize)
if p == nil {
return nil, fmt.Errorf("Can't grow for %d-byte read", readSize)
}
sh := (*reflect.SliceHeader)(unsafe.Pointer(&to.Dst))
sh.Data = uintptr(p)
sh.Len = readSize
sh.Cap = readSize
sh := (*reflect.SliceHeader)(unsafe.Pointer(&to.Dst))
sh.Data = uintptr(p)
sh.Len = readSize
sh.Cap = readSize
} else {
to.Dst = nil
}
case fusekernel.OpListxattr:
type input fusekernel.ListxattrIn
@ -507,14 +689,19 @@ func convertInMessage(
}
to := &fuseops.ListXattrOp{
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
o = to
readSize := int(in.Size)
if readSize != 0 {
p := outMsg.GrowNoZero(readSize)
p := outMsg.Grow(readSize)
if p == nil {
return nil, fmt.Errorf("Can't grow for %d-byte read", readSize)
}
@ -543,11 +730,16 @@ func convertInMessage(
name, value := payload[:i], payload[i+1:len(payload)]
o = &fuseops.SetXattrOp{
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
Name: string(name),
Value: value,
Flags: in.Flags,
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
Name: string(name),
Value: value,
Flags: in.Flags,
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpFallocate:
type input fusekernel.FallocateIn
@ -557,11 +749,52 @@ func convertInMessage(
}
o = &fuseops.FallocateOp{
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
Handle: fuseops.HandleID(in.Fh),
Offset: in.Offset,
Length: in.Length,
Mode: in.Mode,
OpContext: fuseops.OpContext{
FuseID: inMsg.Header().Unique,
Pid: inMsg.Header().Pid,
Uid: inMsg.Header().Uid,
Gid: inMsg.Header().Gid,
},
}
case fusekernel.OpPoll:
type input fusekernel.PollIn
in := (*input)(inMsg.Consume(unsafe.Sizeof(input{})))
if in == nil {
return nil, errors.New("Corrupt OpPoll")
}
o = &fuseops.PollOp{
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
Handle: fuseops.HandleID(in.Fh),
Kh: in.Kh,
Flags: fusekernel.PollFlags(in.Flags),
Events: fusekernel.PollEvents(in.Events),
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
}
case fusekernel.OpNotifyReply:
type input fusekernel.NotifyRetrieveIn
in := (*input)(inMsg.Consume(unsafe.Sizeof(input{})))
if in == nil {
return nil, errors.New("Corrupt OpNotifyReply")
}
buf := inMsg.ConsumeBytes(inMsg.Len())
if len(buf) < int(in.Size) {
return nil, errors.New("Corrupt OpNotifyReply")
}
o = &fuseops.NotifyRetrieveReplyOp{
Inode: fuseops.InodeID(inMsg.Header().Nodeid),
Unique: inMsg.Header().Unique,
Offset: in.Offset,
Length: in.Length,
Mode: in.Mode,
Length: in.Size,
OpContext: fuseops.OpContext{Pid: inMsg.Header().Pid},
}
@ -595,6 +828,12 @@ func (c *Connection) kernelResponse(
case *fuseops.ForgetInodeOp:
return true
case *fuseops.BatchForgetOp:
return true
case *fuseops.NotifyRetrieveReplyOp:
return true
case *interruptOp:
return true
}
@ -638,37 +877,37 @@ func (c *Connection) kernelResponseForOp(
case *fuseops.LookUpInodeOp:
size := int(fusekernel.EntryOutSize(c.protocol))
out := (*fusekernel.EntryOut)(m.Grow(size))
convertChildInodeEntry(&o.Entry, out)
fuseops.ConvertChildInodeEntry(&o.Entry, out)
case *fuseops.GetInodeAttributesOp:
size := int(fusekernel.AttrOutSize(c.protocol))
out := (*fusekernel.AttrOut)(m.Grow(size))
out.AttrValid, out.AttrValidNsec = convertExpirationTime(
out.AttrValid, out.AttrValidNsec = fuseops.ConvertExpirationTime(
o.AttributesExpiration)
convertAttributes(o.Inode, &o.Attributes, &out.Attr)
fuseops.ConvertAttributes(o.Inode, &o.Attributes, &out.Attr)
case *fuseops.SetInodeAttributesOp:
size := int(fusekernel.AttrOutSize(c.protocol))
out := (*fusekernel.AttrOut)(m.Grow(size))
out.AttrValid, out.AttrValidNsec = convertExpirationTime(
out.AttrValid, out.AttrValidNsec = fuseops.ConvertExpirationTime(
o.AttributesExpiration)
convertAttributes(o.Inode, &o.Attributes, &out.Attr)
fuseops.ConvertAttributes(o.Inode, &o.Attributes, &out.Attr)
case *fuseops.MkDirOp:
size := int(fusekernel.EntryOutSize(c.protocol))
out := (*fusekernel.EntryOut)(m.Grow(size))
convertChildInodeEntry(&o.Entry, out)
fuseops.ConvertChildInodeEntry(&o.Entry, out)
case *fuseops.MkNodeOp:
size := int(fusekernel.EntryOutSize(c.protocol))
out := (*fusekernel.EntryOut)(m.Grow(size))
convertChildInodeEntry(&o.Entry, out)
fuseops.ConvertChildInodeEntry(&o.Entry, out)
case *fuseops.CreateFileOp:
eSize := int(fusekernel.EntryOutSize(c.protocol))
e := (*fusekernel.EntryOut)(m.Grow(eSize))
convertChildInodeEntry(&o.Entry, e)
fuseops.ConvertChildInodeEntry(&o.Entry, e)
oo := (*fusekernel.OpenOut)(m.Grow(int(unsafe.Sizeof(fusekernel.OpenOut{}))))
oo.Fh = uint64(o.Handle)
@ -676,12 +915,12 @@ func (c *Connection) kernelResponseForOp(
case *fuseops.CreateSymlinkOp:
size := int(fusekernel.EntryOutSize(c.protocol))
out := (*fusekernel.EntryOut)(m.Grow(size))
convertChildInodeEntry(&o.Entry, out)
fuseops.ConvertChildInodeEntry(&o.Entry, out)
case *fuseops.CreateLinkOp:
size := int(fusekernel.EntryOutSize(c.protocol))
out := (*fusekernel.EntryOut)(m.Grow(size))
convertChildInodeEntry(&o.Entry, out)
fuseops.ConvertChildInodeEntry(&o.Entry, out)
case *fuseops.RenameOp:
// Empty response
@ -696,6 +935,14 @@ func (c *Connection) kernelResponseForOp(
out := (*fusekernel.OpenOut)(m.Grow(int(unsafe.Sizeof(fusekernel.OpenOut{}))))
out.Fh = uint64(o.Handle)
if o.CacheDir {
out.OpenFlags |= uint32(fusekernel.OpenCacheDir)
}
if o.KeepCache {
out.OpenFlags |= uint32(fusekernel.OpenKeepCache)
}
case *fuseops.ReadDirOp:
// convertInMessage already set up the destination buffer to be at the end
// of the out message. We need only shrink to the right size based on how
@ -718,9 +965,11 @@ func (c *Connection) kernelResponseForOp(
}
case *fuseops.ReadFileOp:
// convertInMessage already set up the destination buffer to be at the end
// of the out message. We need only shrink to the right size based on how
// much the user read.
if o.Dst != nil {
m.Append(o.Dst)
} else {
m.Append(o.Data...)
}
m.ShrinkTo(buffer.OutMessageHeaderSize + o.BytesRead)
case *fuseops.WriteFileOp:
@ -748,7 +997,7 @@ func (c *Connection) kernelResponseForOp(
out.St.Ffree = o.InodesFree
out.St.Namelen = 255
// The posix spec for sys/statvfs.h (http://goo.gl/LktgrF) defines the
// The posix spec for sys/statvfs.h (https://tinyurl.com/2juj6ah6) defines the
// following fields of statvfs, among others:
//
// f_bsize File system block size.
@ -758,7 +1007,7 @@ func (c *Connection) kernelResponseForOp(
// It appears as though f_bsize was the only thing supported by most unixes
// originally, but then f_frsize was added when new sorts of file systems
// came about. Quoth The Linux Programming Interface by Michael Kerrisk
// (https://goo.gl/5LZMxQ):
// (https://tinyurl.com/5n8mjtws):
//
// For most Linux file systems, the values of f_bsize and f_frsize are
// the same. However, some file systems support the notion of block
@ -802,6 +1051,9 @@ func (c *Connection) kernelResponseForOp(
case *fuseops.FallocateOp:
// Empty response
case *fuseops.SyncFSOp:
// Empty response
case *initOp:
out := (*fusekernel.InitOut)(m.Grow(int(unsafe.Sizeof(fusekernel.InitOut{}))))
@ -816,6 +1068,13 @@ func (c *Connection) kernelResponseForOp(
out.TimeGran = 1
out.MaxPages = o.MaxPages
case *fuseops.PollOp:
out := (*fusekernel.PollOut)(m.Grow(int(unsafe.Sizeof(fusekernel.PollOut{}))))
out.Revents = uint32(o.Revents)
case *fuseops.NotifyRetrieveReplyOp:
// Empty response
default:
panic(fmt.Sprintf("Unexpected op: %#v", op))
}
@ -823,115 +1082,73 @@ func (c *Connection) kernelResponseForOp(
return
}
// Like kernelResponse, but assumes the user replied with a nil error to the op.
func (c *Connection) kernelNotification(
m *buffer.OutMessage,
op interface{}) {
h := m.OutHeader()
h.Unique = 0
// Create the appropriate output message
switch o := op.(type) {
case *fuseops.NotifyPollWakeup:
h.Error = fusekernel.NotifyCodePoll
out := (*fusekernel.NotifyPollWakeupOut)(m.Grow(int(unsafe.Sizeof(fusekernel.NotifyPollWakeupOut{}))))
out.Kh = o.Kh
case *fuseops.NotifyInvalInode:
h.Error = fusekernel.NotifyCodeInvalInode
out := (*fusekernel.NotifyInvalInodeOut)(m.Grow(int(unsafe.Sizeof(fusekernel.NotifyInvalInodeOut{}))))
out.Ino = uint64(o.Inode)
out.Off = o.Offset
out.Len = o.Length
case *fuseops.NotifyInvalEntry:
h.Error = fusekernel.NotifyCodeInvalEntry
out := (*fusekernel.NotifyInvalEntryOut)(m.Grow(int(unsafe.Sizeof(fusekernel.NotifyInvalEntryOut{}))))
out.Parent = uint64(o.Parent)
out.Namelen = uint32(len(o.Name))
m.AppendString(o.Name)
m.AppendString("\x00")
case *fuseops.NotifyDelete:
h.Error = fusekernel.NotifyCodeDelete
out := (*fusekernel.NotifyDeleteOut)(m.Grow(int(unsafe.Sizeof(fusekernel.NotifyDeleteOut{}))))
out.Parent = uint64(o.Parent)
out.Child = uint64(o.Child)
out.Namelen = uint32(len(o.Name))
m.AppendString(o.Name)
m.AppendString("\x00")
case *fuseops.NotifyStore:
h.Error = fusekernel.NotifyCodeStore
out := (*fusekernel.NotifyStoreOut)(m.Grow(int(unsafe.Sizeof(fusekernel.NotifyStoreOut{}))))
out.Nodeid = uint64(o.Inode)
out.Offset = o.Offset
out.Size = o.Length
m.Append(o.Data...)
m.ShrinkTo(buffer.OutMessageHeaderSize + int(unsafe.Sizeof(fusekernel.NotifyStoreOut{})) + int(o.Length))
case *fuseops.NotifyRetrieve:
h.Error = fusekernel.NotifyCodeRetrieve
out := (*fusekernel.NotifyRetrieveOut)(m.Grow(int(unsafe.Sizeof(fusekernel.NotifyRetrieveOut{}))))
out.Unique = o.Unique
out.Nodeid = uint64(o.Inode)
out.Offset = o.Offset
out.Size = o.Length
default:
panic(fmt.Sprintf("Unexpected notification: %#v", op))
}
return
}
////////////////////////////////////////////////////////////////////////
// General conversions
////////////////////////////////////////////////////////////////////////
func convertTime(t time.Time) (secs uint64, nsec uint32) {
totalNano := t.UnixNano()
secs = uint64(totalNano / 1e9)
nsec = uint32(totalNano % 1e9)
return secs, nsec
}
func convertAttributes(
inodeID fuseops.InodeID,
in *fuseops.InodeAttributes,
out *fusekernel.Attr) {
out.Ino = uint64(inodeID)
out.Size = in.Size
out.Atime, out.AtimeNsec = convertTime(in.Atime)
out.Mtime, out.MtimeNsec = convertTime(in.Mtime)
out.Ctime, out.CtimeNsec = convertTime(in.Ctime)
out.SetCrtime(convertTime(in.Crtime))
out.Nlink = in.Nlink
out.Uid = in.Uid
out.Gid = in.Gid
// round up to the nearest 512 boundary
out.Blocks = (in.Size + 512 - 1) / 512
// Set the mode.
out.Mode = uint32(in.Mode) & 0777
switch {
default:
out.Mode |= syscall.S_IFREG
case in.Mode&os.ModeDir != 0:
out.Mode |= syscall.S_IFDIR
case in.Mode&os.ModeDevice != 0:
if in.Mode&os.ModeCharDevice != 0 {
out.Mode |= syscall.S_IFCHR
} else {
out.Mode |= syscall.S_IFBLK
}
case in.Mode&os.ModeNamedPipe != 0:
out.Mode |= syscall.S_IFIFO
case in.Mode&os.ModeSymlink != 0:
out.Mode |= syscall.S_IFLNK
case in.Mode&os.ModeSocket != 0:
out.Mode |= syscall.S_IFSOCK
}
if in.Mode&os.ModeSetuid != 0 {
out.Mode |= syscall.S_ISUID
}
}
// Convert an absolute cache expiration time to a relative time from now for
// consumption by the fuse kernel module.
func convertExpirationTime(t time.Time) (secs uint64, nsecs uint32) {
// Fuse represents durations as unsigned 64-bit counts of seconds and 32-bit
// counts of nanoseconds (cf. http://goo.gl/EJupJV). So negative durations
// are right out. There is no need to cap the positive magnitude, because
// 2^64 seconds is well longer than the 2^63 ns range of time.Duration.
d := t.Sub(time.Now())
if d > 0 {
secs = uint64(d / time.Second)
nsecs = uint32((d % time.Second) / time.Nanosecond)
}
return secs, nsecs
}
func convertChildInodeEntry(
in *fuseops.ChildInodeEntry,
out *fusekernel.EntryOut) {
out.Nodeid = uint64(in.Child)
out.Generation = uint64(in.Generation)
out.EntryValid, out.EntryValidNsec = convertExpirationTime(in.EntryExpiration)
out.AttrValid, out.AttrValidNsec = convertExpirationTime(in.AttributesExpiration)
convertAttributes(in.Child, &in.Attributes, &out.Attr)
}
func convertFileMode(unixMode uint32) os.FileMode {
mode := os.FileMode(unixMode & 0777)
switch unixMode & syscall.S_IFMT {
case syscall.S_IFREG:
// nothing
case syscall.S_IFDIR:
mode |= os.ModeDir
case syscall.S_IFCHR:
mode |= os.ModeCharDevice | os.ModeDevice
case syscall.S_IFBLK:
mode |= os.ModeDevice
case syscall.S_IFIFO:
mode |= os.ModeNamedPipe
case syscall.S_IFLNK:
mode |= os.ModeSymlink
case syscall.S_IFSOCK:
mode |= os.ModeSocket
default:
// no idea
mode |= os.ModeDevice
}
if unixMode&syscall.S_ISUID != 0 {
mode |= os.ModeSetuid
}
if unixMode&syscall.S_ISGID != 0 {
mode |= os.ModeSetgid
}
return mode
}
func writeXattrSize(m *buffer.OutMessage, size uint32) {
out := (*fusekernel.GetxattrOut)(m.Grow(int(unsafe.Sizeof(fusekernel.GetxattrOut{}))))
out.Size = size

View File

@ -95,7 +95,7 @@ func describeRequest(op interface{}) (s string) {
case *fuseops.ReadFileOp:
addComponent("handle %d", typed.Handle)
addComponent("offset %d", typed.Offset)
addComponent("%d bytes", len(typed.Dst))
addComponent("%d bytes", typed.Size)
case *fuseops.WriteFileOp:
addComponent("handle %d", typed.Handle)
@ -115,6 +115,9 @@ func describeRequest(op interface{}) (s string) {
addComponent("offset %d", typed.Offset)
addComponent("length %d", typed.Length)
addComponent("mode %d", typed.Mode)
case *fuseops.ReleaseFileHandleOp:
addComponent("handle %d", typed.Handle)
}
// Use just the name if there is no extra info.
@ -141,6 +144,10 @@ func describeResponse(op interface{}) string {
addComponent("inode %v", entry.Child)
}
}
switch typed := op.(type) {
case *fuseops.OpenFileOp:
addComponent("handle %d", typed.Handle)
}
return fmt.Sprintf("%s", strings.Join(components, ", "))
return fmt.Sprintf("%s (%s)", opName(op), strings.Join(components, ", "))
}

8
doc.go
View File

@ -16,15 +16,15 @@
//
// The primary elements of interest are:
//
// * The fuseops package, which defines the operations that fuse might send
// - The fuseops package, which defines the operations that fuse might send
// to your userspace daemon.
//
// * The Server interface, which your daemon must implement.
// - The Server interface, which your daemon must implement.
//
// * fuseutil.NewFileSystemServer, which offers a convenient way to implement
// - fuseutil.NewFileSystemServer, which offers a convenient way to implement
// the Server interface.
//
// * Mount, a function that allows for mounting a Server as a file system.
// - Mount, a function that allows for mounting a Server as a file system.
//
// Make sure to see the examples in the sub-packages of samples/, which double
// as tests for this package: http://godoc.org/github.com/jacobsa/fuse/samples

85
fuseops/conv.go Normal file
View File

@ -0,0 +1,85 @@
// Copyright 2023 Vitaliy Filippov
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package fuseops
import (
"time"
"syscall"
"github.com/jacobsa/fuse/internal/fusekernel"
)
////////////////////////////////////////////////////////////////////////
// General conversions
////////////////////////////////////////////////////////////////////////
func ConvertTime(t time.Time) (secs uint64, nsec uint32) {
totalNano := t.UnixNano()
secs = uint64(totalNano / 1e9)
nsec = uint32(totalNano % 1e9)
return secs, nsec
}
func ConvertAttributes(
inodeID InodeID,
in *InodeAttributes,
out *fusekernel.Attr) {
out.Ino = uint64(inodeID)
out.Size = in.Size
out.Atime, out.AtimeNsec = ConvertTime(in.Atime)
out.Mtime, out.MtimeNsec = ConvertTime(in.Mtime)
out.Ctime, out.CtimeNsec = ConvertTime(in.Ctime)
out.SetCrtime(ConvertTime(in.Crtime))
out.Nlink = in.Nlink
out.Uid = in.Uid
out.Gid = in.Gid
// round up to the nearest 512 boundary
out.Blocks = (in.Size + 512 - 1) / 512
// Set the mode.
out.Mode = ConvertGoMode(in.Mode)
if out.Mode & (syscall.S_IFCHR | syscall.S_IFBLK) != 0 {
out.Rdev = in.Rdev
}
}
// Convert an absolute cache expiration time to a relative time from now for
// consumption by the fuse kernel module.
func ConvertExpirationTime(t time.Time) (secs uint64, nsecs uint32) {
// Fuse represents durations as unsigned 64-bit counts of seconds and 32-bit
// counts of nanoseconds (https://tinyurl.com/4muvkr6k). So negative
// durations are right out. There is no need to cap the positive magnitude,
// because 2^64 seconds is well longer than the 2^63 ns range of
// time.Duration.
d := t.Sub(time.Now())
if d > 0 {
secs = uint64(d / time.Second)
nsecs = uint32((d % time.Second) / time.Nanosecond)
}
return secs, nsecs
}
func ConvertChildInodeEntry(
in *ChildInodeEntry,
out *fusekernel.EntryOut) {
out.Nodeid = uint64(in.Child)
out.Generation = uint64(in.Generation)
out.EntryValid, out.EntryValidNsec = ConvertExpirationTime(in.EntryExpiration)
out.AttrValid, out.AttrValidNsec = ConvertExpirationTime(in.AttributesExpiration)
ConvertAttributes(in.Child, &in.Attributes, &out.Attr)
}

74
fuseops/filemode.go Normal file
View File

@ -0,0 +1,74 @@
package fuseops
import (
"os"
"syscall"
)
// ConvertFileMode returns an os.FileMode with the Go mode and permission bits
// set according to the Linux mode and permission bits.
func ConvertFileMode(unixMode uint32) os.FileMode {
mode := os.FileMode(unixMode & 0777)
switch unixMode & syscall.S_IFMT {
case syscall.S_IFREG:
// nothing
case syscall.S_IFDIR:
mode |= os.ModeDir
case syscall.S_IFCHR:
mode |= os.ModeCharDevice | os.ModeDevice
case syscall.S_IFBLK:
mode |= os.ModeDevice
case syscall.S_IFIFO:
mode |= os.ModeNamedPipe
case syscall.S_IFLNK:
mode |= os.ModeSymlink
case syscall.S_IFSOCK:
mode |= os.ModeSocket
default:
// no idea
}
if unixMode&syscall.S_ISUID != 0 {
mode |= os.ModeSetuid
}
if unixMode&syscall.S_ISGID != 0 {
mode |= os.ModeSetgid
}
if unixMode&syscall.S_ISVTX != 0 {
mode |= os.ModeSticky
}
return mode
}
// ConvertGoMode returns an integer with the Linux mode and permission bits
// set according to the Go mode and permission bits.
func ConvertGoMode(inMode os.FileMode) uint32 {
outMode := uint32(inMode) & 0777
switch {
default:
outMode |= syscall.S_IFREG
case inMode&os.ModeDir != 0:
outMode |= syscall.S_IFDIR
case inMode&os.ModeDevice != 0:
if inMode&os.ModeCharDevice != 0 {
outMode |= syscall.S_IFCHR
} else {
outMode |= syscall.S_IFBLK
}
case inMode&os.ModeNamedPipe != 0:
outMode |= syscall.S_IFIFO
case inMode&os.ModeSymlink != 0:
outMode |= syscall.S_IFLNK
case inMode&os.ModeSocket != 0:
outMode |= syscall.S_IFSOCK
}
if inMode&os.ModeSetuid != 0 {
outMode |= syscall.S_ISUID
}
if inMode&os.ModeSetgid != 0 {
outMode |= syscall.S_ISGID
}
if inMode&os.ModeSticky != 0 {
outMode |= syscall.S_ISVTX
}
return outMode
}

View File

@ -17,6 +17,8 @@ package fuseops
import (
"os"
"time"
"github.com/jacobsa/fuse/internal/fusekernel"
)
////////////////////////////////////////////////////////////////////////
@ -26,25 +28,35 @@ import (
// OpContext contains extra context that may be needed by some file systems.
// See https://libfuse.github.io/doxygen/structfuse__context.html as a reference.
type OpContext struct {
// FuseID is the Unique identifier for each operation from the kernel.
FuseID uint64
// PID of the process that is invoking the operation.
// Not filled in case of a writepage operation.
Pid uint32
// UID of the process that is invoking the operation.
// Not filled in case of a writepage operation.
Uid uint32
// GID of the process that is invoking the operation.
// Not filled in case of a writepage operation.
Gid uint32
}
// Return statistics about the file system's capacity and available resources.
//
// Called by statfs(2) and friends:
//
// * (https://goo.gl/Xi1lDr) sys_statfs called user_statfs, which calls
// vfs_statfs, which calls statfs_by_dentry.
// - (https://tinyurl.com/234ppacj) sys_statfs called user_statfs, which calls
// vfs_statfs, which calls statfs_by_dentry.
//
// * (https://goo.gl/VAIOwU) statfs_by_dentry calls the superblock
// operation statfs, which in our case points at
// fuse_statfs (cf. https://goo.gl/L7BTM3)
// - (https://tinyurl.com/u6keadjz) statfs_by_dentry calls the superblock
// operation statfs, which in our case points at
// fuse_statfs (https://tinyurl.com/mr45wd28)
//
// * (https://goo.gl/Zn7Sgl) fuse_statfs sends a statfs op, then uses
// convert_fuse_statfs to convert the response in a straightforward
// manner.
// - (https://tinyurl.com/3wt3dw3c) fuse_statfs sends a statfs op, then uses
// convert_fuse_statfs to convert the response in a straightforward manner.
//
// This op is particularly important on OS X: if you don't implement it, the
// file system will not successfully mount. If you don't model a sane amount of
@ -55,15 +67,15 @@ type StatFSOp struct {
// system's capacity and space availability.
//
// On Linux this is surfaced as statfs::f_frsize, matching the posix standard
// (http://goo.gl/LktgrF), which says that f_blocks and friends are in units
// of f_frsize. On OS X this is surfaced as statfs::f_bsize, which plays the
// same roll.
// (https://tinyurl.com/2juj6ah6), which says that f_blocks and friends are
// in units of f_frsize. On OS X this is surfaced as statfs::f_bsize, which
// plays the same roll.
//
// It appears as though the original intent of statvfs::f_frsize in the posix
// standard was to support a smaller addressable unit than statvfs::f_bsize
// (cf. The Linux Programming Interface by Michael Kerrisk,
// https://goo.gl/5LZMxQ). Therefore users should probably arrange for this
// to be no larger than IoSize.
// https://tinyurl.com/5n8mjtws). Therefore users should probably arrange for
// this to be no larger than IoSize.
//
// On Linux this can be any value, and will be faithfully returned to the
// caller of statfs(2) (see the code walk above). On OS X it appears that
@ -159,6 +171,8 @@ type SetInodeAttributesOp struct {
Handle *HandleID
// The attributes to modify, or nil for attributes that don't need a change.
Uid *uint32
Gid *uint32
Size *uint64
Mode *os.FileMode
Atime *time.Time
@ -179,20 +193,22 @@ type SetInodeAttributesOp struct {
// contain a note of this (but see also the note about the root inode below).
// For example, LookUpInodeOp and MkDirOp. The authoritative source is the
// libfuse documentation, which states that any op that returns
// fuse_reply_entry fuse_reply_create implicitly increments (cf.
// http://goo.gl/o5C7Dx).
// fuse_reply_entry fuse_reply_create implicitly increments
// (https://tinyurl.com/2xd5zssm).
//
// If the reference count hits zero, the file system can forget about that ID
// entirely, and even re-use it in future responses. The kernel guarantees that
// it will not otherwise use it again.
//
// The reference count corresponds to fuse_inode::nlookup
// (http://goo.gl/ut48S4). Some examples of where the kernel manipulates it:
// (https://tinyurl.com/ycka69ck). Some examples of where the kernel
// manipulates it:
//
// * (http://goo.gl/vPD9Oh) Any caller to fuse_iget increases the count.
// * (http://goo.gl/B6tTTC) fuse_lookup_name calls fuse_iget.
// * (http://goo.gl/IlcxWv) fuse_create_open calls fuse_iget.
// * (http://goo.gl/VQMQul) fuse_dentry_revalidate increments after
// - (https://tinyurl.com/s8dz2ays) Any caller to fuse_iget increases the
// count.
// - (https://tinyurl.com/mu37ceua) fuse_lookup_name calls fuse_iget.
// - (https://tinyurl.com/2nyhhnsh) fuse_create_open calls fuse_iget.
// - (https://tinyurl.com/mnjpu3a9) fuse_dentry_revalidate increments after
// revalidating.
//
// In contrast to all other inodes, RootInodeID begins with an implicit
@ -200,12 +216,13 @@ type SetInodeAttributesOp struct {
// could be no such op, because the root cannot be referred to by name.) Code
// walk:
//
// * (http://goo.gl/gWAheU) fuse_fill_super calls fuse_get_root_inode.
// - (https://tinyurl.com/yf8m2drx) fuse_fill_super calls
// fuse_get_root_inode.
//
// * (http://goo.gl/AoLsbb) fuse_get_root_inode calls fuse_iget without
// sending any particular request.
// - (https://tinyurl.com/35f86asu) fuse_get_root_inode calls fuse_iget
// without sending any particular request.
//
// * (http://goo.gl/vPD9Oh) fuse_iget increments nlookup.
// - (https://tinyurl.com/s8dz2ays) fuse_iget increments nlookup.
//
// File systems should tolerate but not rely on receiving forget ops for
// remaining inodes when the file system unmounts, including the root inode.
@ -220,6 +237,32 @@ type ForgetInodeOp struct {
OpContext OpContext
}
// BatchForgetEntry represents one Inode entry to forget in the BatchForgetOp.
//
// Everything written in the ForgetInodeOp docs applies for the BatchForgetEntry
// too.
type BatchForgetEntry struct {
// The inode whose reference count should be decremented.
Inode InodeID
// The amount to decrement the reference count.
N uint64
}
// Decrement the reference counts for a list of inode IDs previously issued by the file
// system.
//
// This operation is a batch of ForgetInodeOp operations. Every entry in
// Entries is one ForgetInodeOp operation. See the docs of ForgetInodeOp
// for further details.
type BatchForgetOp struct {
// Entries is a list of Forget operations. One could treat every entry in the
// list as a single ForgetInodeOp operation.
Entries []BatchForgetEntry
OpContext OpContext
}
////////////////////////////////////////////////////////////////////////
// Inode creation
////////////////////////////////////////////////////////////////////////
@ -229,10 +272,11 @@ type ForgetInodeOp struct {
//
// The Linux kernel appears to verify the name doesn't already exist (mkdir
// calls mkdirat calls user_path_create calls filename_create, which verifies:
// http://goo.gl/FZpLu5). Indeed, the tests in samples/memfs that call in
// parallel appear to bear this out. But osxfuse does not appear to guarantee
// this (cf. https://goo.gl/PqzZDv). And if names may be created outside of the
// kernel's control, it doesn't matter what the kernel does anyway.
// https://tinyurl.com/24yw46mf). Indeed, the tests in samples/memfs that call
// in parallel appear to bear this out. But osxfuse does not appear to
// guarantee this (https://tinyurl.com/22587hcf). And if names may be created
// outside of the kernel's control, it doesn't matter what the kernel does
// anyway.
//
// Therefore the file system should return EEXIST if the name already exists.
type MkDirOp struct {
@ -253,14 +297,14 @@ type MkDirOp struct {
// Create a file inode as a child of an existing directory inode. The kernel
// sends this in response to a mknod(2) call. It may also send it in special
// cases such as an NFS export (cf. https://goo.gl/HiLfnK). It is more typical
// cases such as an NFS export (https://tinyurl.com/5dwxr7c9). It is more typical
// to see CreateFileOp, which is received for an open(2) that creates a file.
//
// The Linux kernel appears to verify the name doesn't already exist (mknod
// calls sys_mknodat calls user_path_create calls filename_create, which
// verifies: http://goo.gl/FZpLu5). But osxfuse may not guarantee this, as with
// mkdir(2). And if names may be created outside of the kernel's control, it
// doesn't matter what the kernel does anyway.
// verifies: https://tinyurl.com/24yw46mf). But osxfuse may not guarantee this,
// as with mkdir(2). And if names may be created outside of the kernel's
// control, it doesn't matter what the kernel does anyway.
//
// Therefore the file system should return EEXIST if the name already exists.
type MkNodeOp struct {
@ -271,6 +315,9 @@ type MkNodeOp struct {
Name string
Mode os.FileMode
// The device number (only valid if created file is a device)
Rdev uint32
// Set by the file system: information about the inode that was created.
//
// The lookup count for the inode is implicitly incremented. See notes on
@ -283,10 +330,10 @@ type MkNodeOp struct {
//
// The kernel sends this when the user asks to open a file with the O_CREAT
// flag and the kernel has observed that the file doesn't exist. (See for
// example lookup_open, http://goo.gl/PlqE9d). However, osxfuse doesn't appear
// to make this check atomically (cf. https://goo.gl/PqzZDv). And if names may
// be created outside of the kernel's control, it doesn't matter what the
// kernel does anyway.
// example lookup_open, https://tinyurl.com/49899mvb). However, osxfuse doesn't
// appear to make this check atomically (https://tinyurl.com/22587hcf). And if
// names may be created outside of the kernel's control, it doesn't matter what
// the kernel does anyway.
//
// Therefore the file system should return EEXIST if the name already exists.
type CreateFileOp struct {
@ -364,27 +411,27 @@ type CreateLinkOp struct {
// Rename a file or directory, given the IDs of the original parent directory
// and the new one (which may be the same).
//
// In Linux, this is called by vfs_rename (https://goo.gl/eERItT), which is
// called by sys_renameat2 (https://goo.gl/fCC9qC).
// In Linux, this is called by vfs_rename (https://tinyurl.com/2xbx9kr2), which
// is called by sys_renameat2 (https://tinyurl.com/4zyak2kt).
//
// The kernel takes care of ensuring that the source and destination are not
// identical (in which case it does nothing), that the rename is not across
// file system boundaries, and that the destination doesn't already exist with
// the wrong type. Some subtleties that the file system must care about:
//
// * If the new name is an existing directory, the file system must ensure it
// - If the new name is an existing directory, the file system must ensure it
// is empty before replacing it, returning ENOTEMPTY otherwise. (This is
// per the posix spec: http://goo.gl/4XtT79)
// per the posix spec: https://tinyurl.com/5n865nx9)
//
// * The rename must be atomic from the point of view of an observer of the
// - The rename must be atomic from the point of view of an observer of the
// new name. That is, if the new name already exists, there must be no
// point at which it doesn't exist.
//
// * It is okay for the new name to be modified before the old name is
// - It is okay for the new name to be modified before the old name is
// removed; these need not be atomic. In fact, the Linux man page
// explicitly says this is likely (cf. https://goo.gl/Y1wVZc).
// explicitly says this is likely (https://tinyurl.com/mdpbpjmr).
//
// * Linux bends over backwards (https://goo.gl/pLDn3r) to ensure that
// - Linux bends over backwards (https://tinyurl.com/3hmt7puy) to ensure that
// neither the old nor the new parent can be concurrently modified. But
// it's not clear whether OS X does this, and in any case it doesn't matter
// for file systems that may be modified remotely. Therefore a careful file
@ -394,7 +441,6 @@ type CreateLinkOp struct {
// posix and the man pages are imprecise about the actual semantics of a
// rename if it's not atomic, so it is probably not disastrous to be loose
// about this.
//
type RenameOp struct {
// The old parent directory, and the name of the entry within it to be
// relocated.
@ -414,7 +460,7 @@ type RenameOp struct {
//
// The file system is responsible for checking that the directory is empty.
//
// Sample implementation in ext2: ext2_rmdir (http://goo.gl/B9QmFf)
// Sample implementation in ext2: ext2_rmdir (https://tinyurl.com/bajkpcf9)
type RmDirOp struct {
// The ID of parent directory inode, and the name of the directory being
// removed within it.
@ -428,7 +474,7 @@ type RmDirOp struct {
// ForgetInodeOp. It may still be referenced before then if a user still has
// the file open.
//
// Sample implementation in ext2: ext2_unlink (http://goo.gl/hY6r6C)
// Sample implementation in ext2: ext2_unlink (https://tinyurl.com/3wpwedcp)
type UnlinkOp struct {
// The ID of parent directory inode, and the name of the entry being removed
// within it.
@ -461,6 +507,14 @@ type OpenDirOp struct {
// a later call to ReleaseDirHandle.
Handle HandleID
OpContext OpContext
// CacheDir conveys to the kernel to cache the response of next
// ReadDirOp as page cache. Once cached, listing on that directory will be
// served from the kernel until invalidated.
CacheDir bool
// KeepCache instructs the kernel to not invalidate the data cache on open calls.
KeepCache bool
}
// Read entries from a directory previously opened with OpenDir.
@ -480,47 +534,51 @@ type ReadDirOp struct {
// at zero and is set by llseek and by the final consumed result returned by
// each call to ReadDir:
//
// * (http://goo.gl/2nWJPL) iterate_dir, which is called by getdents(2) and
// readdir(2), sets dir_context::pos to file::f_pos before calling
// f_op->iterate, and then does the opposite assignment afterward.
// * (https://tinyurl.com/3ueykmaj) iterate_dir, which is called by
// getdents(2) and readdir(2), sets dir_context::pos to file::f_pos
// before calling f_op->iterate, and then does the opposite assignment
// afterward.
//
// * (http://goo.gl/rTQVSL) fuse_readdir, which implements iterate for fuse
// directories, passes dir_context::pos as the offset to fuse_read_fill,
// which passes it on to user-space. fuse_readdir later calls
// parse_dirfile with the same context.
// * (https://tinyurl.com/a8urhfy9) fuse_readdir, which implements iterate
// for fuse directories, passes dir_context::pos as the offset to
// fuse_read_fill, which passes it on to user-space. fuse_readdir later
// calls parse_dirfile with the same context.
//
// * (http://goo.gl/vU5ukv) For each returned result (except perhaps the
// last, which may be truncated by the page boundary), parse_dirfile
// updates dir_context::pos with fuse_dirent::off.
// * (https://tinyurl.com/5cev5fn4) For each returned result (except
// perhaps the last, which may be truncated by the page boundary),
// parse_dirfile updates dir_context::pos with fuse_dirent::off.
//
// It is affected by the Posix directory stream interfaces in the following
// manner:
//
// * (http://goo.gl/fQhbyn, http://goo.gl/ns1kDF) opendir initially causes
// filepos to be set to zero.
// * (https://tinyurl.com/2pjv5jvz, https://tinyurl.com/2r6h4mkj) opendir
// initially causes filepos to be set to zero.
//
// * (http://goo.gl/ezNKyR, http://goo.gl/xOmDv0) readdir allows the user
// to iterate through the directory one entry at a time. As each entry is
// consumed, its d_off field is stored in __dirstream::filepos.
// * (https://tinyurl.com/2yvcbcpv, https://tinyurl.com/bddezwp4) readdir
// allows the user to iterate through the directory one entry at a time.
// As each entry is consumed, its d_off field is stored in
// __dirstream::filepos.
//
// * (http://goo.gl/WEOXG8, http://goo.gl/rjSXl3) telldir allows the user
// to obtain the d_off field from the most recently returned entry.
// * (https://tinyurl.com/2pfbfe9v, https://tinyurl.com/4wtat58a) telldir
// allows the user to obtain the d_off field from the most recently
// returned entry.
//
// * (http://goo.gl/WG3nDZ, http://goo.gl/Lp0U6W) seekdir allows the user
// to seek backward to an offset previously returned by telldir. It
// stores the new offset in filepos, and calls llseek to update the
// kernel's struct file.
// * (https://tinyurl.com/bdynryef, https://tinyurl.com/4hysrnb8) seekdir
// allows the user to seek backward to an offset previously returned by
// telldir. It stores the new offset in filepos, and calls llseek to
// update the kernel's struct file.
//
// * (http://goo.gl/gONQhz, http://goo.gl/VlrQkc) rewinddir allows the user
// to go back to the beginning of the directory, obtaining a fresh view.
// It updates filepos and calls llseek to update the kernel's struct
// file.
// * (https://tinyurl.com/5n8dkb44, https://tinyurl.com/3jnn5nnn) rewinddir
// allows the user to go back to the beginning of the directory,
// obtaining a fresh view. It updates filepos and calls llseek to update
// the kernel's struct file.
//
// Unfortunately, FUSE offers no way to intercept seeks
// (http://goo.gl/H6gEXa), so there is no way to cause seekdir or rewinddir
// to fail. Additionally, there is no way to distinguish an explicit
// rewinddir followed by readdir from the initial readdir, or a rewinddir
// from a seekdir to the value returned by telldir just after opendir.
// (https://tinyurl.com/4bm2sfjd), so there is no way to cause seekdir or
// rewinddir to fail. Additionally, there is no way to distinguish an
// explicit rewinddir followed by readdir from the initial readdir, or a
// rewinddir from a seekdir to the value returned by telldir just after
// opendir.
//
// Luckily, Posix is vague about what the user will see if they seek
// backwards, and requires the user not to seek to an old offset after a
@ -530,12 +588,18 @@ type ReadDirOp struct {
// offset, and return array offsets into that cached listing.
Offset DirOffset
// Whether this operation is a READDIRPLUS
//
// If true, then the FS must return inode attributes and expiration time
// along with each directory entry and increment its reference count.
Plus bool
// The destination buffer, whose length gives the size of the read.
//
// The output data should consist of a sequence of FUSE directory entries in
// the format generated by fuse_add_direntry (http://goo.gl/qCcHCV), which is
// consumed by parse_dirfile (http://goo.gl/2WUmD2). Use fuseutil.WriteDirent
// to generate this data.
// the format generated by fuse_add_direntry (https://tinyurl.com/3r9t7d2p),
// which is consumed by parse_dirfile (https://tinyurl.com/bevwty74). Use
// fuseutil.WriteDirent or fuseutil.WriteDirentPlus to generate this data.
//
// Each entry returned exposes a directory offset to the user that may later
// show up in ReadDirRequest.Offset. See notes on that field for more
@ -548,10 +612,10 @@ type ReadDirOp struct {
// entries available or the final entry would not fit.
//
// Zero means that the end of the directory has been reached. This is
// unambiguous because NAME_MAX (https://goo.gl/ZxzKaE) plus the size of
// fuse_dirent (https://goo.gl/WO8s3F) plus the 8-byte alignment of
// FUSE_DIRENT_ALIGN (http://goo.gl/UziWvH) is less than the read size of
// PAGE_SIZE used by fuse_readdir (cf. https://goo.gl/VajtS2).
// unambiguous because NAME_MAX (https://tinyurl.com/4r2b68jp) plus the size
// of fuse_dirent (https://tinyurl.com/mp43bu8) plus the 8-byte alignment of
// FUSE_DIRENT_ALIGN (https://tinyurl.com/3m3ewu7h) is less than the read
// size of PAGE_SIZE used by fuse_readdir (https://tinyurl.com/mrwxsfxw).
BytesRead int
OpContext OpContext
}
@ -563,7 +627,8 @@ type ReadDirOp struct {
// The kernel guarantees that the handle ID will not be used in further ops
// sent to the file system (unless it is reissued by the file system).
//
// Errors from this op are ignored by the kernel (cf. http://goo.gl/RL38Do).
// Errors from this op are ignored by the kernel
// (https://tinyurl.com/2aaccyzk).
type ReleaseDirHandleOp struct {
// The handle ID to be released. The kernel guarantees that this ID will not
// be used in further calls to the file system (unless it is reissued by the
@ -596,15 +661,15 @@ type OpenFileOp struct {
Handle HandleID
// By default, fuse invalidates the kernel's page cache for an inode when a
// new file handle is opened for that inode (cf. https://goo.gl/2rZ9uk). The
// intent appears to be to allow users to "see" content that has changed
// new file handle is opened for that inode (https://tinyurl.com/yyb497zy).
// The intent appears to be to allow users to "see" content that has changed
// remotely on a networked file system by re-opening the file.
//
// For file systems where this is not a concern because all modifications for
// a particular inode go through the kernel, set this field to true to
// disable this behavior.
//
// (More discussion: http://goo.gl/cafzWF)
// (More discussion: https://tinyurl.com/4znxvzwh)
//
// Note that on OS X it appears that the behavior is always as if this field
// is set to true, regardless of its value, at least for files opened in the
@ -620,6 +685,8 @@ type OpenFileOp struct {
// advance, for example, because contents are generated on the fly.
UseDirectIO bool
OpenFlags fusekernel.OpenFlags
OpContext OpContext
}
@ -637,21 +704,35 @@ type ReadFileOp struct {
// The offset within the file at which to read.
Offset int64
// The size of the read.
Size int64
// The destination buffer, whose length gives the size of the read.
// For vectored reads, this field is always nil as the buffer is not provided.
Dst []byte
// Set by the file system:
// A list of slices of data to send back to the client for vectored reads.
Data [][]byte
// Set by the file system: the number of bytes read.
//
// The FUSE documentation requires that exactly the requested number of bytes
// be returned, except in the case of EOF or error (http://goo.gl/ZgfBkF).
// This appears to be because it uses file mmapping machinery
// (http://goo.gl/SGxnaN) to read a page at a time. It appears to understand
// where EOF is by checking the inode size (http://goo.gl/0BkqKD), returned
// by a previous call to LookUpInode, GetInodeAttributes, etc.
// be returned, except in the case of EOF or error
// (https://tinyurl.com/2mzewn35). This appears to be because it uses file
// mmapping machinery (https://tinyurl.com/avxy3dvm) to read a page at a
// time. It appears to understand where EOF is by checking the inode size
// (https://tinyurl.com/2eteerzt), returned by a previous call to
// LookUpInode, GetInodeAttributes, etc.
//
// If direct IO is enabled, semantics should match those of read(2).
BytesRead int
OpContext OpContext
// If set, this function will be invoked after the operation response has been
// sent to the kernel and before the buffers containing the response data are
// freed.
Callback func()
}
// Write data to a file previously opened with CreateFile or OpenFile.
@ -660,22 +741,23 @@ type ReadFileOp struct {
// cache and the page is marked dirty. Later the kernel may write back the
// page via the FUSE VFS layer, causing this op to be sent:
//
// * The kernel calls address_space_operations::writepage when a dirty page
// needs to be written to backing store (cf. http://goo.gl/Ezbewg). Fuse
// sets this to fuse_writepage (cf. http://goo.gl/IeNvLT).
// - The kernel calls address_space_operations::writepage when a dirty page
// needs to be written to backing store (https://tinyurl.com/yck2sf5u).
// Fuse sets this to fuse_writepage (https://tinyurl.com/5n989f8p).
//
// * (http://goo.gl/Eestuy) fuse_writepage calls fuse_writepage_locked.
// - (https://tinyurl.com/mvn6zv3j) fuse_writepage calls
// fuse_writepage_locked.
//
// * (http://goo.gl/RqYIxY) fuse_writepage_locked makes a write request to
// the userspace server.
// - (https://tinyurl.com/2wn8scwb) fuse_writepage_locked makes a write
// request to the userspace server.
//
// Note that the kernel *will* ensure that writes are received and acknowledged
// by the file system before sending a FlushFileOp when closing the file
// descriptor to which they were written. Cf. the notes on
// fuse.MountConfig.DisableWritebackCaching.
//
// (See also http://goo.gl/ocdTdM, fuse-devel thread "Fuse guarantees on
// concurrent requests".)
// (See also https://tinyurl.com/5dchkdtx, fuse-devel thread "Fuse guarantees
// on concurrent requests".)
type WriteFileOp struct {
// The file inode that we are modifying, and the handle previously returned
// by CreateFile or OpenFile when opening that inode.
@ -707,25 +789,41 @@ type WriteFileOp struct {
// The data to write.
//
// The FUSE documentation requires that exactly the number of bytes supplied
// be written, except on error (http://goo.gl/KUpwwn). This appears to be
// because it uses file mmapping machinery (http://goo.gl/SGxnaN) to write a
// page at a time.
// be written, except on error (https://tinyurl.com/yuruk5tx). This appears
// to be because it uses file mmapping machinery
// (https://tinyurl.com/avxy3dvm) to write a page at a time.
Data []byte
// Set by the file system: "no reuse" flag.
//
// By default, the Data buffer is reused by the library, so the file system
// must copy the data if it wants to use it later.
//
// However, if the file system sets this flag to true, the library doesn't
// reuse this buffer, so the file system can safely store and use Data slice
// without copying memory.
SuppressReuse bool
OpContext OpContext
// If set, this function will be invoked after the operation response has been
// sent to the kernel and before the buffers containing the response data are
// freed.
Callback func()
}
// Synchronize the current contents of an open file to storage.
//
// vfs.txt documents this as being called for by the fsync(2) system call
// (cf. http://goo.gl/j9X8nB). Code walk for that case:
// (https://tinyurl.com/y2kdrfzw). Code walk for that case:
//
// * (http://goo.gl/IQkWZa) sys_fsync calls do_fsync, calls vfs_fsync, calls
// vfs_fsync_range.
// - (https://tinyurl.com/2s44cefz) sys_fsync calls do_fsync, calls
// vfs_fsync, calls vfs_fsync_range.
//
// * (http://goo.gl/5L2SMy) vfs_fsync_range calls f_op->fsync.
// - (https://tinyurl.com/bdhhfam5) vfs_fsync_range calls f_op->fsync.
//
// Note that this is also sent by fdatasync(2) (cf. http://goo.gl/01R7rF), and
// may be sent for msync(2) with the MS_SYNC flag (see the notes on
// Note that this is also sent by fdatasync(2) (https://tinyurl.com/ja5wtszf),
// and may be sent for msync(2) with the MS_SYNC flag (see the notes on
// FlushFileOp).
//
// See also: FlushFileOp, which may perform a similar function when closing a
@ -740,39 +838,42 @@ type SyncFileOp struct {
// Flush the current state of an open file to storage upon closing a file
// descriptor.
//
// vfs.txt documents this as being sent for each close(2) system call (cf.
// http://goo.gl/FSkbrq). Code walk for that case:
// vfs.txt documents this as being sent for each close(2) system call
// (https://tinyurl.com/r4ujfxkc). Code walk for that case:
//
// * (http://goo.gl/e3lv0e) sys_close calls __close_fd, calls filp_close.
// * (http://goo.gl/nI8fxD) filp_close calls f_op->flush (fuse_flush).
// - (https://tinyurl.com/2kzyyjcu) sys_close calls __close_fd, calls
// filp_close.
// - (https://tinyurl.com/4zdxrz52) filp_close calls f_op->flush
// (fuse_flush).
//
// But note that this is also sent in other contexts where a file descriptor is
// closed, such as dup2(2) (cf. http://goo.gl/NQDvFS). In the case of close(2),
// a flush error is returned to the user. For dup2(2), it is not.
// closed, such as dup2(2) (https://tinyurl.com/5bj3z3f5). In the case of
// close(2), a flush error is returned to the user. For dup2(2), it is not.
//
// One potentially significant case where this may not be sent is mmap'd files,
// where the behavior is complicated:
//
// * munmap(2) does not cause flushes (cf. http://goo.gl/j8B9g0).
// - munmap(2) does not cause flushes (https://tinyurl.com/ycy9z2jb).
//
// * On OS X, if a user modifies a mapped file via the mapping before
// closing the file with close(2), the WriteFileOps for the modifications
// may not be received before the FlushFileOp for the close(2) (cf.
// - On OS X, if a user modifies a mapped file via the mapping before closing
// the file with close(2), the WriteFileOps for the modifications may not
// be received before the FlushFileOp for the close(2) (cf.
// https://github.com/osxfuse/osxfuse/issues/202). It appears that this may
// be fixed in osxfuse 3 (cf. https://goo.gl/rtvbko).
// be fixed in osxfuse 3 (https://tinyurl.com/2ne2jv8u).
//
// * However, you safely can arrange for writes via a mapping to be
// flushed by calling msync(2) followed by close(2). On OS X msync(2)
// will cause a WriteFileOps to go through and close(2) will cause a
// FlushFile as usual (cf. http://goo.gl/kVmNcx). On Linux, msync(2) does
// nothing unless you set the MS_SYNC flag, in which case it causes a
// SyncFileOp to be sent (cf. http://goo.gl/P3mErk).
// - However, you safely can arrange for writes via a mapping to be flushed
// by calling msync(2) followed by close(2). On OS X msync(2) will cause a
// WriteFileOps to go through and close(2) will cause a FlushFile as usual
// (https://tinyurl.com/2p9b4axf). On Linux, msync(2) does nothing unless
// you set the MS_SYNC flag, in which case it causes a SyncFileOp to be
// sent (https://tinyurl.com/2y3d9hhj).
//
// In summary: if you make data durable in both FlushFile and SyncFile, then
// your users can get safe behavior from mapped files on both operating systems
// by calling msync(2) with MS_SYNC, followed by munmap(2), followed by
// close(2). On Linux, the msync(2) is optional (cf. http://goo.gl/EIhAxv and
// the notes on WriteFileOp).
// close(2). On Linux, the msync(2) is optional (cf.
// https://tinyurl.com/unesszdp and the notes on WriteFileOp).
//
// Because of cases like dup2(2), FlushFileOps are not necessarily one to one
// with OpenFileOps. They should not be used for reference counting, and the
@ -799,7 +900,8 @@ type FlushFileOp struct {
// The kernel guarantees that the handle ID will not be used in further calls
// to the file system (unless it is reissued by the file system).
//
// Errors from this op are ignored by the kernel (cf. http://goo.gl/RL38Do).
// Errors from this op are ignored by the kernel
// (https://tinyurl.com/2aaccyzk).
type ReleaseFileHandleOp struct {
// The handle ID to be released. The kernel guarantees that this ID will not
// be used in further calls to the file system (unless it is reissued by the
@ -923,3 +1025,132 @@ type FallocateOp struct {
Mode uint32
OpContext OpContext
}
type SyncFSOp struct {
Inode InodeID
OpContext OpContext
}
// Request notifications when the file system user calls poll/select or
// similar operations on a file.
type PollOp struct {
// The inode and handle the user wants to poll
Inode InodeID
Handle HandleID
// Kh is the "kernel handle". The reason behind it is that it's allocated
// by the kernel on file allocation and guaranteed to be unique as opposed
// to regular file handles (HandleID) generated by the userland server
// (by us). Kh has to be used in NotifyPollWakeupOut replies.
Kh uint64
// Poll flags
Flags fusekernel.PollFlags
// Requested events
Events fusekernel.PollEvents
// Set by the file system: the actual events that have happened
// since the last poll
Revents fusekernel.PollEvents
OpContext OpContext
}
// Notify consumers waiting for poll/epoll that events are incoming
// for the specified kernel handle. The kernel will send a PollOp request
// to get the event mask after receiving this notification
type NotifyPollWakeup struct {
Kh uint64
}
// Notify to invalidate cache for an inode.
//
// If the filesystem has writeback caching enabled, invalidating an inode
// will first trigger a writeback of all dirty pages. The call will block
// until all writeback requests have completed and the inode has been
// invalidated. It will, however, not wait for completion of pending writeback
// requests that have been issued before.
type NotifyInvalInode struct {
Inode InodeID
Offset int64
Length int64
}
// Notify to invalidate parent attributes and the dentry matching parent/name
//
// To avoid a deadlock this request must not be sent in the execution path
// of a related filesytem operation or within any code that could hold a lock
// that could be needed to execute such an operation. As of kernel 4.18, a
// "related operation" is a lookup(), symlink(), mknod(), mkdir(), unlink(),
// rename(), link() or create() request for the parent, and a setattr(),
// unlink(), rmdir(), rename(), setxattr(), removexattr(), readdir() or
// readdirplus() request for the inode itself.
//
// When called correctly, it will never block.
type NotifyInvalEntry struct {
Parent InodeID
Name string
}
// This request behaves like NotifyInvalEntry with the following additional
// effect (at least as of Linux kernel 4.8):
//
// If the provided child inode matches the inode that is currently associated
// with the cached dentry, and if there are any inotify watches registered for
// the dentry, then the watchers are informed that the dentry has been deleted.
//
// To avoid a deadlock this request must not be sent while executing a
// related filesytem operation or while holding a lock that could be needed to
// execute such an operation.
type NotifyDelete struct {
Parent InodeID
Child InodeID
Name string
}
// Store data to the kernel buffers
//
// Synchronously store data in the kernel buffers belonging to the given inode.
// The stored data is marked up-to-date (no read will be performed against it,
// unless it's invalidated or evicted from the cache).
//
// If the stored data overflows the current file size, then the size is extended,
// similarly to a write(2) on the filesystem.
//
// If this request returns an error, then the store wasn't fully completed, but
// it may have been partially completed.
type NotifyStore struct {
Inode InodeID
Offset uint64
Length uint32
Data [][]byte
}
// Retrieve data from the kernel buffers belonging to the given inode
//
// If successful then the kernel will send a NotifyRetrieveReplyOp as a reply.
// Only present pages are returned in the retrieve reply. Retrieving stops when it
// finds a non-present page and only data prior to that is returned.
//
// If this request returns an error, then the retrieve will not be completed and
// no reply will be sent.
//
// This request doesn't change the dirty state of pages in the kernel buffer. For
// dirty pages the write() method will be called regardless of having been retrieved
// previously.
type NotifyRetrieve struct {
Inode InodeID
Unique uint64
Offset uint64
Length uint32
}
// Matches the size of WriteIn
type NotifyRetrieveReplyOp struct {
Inode InodeID
Unique uint64
Offset uint64
Length uint32
OpContext OpContext
}

View File

@ -27,7 +27,7 @@ import (
// RootInodeID.
//
// This corresponds to struct inode::i_no in the VFS layer.
// (Cf. http://goo.gl/tvYyQt)
// (https://tinyurl.com/23sr9svd)
type InodeID uint64
// RootInodeID is a distinguished inode ID that identifies the root of the file
@ -57,7 +57,7 @@ func init() {
}
// InodeAttributes contains attributes for a file or directory inode. It
// corresponds to struct inode (cf. http://goo.gl/tvYyQt).
// corresponds to struct inode (https://tinyurl.com/23sr9svd).
type InodeAttributes struct {
Size uint64
@ -70,23 +70,26 @@ type InodeAttributes struct {
// Note that in contrast to the defaults for FUSE, this package mounts file
// systems in a manner such that the kernel checks inode permissions in the
// standard posix way. This is implemented by setting the default_permissions
// mount option (cf. http://goo.gl/1LxOop and http://goo.gl/1pTjuk).
// mount option (https://tinyurl.com/ytun2zsn, https://tinyurl.com/52hz9vya).
//
// For example, in the case of mkdir:
//
// * (http://goo.gl/JkdxDI) sys_mkdirat calls inode_permission.
// * (https://tinyurl.com/4yp9bu3h) sys_mkdirat calls inode_permission.
//
// * (...) inode_permission eventually calls do_inode_permission.
//
// * (http://goo.gl/aGCsmZ) calls i_op->permission, which is
// fuse_permission (cf. http://goo.gl/VZ9beH).
// * (https://tinyurl.com/5f9k2eya) calls i_op->permission, which is
// fuse_permission (https://tinyurl.com/4kevbw27).
//
// * (http://goo.gl/5kqUKO) fuse_permission doesn't do anything at all for
// several code paths if FUSE_DEFAULT_PERMISSIONS is unset. In contrast,
// if that flag *is* set, then it calls generic_permission.
// * (https://tinyurl.com/nfea3pwj) fuse_permission doesn't do anything at
// all for several code paths if FUSE_DEFAULT_PERMISSIONS is unset. In
// contrast, if that flag *is* set, then it calls generic_permission.
//
Mode os.FileMode
// The device number. Only valid if the file is a device
Rdev uint32
// Time information. See `man 2 stat` for full details.
Atime time.Time // Time of last access
Mtime time.Time // Time of last modification
@ -114,16 +117,15 @@ func (a *InodeAttributes) DebugString() string {
// when an ID is reused.
//
// This corresponds to struct inode::i_generation in the VFS layer.
// (Cf. http://goo.gl/tvYyQt)
// (https://tinyurl.com/23sr9svd)
//
// Some related reading:
//
// http://fuse.sourceforge.net/doxygen/structfuse__entry__param.html
// http://stackoverflow.com/q/11071996/1505451
// http://goo.gl/CqvwyX
// http://julipedia.meroh.net/2005/09/nfs-file-handles.html
// http://goo.gl/wvo3MB
//
// http://fuse.sourceforge.net/doxygen/structfuse__entry__param.html
// http://stackoverflow.com/q/11071996/1505451
// https://tinyurl.com/yn7wmcmy
// http://julipedia.meroh.net/2005/09/nfs-file-handles.html
// https://tinyurl.com/2c8vsfrs
type GenerationNumber uint64
// HandleID is an opaque 64-bit number used to identify a particular open
@ -158,7 +160,7 @@ type ChildInodeEntry struct {
// Ownership information in particular must be set to something reasonable or
// by default root will own everything and unprivileged users won't be able
// to do anything useful. In traditional file systems in the kernel, the
// function inode_init_owner (http://goo.gl/5qavg8) contains the
// function inode_init_owner (https://tinyurl.com/5yfdrfdf) contains the
// standards-compliant logic for this.
Attributes InodeAttributes
@ -167,16 +169,19 @@ type ChildInodeEntry struct {
//
// For example, this is the abridged call chain for fstat(2):
//
// * (http://goo.gl/tKBH1p) fstat calls vfs_fstat.
// * (http://goo.gl/3HeITq) vfs_fstat eventuall calls vfs_getattr_nosec.
// * (http://goo.gl/DccFQr) vfs_getattr_nosec calls i_op->getattr.
// * (http://goo.gl/dpKkst) fuse_getattr calls fuse_update_attributes.
// * (http://goo.gl/yNlqPw) fuse_update_attributes uses the values in the
// struct inode if allowed, otherwise calling out to the user-space code.
// * (https://tinyurl.com/bdd6ek3c) fstat calls vfs_fstat.
// * (https://tinyurl.com/3enne935) vfs_fstat eventuall calls
// vfs_getattr_nosec.
// * (https://tinyurl.com/y5rkhzx4) vfs_getattr_nosec calls i_op->getattr.
// * (https://tinyurl.com/33hawubc) fuse_getattr calls
// fuse_update_attributes.
// * (https://tinyurl.com/ywhhshxt) fuse_update_attributes uses the values
// in the struct inode if allowed, otherwise calling out to the
// user-space code.
//
// In addition to obvious cases like fstat, this is also used in more subtle
// cases like updating size information before seeking (http://goo.gl/2nnMFa)
// or reading (http://goo.gl/FQSWs8).
// cases like updating size information before seeking
// (https://tinyurl.com/hv2jabnh) or reading (https://tinyurl.com/bdkpz96v).
//
// Most 'real' file systems do not set inode_operations::getattr, and
// therefore vfs_getattr_nosec calls generic_fillattr which simply grabs the
@ -203,16 +208,18 @@ type ChildInodeEntry struct {
// As in the discussion of attribute caching above, unlike real file systems,
// FUSE file systems may spontaneously change their name -> inode mapping.
// Therefore the FUSE VFS layer uses dentry_operations::d_revalidate
// (http://goo.gl/dVea0h) to intercept lookups and revalidate by calling the
// user-space LookUpInode method. However the latter may be slow, so it
// caches the entries until the time defined by this field.
// (https://tinyurl.com/ydb8ncrk) to intercept lookups and revalidate by
// calling the user-space LookUpInode method. However the latter may be slow,
// so it caches the entries until the time defined by this field.
//
// Example code walk:
//
// * (http://goo.gl/M2G3tO) lookup_dcache calls d_revalidate if enabled.
// * (http://goo.gl/ef0Elu) fuse_dentry_revalidate just uses the dentry's
// inode if fuse_dentry_time(entry) hasn't passed. Otherwise it sends a
// lookup request.
// * (https://tinyurl.com/crddueft) lookup_dcache calls d_revalidate if
// enabled.
//
// * (https://tinyurl.com/bdsxacjy) fuse_dentry_revalidate just uses the
// dentry's inode if fuse_dentry_time(entry) hasn't passed. Otherwise
// it sends a lookup request.
//
// Leave at the zero value to disable caching.
//

View File

@ -30,7 +30,7 @@ func (f sortedEntries) Swap(i, j int) { f[i], f[j] = f[j], f[i] }
// Read the directory with the given name and return a list of directory
// entries, sorted by name.
//
// Unlike ioutil.ReadDir (cf. http://goo.gl/i0nNP4), this function does not
// Unlike ioutil.ReadDir (https://tinyurl.com/yft8kkxb), this function does not
// silently ignore "file not found" errors when stat'ing the names read from
// the directory.
func ReadDirPicky(dirname string) (entries []os.FileInfo, err error) {

View File

@ -28,7 +28,7 @@ func extractBirthtime(sys interface{}) (birthtime time.Time, ok bool) {
}
func extractNlink(sys interface{}) (nlink uint64, ok bool) {
return sys.(*syscall.Stat_t).Nlink, true
return uint64(sys.(*syscall.Stat_t).Nlink), true
}
func getTimes(stat *syscall.Stat_t) (atime, ctime, mtime time.Time) {

View File

@ -19,6 +19,7 @@ import (
"unsafe"
"github.com/jacobsa/fuse/fuseops"
"github.com/jacobsa/fuse/internal/fusekernel"
)
type DirentType uint32
@ -50,14 +51,22 @@ type Dirent struct {
Type DirentType
}
// Write the supplied directory entry intto the given buffer in the format
// expected in fuseops.ReadFileOp.Data, returning the number of bytes written.
// Write the supplied directory entry into the given buffer in the format
// expected in fuseops.ReadDirOp.Data, returning the number of bytes written.
// Return zero if the entry would not fit.
func WriteDirent(buf []byte, d Dirent) (n int) {
return WriteDirentPlus(buf, nil, d)
}
// Write the supplied directory entry and, optionally, inode entry into the
// given buffer in the format expected in fuseops.ReadDirOp.Data with enabled
// READDIRPLUS capability, returning the number of bytes written.
// Returns zero if the entry would not fit.
func WriteDirentPlus(buf []byte, e *fuseops.ChildInodeEntry, d Dirent) (n int) {
// We want to write bytes with the layout of fuse_dirent
// (http://goo.gl/BmFxob) in host order. The struct must be aligned according
// to FUSE_DIRENT_ALIGN (http://goo.gl/UziWvH), which dictates 8-byte
// alignment.
// (https://tinyurl.com/4k7y2h9r) in host order. The struct must be aligned
// according to FUSE_DIRENT_ALIGN (https://tinyurl.com/3m3ewu7h), which
// dictates 8-byte alignment.
type fuse_dirent struct {
ino uint64
off uint64
@ -78,10 +87,21 @@ func WriteDirent(buf []byte, d Dirent) (n int) {
// Do we have enough room?
totalLen := direntSize + len(d.Name) + padLen
if e != nil {
// READDIRPLUS was added in protocol 7.21, entry attributes were added in 7.9
// So here EntryOut is always full-length
totalLen += int(unsafe.Sizeof(fusekernel.EntryOut{}))
}
if totalLen > len(buf) {
return n
}
if e != nil {
out := (*fusekernel.EntryOut)(unsafe.Pointer(&buf[n]))
fuseops.ConvertChildInodeEntry(e, out)
n += int(unsafe.Sizeof(fusekernel.EntryOut{}))
}
// Write the header.
de := fuse_dirent{
ino: uint64(d.Inode),

View File

@ -39,6 +39,7 @@ type FileSystem interface {
GetInodeAttributes(context.Context, *fuseops.GetInodeAttributesOp) error
SetInodeAttributes(context.Context, *fuseops.SetInodeAttributesOp) error
ForgetInode(context.Context, *fuseops.ForgetInodeOp) error
BatchForget(context.Context, *fuseops.BatchForgetOp) error
MkDir(context.Context, *fuseops.MkDirOp) error
MkNode(context.Context, *fuseops.MkNodeOp) error
CreateFile(context.Context, *fuseops.CreateFileOp) error
@ -62,6 +63,10 @@ type FileSystem interface {
ListXattr(context.Context, *fuseops.ListXattrOp) error
SetXattr(context.Context, *fuseops.SetXattrOp) error
Fallocate(context.Context, *fuseops.FallocateOp) error
SyncFS(context.Context, *fuseops.SyncFSOp) error
Poll(context.Context, *fuseops.PollOp) error
SetConnection(*fuse.Connection)
// Regard all inodes (including the root inode) as having their lookup counts
// decremented to zero, and clean up any resources associated with the file
@ -80,8 +85,8 @@ type FileSystem interface {
//
// (It is safe to naively process ops concurrently because the kernel
// guarantees to serialize operations that the user expects to happen in order,
// cf. http://goo.gl/jnkHPO, fuse-devel thread "Fuse guarantees on concurrent
// requests").
// cf. https://tinyurl.com/bddm85v5, fuse-devel thread "Fuse guarantees on
// concurrent requests").
func NewFileSystemServer(fs FileSystem) fuse.Server {
return &fileSystemServer{
fs: fs,
@ -94,6 +99,8 @@ type fileSystemServer struct {
}
func (s *fileSystemServer) ServeOps(c *fuse.Connection) {
s.fs.SetConnection(c)
// When we are done, we clean up by waiting for all in-flight ops then
// destroying the file system.
defer func() {
@ -151,6 +158,22 @@ func (s *fileSystemServer) handleOp(
case *fuseops.ForgetInodeOp:
err = s.fs.ForgetInode(ctx, typed)
case *fuseops.BatchForgetOp:
err = s.fs.BatchForget(ctx, typed)
if err == fuse.ENOSYS {
// Handle as a series of single-inode forget operations
for _, entry := range typed.Entries {
err = s.fs.ForgetInode(ctx, &fuseops.ForgetInodeOp{
Inode: entry.Inode,
N: entry.N,
OpContext: typed.OpContext,
})
if err != nil {
break
}
}
}
case *fuseops.MkDirOp:
err = s.fs.MkDir(ctx, typed)
@ -219,6 +242,12 @@ func (s *fileSystemServer) handleOp(
case *fuseops.FallocateOp:
err = s.fs.Fallocate(ctx, typed)
case *fuseops.SyncFSOp:
err = s.fs.SyncFS(ctx, typed)
case *fuseops.PollOp:
err = s.fs.Poll(ctx, typed)
}
c.Reply(ctx, err)

View File

@ -60,6 +60,12 @@ func (fs *NotImplementedFileSystem) ForgetInode(
return fuse.ENOSYS
}
func (fs *NotImplementedFileSystem) BatchForget(
ctx context.Context,
op *fuseops.BatchForgetOp) error {
return fuse.ENOSYS
}
func (fs *NotImplementedFileSystem) MkDir(
ctx context.Context,
op *fuseops.MkDirOp) error {
@ -198,5 +204,20 @@ func (fs *NotImplementedFileSystem) Fallocate(
return fuse.ENOSYS
}
func (fs *NotImplementedFileSystem) SyncFS(
ctx context.Context,
op *fuseops.SyncFSOp) error {
return fuse.ENOSYS
}
func (fs *NotImplementedFileSystem) Poll(
ctx context.Context,
op *fuseops.PollOp) error {
return fuse.ENOSYS
}
func (fs *NotImplementedFileSystem) SetConnection(*fuse.Connection) {
}
func (fs *NotImplementedFileSystem) Destroy() {
}

13
go.mod
View File

@ -1,16 +1,19 @@
module github.com/jacobsa/fuse
go 1.16
go 1.20
require (
github.com/detailyang/go-fallocate v0.0.0-20180908115635-432fa640bd2e
github.com/jacobsa/oglematchers v0.0.0-20150720000706-141901ea67cd
github.com/jacobsa/oglemock v0.0.0-20150831005832-e94d794d06ff // indirect
github.com/jacobsa/ogletest v0.0.0-20170503003838-80d50a735a11
github.com/jacobsa/reqtrace v0.0.0-20150505043853-245c9e0234cb // indirect
github.com/jacobsa/syncutil v0.0.0-20180201203307-228ac8e5a6c3
github.com/jacobsa/timeutil v0.0.0-20170205232429-577e5acbbcf6
github.com/kylelemons/godebug v1.1.0
golang.org/x/net v0.0.0-20200301022130-244492dfa37a
golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527
golang.org/x/net v0.23.0
golang.org/x/sys v0.18.0
)
require (
github.com/jacobsa/oglemock v0.0.0-20150831005832-e94d794d06ff // indirect
github.com/jacobsa/reqtrace v0.0.0-20150505043853-245c9e0234cb // indirect
)

11
go.sum
View File

@ -14,10 +14,7 @@ github.com/jacobsa/timeutil v0.0.0-20170205232429-577e5acbbcf6 h1:XKHJmHcgU9glxk
github.com/jacobsa/timeutil v0.0.0-20170205232429-577e5acbbcf6/go.mod h1:JEWKD6V8xETMW+DEv+IQVz++f8Cn8O/X0HPeDY3qNis=
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/net v0.0.0-20200301022130-244492dfa37a h1:GuSPYbZzB5/dcLNCwLQLsg3obCJtX9IJhpXkvY7kzk0=
golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527 h1:uYVVQ9WP/Ds2ROhcaGPeIdVq0RIXVLwsHlnvJ+cT1So=
golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/net v0.23.0 h1:7EYJ93RZ9vYSZAIb2x3lnuvqO5zneoD6IvWjuhfxjTs=
golang.org/x/net v0.23.0/go.mod h1:JKghWKKOSdJwpW2GEx0Ja7fmaKnMsbu+MWVZTokSYmg=
golang.org/x/sys v0.18.0 h1:DBdB3niSjOA/O0blCZBqDefyWNYveAYMNF1Wum0DYQ4=
golang.org/x/sys v0.18.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=

View File

@ -17,6 +17,7 @@ package buffer
import (
"fmt"
"io"
"sync"
"syscall"
"unsafe"
@ -42,6 +43,7 @@ func init() {
type InMessage struct {
remaining []byte
storage []byte
size int
}
// NewInMessage creates a new InMessage with its storage initialized.
@ -51,11 +53,38 @@ func NewInMessage() *InMessage {
}
}
var readLock sync.Mutex
func (m *InMessage) ReadSingle(r io.Reader) (int, error) {
readLock.Lock()
defer readLock.Unlock()
// read request length
if _, err := io.ReadFull(r, m.storage[0:4]); err != nil {
return 0, err
}
l := m.Header().Len
// read remaining request
if n, err := io.ReadFull(r, m.storage[4:l]); err != nil {
return n, err
}
return int(l), nil
}
// Initialize with the data read by a single call to r.Read. The first call to
// Consume will consume the bytes directly after the fusekernel.InHeader
// struct.
func (m *InMessage) Init(r io.Reader) error {
n, err := r.Read(m.storage[:])
var n int
var err error
if fusekernel.IsPlatformFuseT {
n, err = m.ReadSingle(r)
} else {
n, err = r.Read(m.storage[:])
}
if err != nil {
return err
}
@ -66,6 +95,7 @@ func (m *InMessage) Init(r io.Reader) error {
return fmt.Errorf("Unexpectedly read only %d bytes.", n)
}
m.size = n
m.remaining = m.storage[headerSize:n]
// Check the header's length.
@ -114,3 +144,11 @@ func (m *InMessage) ConsumeBytes(n uintptr) []byte {
return b
}
// Get the next n bytes after the message to use them as a temporary buffer
func (m *InMessage) GetFree(n int) []byte {
if n <= 0 || n > len(m.storage)-m.size {
return nil
}
return m.storage[m.size : m.size+n]
}

View File

@ -17,4 +17,4 @@ package buffer
// The maximum fuse write request size that InMessage can acommodate.
//
// As of kernel 4.20 Linux accepts writes up to 256 pages or 1MiB
const MaxWriteSize = 1 << 20
const MaxWriteSize = 1 << 17

View File

@ -16,7 +16,6 @@ package buffer
import (
"fmt"
"log"
"reflect"
"unsafe"
@ -33,30 +32,15 @@ const OutMessageHeaderSize = int(unsafe.Sizeof(fusekernel.OutHeader{}))
//
// Must be initialized with Reset.
type OutMessage struct {
// The offset into payload to which we're currently writing.
payloadOffset int
header fusekernel.OutHeader
payload [MaxReadSize]byte
}
// Make sure that the header and payload are contiguous.
func init() {
a := unsafe.Offsetof(OutMessage{}.header) + uintptr(OutMessageHeaderSize)
b := unsafe.Offsetof(OutMessage{}.payload)
if a != b {
log.Panicf(
"header ends at offset %d, but payload starts at offset %d",
a, b)
}
header fusekernel.OutHeader
Sglist [][]byte
}
// Reset resets m so that it's ready to be used again. Afterward, the contents
// are solely a zeroed fusekernel.OutHeader struct.
func (m *OutMessage) Reset() {
m.payloadOffset = 0
m.header = fusekernel.OutHeader{}
m.Sglist = nil
}
// OutHeader returns a pointer to the header at the start of the message.
@ -64,30 +48,12 @@ func (m *OutMessage) OutHeader() *fusekernel.OutHeader {
return &m.header
}
// Grow grows m's buffer by the given number of bytes, returning a pointer to
// the start of the new segment, which is guaranteed to be zeroed. If there is
// insufficient space, it returns nil.
// Grow adds a new buffer of <n> bytes to the message, returning a pointer to
// the start of the new segment, which is guaranteed to be zeroed.
func (m *OutMessage) Grow(n int) unsafe.Pointer {
p := m.GrowNoZero(n)
if p != nil {
jacobsa_fuse_memclr(p, uintptr(n))
}
return p
}
// GrowNoZero is equivalent to Grow, except the new segment is not zeroed. Use
// with caution!
func (m *OutMessage) GrowNoZero(n int) unsafe.Pointer {
// Will we overflow the buffer?
o := m.payloadOffset
if len(m.payload)-o < n {
return nil
}
p := unsafe.Pointer(uintptr(unsafe.Pointer(&m.payload)) + uintptr(o))
m.payloadOffset = o + n
b := make([]byte, n)
m.Append(b)
p := unsafe.Pointer(&b[0])
return p
}
@ -100,51 +66,62 @@ func (m *OutMessage) ShrinkTo(n int) {
n,
m.Len()))
}
m.payloadOffset = n - OutMessageHeaderSize
if n == OutMessageHeaderSize {
m.Sglist = nil
} else {
i := 1
n -= OutMessageHeaderSize
for len(m.Sglist) > i && n >= len(m.Sglist[i]) {
n -= len(m.Sglist[i])
i++
}
if n > 0 {
m.Sglist[i] = m.Sglist[i][0:n]
i++
}
m.Sglist = m.Sglist[0:i]
}
}
// Append is equivalent to growing by len(src), then copying src over the new
// segment. Int panics if there is not enough room available.
func (m *OutMessage) Append(src []byte) {
p := m.GrowNoZero(len(src))
if p == nil {
panic(fmt.Sprintf("Can't grow %d bytes", len(src)))
func (m *OutMessage) Append(src ...[]byte) {
if m.Sglist == nil {
// First element of Sglist is pre-filled with a pointer to the header
// to allow sending it with a single writev() call without copying the
// slice again
m.Sglist = append(m.Sglist, m.OutHeaderBytes())
}
sh := (*reflect.SliceHeader)(unsafe.Pointer(&src))
jacobsa_fuse_memmove(p, unsafe.Pointer(sh.Data), uintptr(sh.Len))
m.Sglist = append(m.Sglist, src...)
return
}
// AppendString is like Append, but accepts string input.
func (m *OutMessage) AppendString(src string) {
p := m.GrowNoZero(len(src))
if p == nil {
panic(fmt.Sprintf("Can't grow %d bytes", len(src)))
}
sh := (*reflect.StringHeader)(unsafe.Pointer(&src))
jacobsa_fuse_memmove(p, unsafe.Pointer(sh.Data), uintptr(sh.Len))
m.Append([]byte(src))
return
}
// Len returns the current size of the message, including the leading header.
func (m *OutMessage) Len() int {
return OutMessageHeaderSize + m.payloadOffset
if m.Sglist == nil {
return OutMessageHeaderSize
}
// First element of Sglist is the header, so we don't need to count it here
r := 0
for _, b := range m.Sglist {
r += len(b)
}
return r
}
// Bytes returns a reference to the current contents of the buffer, including
// the leading header.
func (m *OutMessage) Bytes() []byte {
l := m.Len()
// OutHeaderBytes returns a byte slice containing the current header.
func (m *OutMessage) OutHeaderBytes() []byte {
l := OutMessageHeaderSize
sh := reflect.SliceHeader{
Data: uintptr(unsafe.Pointer(&m.header)),
Len: l,
Cap: l,
}
return *(*[]byte)(unsafe.Pointer(&sh))
}

View File

@ -49,47 +49,6 @@ func findNonZero(p unsafe.Pointer, n int) int {
return n
}
func TestMemclr(t *testing.T) {
// All sizes up to 32 bytes.
var sizes []int
for i := 0; i <= 32; i++ {
sizes = append(sizes, i)
}
// And a few hand-chosen sizes.
sizes = append(sizes, []int{
39, 41, 64, 127, 128, 129,
1<<20 - 1,
1 << 20,
1<<20 + 1,
}...)
// For each size, fill a buffer with random bytes and then zero it.
for _, size := range sizes {
size := size
t.Run(fmt.Sprintf("size=%d", size), func(t *testing.T) {
// Generate
b, err := randBytes(size)
if err != nil {
t.Fatalf("randBytes: %v", err)
}
// Clear
var p unsafe.Pointer
if len(b) != 0 {
p = unsafe.Pointer(&b[0])
}
jacobsa_fuse_memclr(p, uintptr(len(b)))
// Check
if i := findNonZero(p, len(b)); i != len(b) {
t.Fatalf("non-zero byte at offset %d", i)
}
})
}
}
func TestOutMessageAppend(t *testing.T) {
var om OutMessage
om.Reset()
@ -107,9 +66,12 @@ func TestOutMessageAppend(t *testing.T) {
t.Errorf("om.Len() = %d, want %d", got, want)
}
b := om.Bytes()
b := []byte(nil)
for i := 0; i < len(om.Sglist); i++ {
b = append(b, om.Sglist[i]...)
}
if got, want := len(b), wantLen; got != want {
t.Fatalf("len(om.Bytes()) = %d, want %d", got, want)
t.Fatalf("len(om.OutHeaderBytes()) = %d, want %d", got, want)
}
want := append(
@ -137,9 +99,12 @@ func TestOutMessageAppendString(t *testing.T) {
t.Errorf("om.Len() = %d, want %d", got, want)
}
b := om.Bytes()
b := []byte(nil)
for i := 0; i < len(om.Sglist); i++ {
b = append(b, om.Sglist[i]...)
}
if got, want := len(b), wantLen; got != want {
t.Fatalf("len(om.Bytes()) = %d, want %d", got, want)
t.Fatalf("len(om.OutHeaderBytes()) = %d, want %d", got, want)
}
want := append(
@ -168,9 +133,12 @@ func TestOutMessageShrinkTo(t *testing.T) {
t.Errorf("om.Len() = %d, want %d", got, want)
}
b := om.Bytes()
b := []byte(nil)
for i := 0; i < len(om.Sglist); i++ {
b = append(b, om.Sglist[i]...)
}
if got, want := len(b), wantLen; got != want {
t.Fatalf("len(om.Bytes()) = %d, want %d", got, want)
t.Fatalf("len(om.OutHeaderBytes()) = %d, want %d", got, want)
}
want := append(
@ -201,7 +169,7 @@ func TestOutMessageHeader(t *testing.T) {
*h = want
// Check that the result is as expected.
b := om.Bytes()
b := om.OutHeaderBytes()
if len(b) != int(unsafe.Sizeof(want)) {
t.Fatalf("unexpected length %d; want %d", len(b), unsafe.Sizeof(want))
}
@ -225,9 +193,7 @@ func TestOutMessageReset(t *testing.T) {
}
// Ensure a non-zero payload length.
if p := om.GrowNoZero(128); p == nil {
t.Fatal("GrowNoZero failed")
}
om.Grow(128)
// Reset.
om.Reset()
@ -259,10 +225,7 @@ func TestOutMessageGrow(t *testing.T) {
// Set up garbage where the payload will soon be.
const payloadSize = 1234
{
p := om.GrowNoZero(payloadSize)
if p == nil {
t.Fatal("GrowNoZero failed")
}
p := om.Grow(payloadSize)
err := fillWithGarbage(p, payloadSize)
if err != nil {
@ -283,7 +246,10 @@ func TestOutMessageGrow(t *testing.T) {
t.Errorf("om.Len() = %d, want %d", got, want)
}
b := om.Bytes()
b := []byte(nil)
for i := 0; i < len(om.Sglist); i++ {
b = append(b, om.Sglist[i]...)
}
if got, want := len(b), wantLen; got != want {
t.Fatalf("len(om.Len()) = %d, want %d", got, want)
}
@ -304,7 +270,7 @@ func BenchmarkOutMessageReset(b *testing.B) {
om.Reset()
}
b.SetBytes(int64(unsafe.Offsetof(om.payload)))
b.SetBytes(int64(om.Len()))
})
// Many megabytes worth of buffers, which should defeat the CPU cache.
@ -321,7 +287,7 @@ func BenchmarkOutMessageReset(b *testing.B) {
oms[i%numMessages].Reset()
}
b.SetBytes(int64(unsafe.Offsetof(oms[0].payload)))
b.SetBytes(int64(oms[0].Len()))
})
}

View File

@ -1,31 +0,0 @@
// Copyright 2015 Google Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package buffer
import "unsafe"
//go:noescape
// Zero the n bytes starting at p.
//
// REQUIRES: the region does not contain any Go pointers.
//go:linkname jacobsa_fuse_memclr runtime.memclrNoHeapPointers
func jacobsa_fuse_memclr(p unsafe.Pointer, n uintptr)
//go:noescape
// Copy from src to dst, allowing overlap.
//go:linkname jacobsa_fuse_memmove runtime.memmove
func jacobsa_fuse_memmove(dst unsafe.Pointer, src unsafe.Pointer, n uintptr)

View File

@ -1,18 +0,0 @@
// Copyright 2015 Google Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// The buffer package uses //go:linkname to push a few functions into this
// package but we still need a .s file so the Go tool does not pass -complete
// to the go tool compile so the latter does not complain about Go functions
// with no bodies.

View File

@ -41,12 +41,14 @@ import (
"unsafe"
)
var IsPlatformFuseT bool
// The FUSE version implemented by the package.
const (
ProtoVersionMinMajor = 7
ProtoVersionMinMinor = 19
ProtoVersionMinMinor = 18
ProtoVersionMaxMajor = 7
ProtoVersionMaxMinor = 31
ProtoVersionMaxMinor = 34
)
const (
@ -168,7 +170,7 @@ const (
// OpenAccessModeMask is a bitmask that separates the access mode
// from the other flags in OpenFlags.
const OpenAccessModeMask OpenFlags = syscall.O_ACCMODE
const OpenAccessModeMask OpenFlags = OpenReadOnly | OpenWriteOnly | OpenReadWrite
// OpenFlags are the O_FOO flags passed to open/create/etc calls. For
// example, os.O_WRONLY | os.O_APPEND.
@ -227,6 +229,7 @@ const (
OpenDirectIO OpenResponseFlags = 1 << 0 // bypass page cache for this open file
OpenKeepCache OpenResponseFlags = 1 << 1 // don't invalidate the data cache on open
OpenNonSeekable OpenResponseFlags = 1 << 2 // mark the file as non-seekable (not supported on OS X)
OpenCacheDir OpenResponseFlags = 1 << 3 // allow caching this directory
OpenPurgeAttr OpenResponseFlags = 1 << 30 // OS X
OpenPurgeUBC OpenResponseFlags = 1 << 31 // OS X
@ -240,6 +243,7 @@ var openResponseFlagNames = []flagName{
{uint32(OpenDirectIO), "OpenDirectIO"},
{uint32(OpenKeepCache), "OpenKeepCache"},
{uint32(OpenNonSeekable), "OpenNonSeekable"},
{uint32(OpenCacheDir), "OpenCacheDir"},
{uint32(OpenPurgeAttr), "OpenPurgeAttr"},
{uint32(OpenPurgeUBC), "OpenPurgeUBC"},
}
@ -266,6 +270,7 @@ const (
InitAsyncDIO InitFlags = 1 << 15
InitWritebackCache InitFlags = 1 << 16
InitNoOpenSupport InitFlags = 1 << 17
InitParallelDirOps InitFlags = 1 << 18
InitMaxPages InitFlags = 1 << 22
InitCacheSymlinks InitFlags = 1 << 23
InitNoOpendirSupport InitFlags = 1 << 24
@ -346,6 +351,34 @@ var releaseFlagNames = []flagName{
{uint32(ReleaseFlush), "ReleaseFlush"},
}
// Poll flags and events are used in the Poll exchange.
type PollFlags uint32
const (
// From the kernel source:
// Ask for notification if there's someone waiting for it.
// The client may ignore the flag and always notify.
PollScheduleNotify PollFlags = 1 << 0
)
type PollEvents uint32
const (
PollInEvent PollEvents = 0x0001
PollPriEvent PollEvents = 0x0002
PollOutEvent PollEvents = 0x0004
PollErrEvent PollEvents = 0x0008
PollHupEvent PollEvents = 0x0010
PollNvalEvent PollEvents = 0x0020
PollRdNormEvent PollEvents = 0x0040
PollRdBandEvent PollEvents = 0x0080
PollWrNormEvent PollEvents = 0x0100
PollWrBandEvent PollEvents = 0x0200
PollMsgEvent PollEvents = 0x0400
PollRemoveEvent PollEvents = 0x1000
PollRdHupEvent PollEvents = 0x2000
)
// Opcodes
const (
OpLookup = 1
@ -386,7 +419,17 @@ const (
OpDestroy = 38
OpIoctl = 39 // Linux?
OpPoll = 40 // Linux?
OpNotifyReply = 41
OpBatchForget = 42
OpFallocate = 43
OpReaddirplus = 44
//
OpRename2 = 45
OpLseek = 46
OpCopyFileRange = 47
OpSetupMapping = 48
OpRemoveMapping = 49
OpSyncFS = 50
// OS X
OpSetvolname = 61
@ -417,6 +460,16 @@ type ForgetIn struct {
Nlookup uint64
}
type BatchForgetCountIn struct {
Count uint32
dummy uint32
}
type BatchForgetEntryIn struct {
Inode int64
Nlookup uint64
}
type GetattrIn struct {
GetattrFlags uint32
dummy uint32
@ -541,6 +594,18 @@ func CreateInSize(p Protocol) uintptr {
}
}
type PollIn struct {
Fh uint64
Kh uint64
Flags uint32
Events uint32
}
type PollOut struct {
Revents uint32
padding uint32
}
type ReleaseIn struct {
Fh uint64
Flags uint32
@ -776,8 +841,15 @@ const (
NotifyCodePoll int32 = 1
NotifyCodeInvalInode int32 = 2
NotifyCodeInvalEntry int32 = 3
NotifyCodeStore int32 = 4
NotifyCodeRetrieve int32 = 5
NotifyCodeDelete int32 = 6
)
type NotifyPollWakeupOut struct {
Kh uint64
}
type NotifyInvalInodeOut struct {
Ino uint64
Off int64
@ -789,3 +861,39 @@ type NotifyInvalEntryOut struct {
Namelen uint32
padding uint32
}
type SyncFSIn struct {
Padding uint64
}
type NotifyDeleteOut struct {
Parent uint64
Child uint64
Namelen uint32
padding uint32
}
type NotifyStoreOut struct {
Nodeid uint64
Offset uint64
Size uint32
padding uint32
}
type NotifyRetrieveOut struct {
Unique uint64
Nodeid uint64
Offset uint64
Size uint32
padding uint32
}
// Matches the size of WriteIn
type NotifyRetrieveIn struct {
dummy1 uint64
Offset uint64
Size uint32
dummy2 uint32
dummy3 uint64
dummy4 uint64
}

View File

@ -1,3 +1,5 @@
// +build linux windows
package fusekernel
import "time"

View File

@ -1 +0,0 @@
package fusekernel

View File

@ -17,9 +17,11 @@ package fuse
import (
"context"
"fmt"
"log"
"net"
"os"
"os/exec"
"strings"
"syscall"
)
@ -41,16 +43,8 @@ func Mount(
config *MountConfig) (*MountedFileSystem, error) {
// Sanity check: make sure the mount point exists and is a directory. This
// saves us from some confusing errors later on OS X.
fi, err := os.Stat(dir)
switch {
case os.IsNotExist(err):
if err := checkMountPoint(dir); err != nil {
return nil, err
case err != nil:
return nil, fmt.Errorf("Statting mount point: %v", err)
case !fi.IsDir():
return nil, fmt.Errorf("Mount point %s is not a directory", dir)
}
// Initialize the struct.
@ -60,11 +54,17 @@ func Mount(
}
// Begin the mounting process, which will continue in the background.
if config.DebugLogger != nil {
config.DebugLogger.Println("Beginning the mounting kickoff process")
}
ready := make(chan error, 1)
dev, err := mount(dir, config, ready)
if err != nil {
return nil, fmt.Errorf("mount: %v", err)
}
if config.DebugLogger != nil {
config.DebugLogger.Println("Completed the mounting kickoff process")
}
// Choose a parent context for ops.
cfgCopy := *config
@ -72,6 +72,9 @@ func Mount(
cfgCopy.OpContext = context.Background()
}
if config.DebugLogger != nil {
config.DebugLogger.Println("Creating a connection object")
}
// Create a Connection object wrapping the device.
connection, err := newConnection(
cfgCopy,
@ -81,6 +84,9 @@ func Mount(
if err != nil {
return nil, fmt.Errorf("newConnection: %v", err)
}
if config.DebugLogger != nil {
config.DebugLogger.Println("Successfully created the connection")
}
// Serve the connection in the background. When done, set the join status.
go func() {
@ -89,6 +95,10 @@ func Mount(
close(mfs.joinStatusAvailable)
}()
if config.DebugLogger != nil {
config.DebugLogger.Println("Waiting for mounting process to complete")
}
// Wait for the mount process to complete.
if err := <-ready; err != nil {
return nil, fmt.Errorf("mount (background): %v", err)
@ -97,13 +107,39 @@ func Mount(
return mfs, nil
}
func fusermount(binary string, argv []string, additionalEnv []string, wait bool) (*os.File, error) {
func checkMountPoint(dir string) error {
if strings.HasPrefix(dir, "/dev/fd") {
return nil
}
fi, err := os.Stat(dir)
switch {
case os.IsNotExist(err):
return err
case err != nil:
return fmt.Errorf("Statting mount point: %v", err)
case !fi.IsDir():
return fmt.Errorf("Mount point %s is not a directory", dir)
}
return nil
}
func fusermount(binary string, argv []string, additionalEnv []string, wait bool, debugLogger *log.Logger) (*os.File, error) {
if debugLogger != nil {
debugLogger.Println("Creating a socket pair")
}
// Create a socket pair.
fds, err := syscall.Socketpair(syscall.AF_UNIX, syscall.SOCK_STREAM, 0)
if err != nil {
return nil, fmt.Errorf("Socketpair: %v", err)
}
if debugLogger != nil {
debugLogger.Println("Creating files to wrap the sockets")
}
// Wrap the sockets into os.File objects that we will pass off to fusermount.
writeFile := os.NewFile(uintptr(fds[0]), "fusermount-child-writes")
defer writeFile.Close()
@ -111,6 +147,9 @@ func fusermount(binary string, argv []string, additionalEnv []string, wait bool)
readFile := os.NewFile(uintptr(fds[1]), "fusermount-parent-reads")
defer readFile.Close()
if debugLogger != nil {
debugLogger.Println("Starting fusermount/os mount")
}
// Start fusermount/mount_macfuse/mount_osxfuse.
cmd := exec.Command(binary, argv...)
cmd.Env = append(os.Environ(), "_FUSE_COMMFD=3")
@ -128,6 +167,9 @@ func fusermount(binary string, argv []string, additionalEnv []string, wait bool)
return nil, fmt.Errorf("running %v: %v", binary, err)
}
if debugLogger != nil {
debugLogger.Println("Wrapping socket pair in a connection")
}
// Wrap the socket file in a connection.
c, err := net.FileConn(readFile)
if err != nil {
@ -135,12 +177,18 @@ func fusermount(binary string, argv []string, additionalEnv []string, wait bool)
}
defer c.Close()
if debugLogger != nil {
debugLogger.Println("Checking that we have a unix domain socket")
}
// We expect to have a Unix domain socket.
uc, ok := c.(*net.UnixConn)
if !ok {
return nil, fmt.Errorf("Expected UnixConn, got %T", c)
}
if debugLogger != nil {
debugLogger.Println("Read a message from socket")
}
// Read a message.
buf := make([]byte, 32) // expect 1 byte
oob := make([]byte, 32) // expect 24 bytes
@ -162,6 +210,10 @@ func fusermount(binary string, argv []string, additionalEnv []string, wait bool)
scm := scms[0]
if debugLogger != nil {
debugLogger.Println("Successfully read the socket message.")
}
// Pull out the FD returned by fusermount
gotFds, err := syscall.ParseUnixRights(&scm)
if err != nil {
@ -172,6 +224,9 @@ func fusermount(binary string, argv []string, additionalEnv []string, wait bool)
return nil, fmt.Errorf("wanted 1 fd; got %#v", gotFds)
}
if debugLogger != nil {
debugLogger.Println("Converting FD into os.File")
}
// Turn the FD into an os.File.
return os.NewFile(uintptr(gotFds[0]), "/dev/fuse"), nil
}

View File

@ -50,7 +50,7 @@ type MountConfig struct {
// Linux only. OS X always behaves as if writeback caching is disabled.
//
// By default on Linux we allow the kernel to perform writeback caching
// (cf. http://goo.gl/LdZzo1):
// (https://tinyurl.com/3ma8ypeu):
//
// * When the user calls write(2), the kernel sticks the user's data into
// its page cache. Only later does it call through to the file system,
@ -65,7 +65,7 @@ type MountConfig struct {
//
// * close(2) (and anything else calling f_op->flush) causes all dirty
// pages to be written out before it proceeds to send a FlushFileOp
// (cf. https://goo.gl/TMrY6X).
// (https://tinyurl.com/3ur6vmsv).
//
// * Similarly, close(2) causes the kernel to send a setattr request
// filling in the mtime if any dirty pages were flushed, since the time
@ -78,22 +78,23 @@ type MountConfig struct {
//
// Code walk:
//
// * (https://goo.gl/zTIZQ9) fuse_flush calls write_inode_now before
// calling the file system. The latter eventually calls into
// * (https://tinyurl.com/3ur6vmsv) fuse_flush calls write_inode_now
// before calling the file system. The latter eventually calls into
// __writeback_single_inode.
//
// * (https://goo.gl/L7Z2w5) __writeback_single_inode calls
// * (https://tinyurl.com/35vtmtsz) __writeback_single_inode calls
// do_writepages, which writes out any dirty pages.
//
// * (https://goo.gl/DOPgla) __writeback_single_inode later calls
// write_inode, which calls into the superblock op struct's write_inode
// member. For fuse, this is fuse_write_inode
// (cf. https://goo.gl/eDSKOX).
// * (https://tinyurl.com/3wv4paaf) __writeback_single_inode later
// calls write_inode, which calls into the superblock op struct's
// write_inode member. For fuse, this is fuse_write_inode
// (https://tinyurl.com/mrxupe98).
//
// * (https://goo.gl/PbkGA1) fuse_write_inode calls fuse_flush_times.
// * (https://tinyurl.com/mrxt9bta) fuse_write_inode calls
// fuse_flush_times.
//
// * (https://goo.gl/ig8x9V) fuse_flush_times sends a setttr request
// for setting the inode's mtime.
// * (https://tinyurl.com/mr49cjdf) fuse_flush_times sends a setttr
// request for setting the inode's mtime.
//
// However, this brings along some caveats:
//
@ -102,11 +103,11 @@ type MountConfig struct {
//
// * The kernel caches mtime and ctime regardless of whether the file
// system tells it to do so, disregarding the result of further getattr
// requests (cf. https://goo.gl/3ZZMUw, https://goo.gl/7WtQUp). It
// appears this may be true of the file size, too. Writeback caching may
// therefore not be suitable for file systems where these attributes can
// spontaneously change for reasons the kernel doesn't observe. See
// http://goo.gl/V5WQCN for more discussion.
// requests (https://tinyurl.com/mrxnfatv, https://tinyurl.com/27jju8n4).
// It appears this may be true of the file size, too. Writeback caching
// may therefore not be suitable for file systems where these attributes
// can spontaneously change for reasons the kernel doesn't observe. See
// https://tinyurl.com/yyprvjvs for more discussion.
//
// Setting DisableWritebackCaching disables this behavior. Instead the file
// system is called one or more times for each write(2), and the user's
@ -116,12 +117,12 @@ type MountConfig struct {
// OS X only.
//
// Normally on OS X we mount with the novncache option
// (cf. http://goo.gl/1pTjuk), which disables entry caching in the kernel.
// This is because osxfuse does not honor the entry expiration values we
// return to it, instead caching potentially forever (cf.
// http://goo.gl/8yR0Ie), and it is probably better to fail to cache than to
// cache for too long, since the latter is more likely to hide consistency
// bugs that are difficult to detect and diagnose.
// (https://tinyurl.com/52hz9vya), which disables entry caching in the
// kernel. This is because macFUSE (osxfuse) does not honor the entry
// expiration values we return to it, instead caching potentially forever
// (https://tinyurl.com/2rr6cd3m), and it is probably better to fail to cache
// than to cache for too long, since the latter is more likely to hide
// consistency bugs that are difficult to detect and diagnose.
//
// This field disables the use of novncache, restoring entry caching. Beware:
// the value of ChildInodeEntry.EntryExpiration is ignored by the kernel, and
@ -151,17 +152,36 @@ type MountConfig struct {
// OpenDir calls at all (Linux >= 5.1):
EnableNoOpendirSupport bool
// Tell the kernel to use READDIRPLUS.
// Note that the implementation may still fall back to READDIR if the running
// kernel doesn't have support for READDIRPLUS.
UseReadDirPlus bool
// Disable FUSE default permissions.
// This is useful for situations where the backing data store (e.g., S3) doesn't
// actually utilise any form of qualifiable UNIX permissions.
DisableDefaultPermissions bool
// Use vectored reads.
// Vectored read allows file systems to avoid memory copying overhead if
// the data is already in memory when they return it to FUSE.
// When turned on, ReadFileOp.Dst is always nil and the FS must return data
// being read from the file as a list of slices in ReadFileOp.Data.
UseVectoredRead bool
// OS X only.
//
// The name of the mounted volume, as displayed in the Finder. If empty, a
// default name involving the string 'osxfuse' is used.
// default name involving the string 'osxfuse' (the old name of macFUSE)
// is used.
VolumeName string
// OS X only.
//
// The FUSE implementation to use. One of FUSEImplFuseT (default) or
// FUSEImplMacFUSE.
FuseImpl FUSEImpl
// Additional key=value options to pass unadulterated to the underlying mount
// command. See `man 8 mount`, the fuse documentation, etc. for
// system-specific information.
@ -178,8 +198,20 @@ type MountConfig struct {
// Flag to enable async reads that are received from
// the kernel
EnableAsyncReads bool
// Flag to enable parallel lookup and readdir operations from the
// kernel
// Ref: https://github.com/torvalds/linux/commit/5c672ab3f0ee0f78f7acad183f34db0f8781a200
EnableParallelDirOps bool
}
type FUSEImpl uint8
const (
FUSEImplFuseT = iota
FUSEImplMacFUSE
)
// Create a map containing all of the key=value mount options to be given to
// the mount helper.
func (c *MountConfig) toMap() (opts map[string]string) {

View File

@ -4,6 +4,7 @@ import (
"bytes"
"errors"
"fmt"
"log"
"os"
"os/exec"
"strconv"
@ -11,6 +12,7 @@ import (
"syscall"
"github.com/jacobsa/fuse/internal/buffer"
"github.com/jacobsa/fuse/internal/fusekernel"
)
var errNoAvail = errors.New("no available fuse devices")
@ -78,6 +80,8 @@ var (
}
)
const FUSET_SRV_PATH = "/usr/local/bin/go-nfsv4"
func loadOSXFUSE(bin string) error {
cmd := exec.Command(bin)
cmd.Dir = "/"
@ -205,14 +209,14 @@ func callMountCommFD(
env = append(env, "_FUSE_COMMVERS=2")
argv = append(argv, dir)
return fusermount(bin, argv, env, false)
return fusermount(bin, argv, env, false, cfg.DebugLogger)
}
// Begin the process of mounting at the given directory, returning a connection
// to the kernel. Mounting continues in the background, and is complete when an
// error is written to the supplied channel. The file system may need to
// service the connection in order for mounting to complete.
func mount(
func mountOsxFuse(
dir string,
cfg *MountConfig,
ready chan<- error) (dev *os.File, err error) {
@ -263,3 +267,161 @@ func mount(
return nil, errOSXFUSENotFound
}
func fusetBinary() (string, error) {
srv_path := os.Getenv("FUSE_NFSSRV_PATH")
if srv_path == "" {
srv_path = FUSET_SRV_PATH
}
if _, err := os.Stat(srv_path); err == nil {
return srv_path, nil
}
return "", fmt.Errorf("FUSE-T not found")
}
func unixgramSocketpair() (l, r *os.File, err error) {
fd, err := syscall.Socketpair(syscall.AF_UNIX, syscall.SOCK_STREAM, 0)
if err != nil {
return nil, nil, os.NewSyscallError("socketpair",
err.(syscall.Errno))
}
l = os.NewFile(uintptr(fd[0]), fmt.Sprintf("socketpair-half%d", fd[0]))
r = os.NewFile(uintptr(fd[1]), fmt.Sprintf("socketpair-half%d", fd[1]))
return
}
var local, local_mon, remote, remote_mon *os.File
func startFuseTServer(binary string, argv []string,
additionalEnv []string,
wait bool,
debugLogger *log.Logger,
ready chan<- error) (*os.File, error) {
if debugLogger != nil {
debugLogger.Println("Creating a socket pair")
}
var err error
local, remote, err = unixgramSocketpair()
if err != nil {
return nil, err
}
defer remote.Close()
local_mon, remote_mon, err = unixgramSocketpair()
if err != nil {
return nil, err
}
defer remote_mon.Close()
syscall.CloseOnExec(int(local.Fd()))
syscall.CloseOnExec(int(local_mon.Fd()))
if debugLogger != nil {
debugLogger.Println("Creating files to wrap the sockets")
}
if debugLogger != nil {
debugLogger.Println("Starting fusermount/os mount")
}
// Start fusermount/mount_macfuse/mount_osxfuse.
cmd := exec.Command(binary, argv...)
cmd.Env = append(os.Environ(), "_FUSE_COMMFD=3")
cmd.Env = append(cmd.Env, "_FUSE_MONFD=4")
cmd.Env = append(cmd.Env, additionalEnv...)
cmd.ExtraFiles = []*os.File{remote, remote_mon}
cmd.Stderr = nil
cmd.Stdout = nil
// daemonize
cmd.SysProcAttr = &syscall.SysProcAttr{
Setsid: true,
}
// Run the command.
err = cmd.Start()
cmd.Process.Release()
if err != nil {
return nil, fmt.Errorf("running %v: %v", binary, err)
}
if debugLogger != nil {
debugLogger.Println("Wrapping socket pair in a connection")
}
if debugLogger != nil {
debugLogger.Println("Checking that we have a unix domain socket")
}
if debugLogger != nil {
debugLogger.Println("Read a message from socket")
}
go func() {
if _, err = local_mon.Write([]byte("mount")); err != nil {
err = fmt.Errorf("fuse-t failed: %v", err)
} else {
reply := make([]byte, 4)
if _, err = local_mon.Read(reply); err != nil {
fmt.Printf("mount read %v\n", err)
err = fmt.Errorf("fuse-t failed: %v", err)
}
}
ready <- err
close(ready)
}()
if debugLogger != nil {
debugLogger.Println("Successfully read the socket message.")
}
return local, nil
}
func mountFuset(
dir string,
cfg *MountConfig,
ready chan<- error) (dev *os.File, err error) {
fuseTBin, err := fusetBinary()
if err != nil {
return nil, err
}
fusekernel.IsPlatformFuseT = true
env := []string{}
argv := []string{
fmt.Sprintf("--rwsize=%d", buffer.MaxWriteSize),
}
if cfg.VolumeName != "" {
argv = append(argv, "--volname")
argv = append(argv, cfg.VolumeName)
}
if cfg.ReadOnly {
argv = append(argv, "-r")
}
env = append(env, "_FUSE_COMMVERS=2")
argv = append(argv, dir)
return startFuseTServer(fuseTBin, argv, env, false, cfg.DebugLogger, ready)
}
func mount(
dir string,
cfg *MountConfig,
ready chan<- error) (dev *os.File, err error) {
fusekernel.IsPlatformFuseT = false
switch cfg.FuseImpl {
case FUSEImplMacFUSE:
dev, err = mountOsxFuse(dir, cfg, ready)
case FUSEImplFuseT:
fallthrough
default:
dev, err = mountFuset(dir, cfg, ready)
}
return
}

View File

@ -5,6 +5,8 @@ import (
"fmt"
"os"
"os/exec"
"strconv"
"strings"
"syscall"
"golang.org/x/sys/unix"
@ -53,6 +55,9 @@ var mountflagopts = map[string]func(uintptr) uintptr{
var errFallback = errors.New("sentinel: fallback to fusermount(1)")
func directmount(dir string, cfg *MountConfig) (*os.File, error) {
if cfg.DebugLogger != nil {
cfg.DebugLogger.Println("Preparing for direct mounting")
}
// We use syscall.Open + os.NewFile instead of os.OpenFile so that the file
// is opened in blocking mode. When opened in non-blocking mode, the Go
// runtime tries to use poll(2), which does not work with /dev/fuse.
@ -61,6 +66,10 @@ func directmount(dir string, cfg *MountConfig) (*os.File, error) {
return nil, errFallback
}
dev := os.NewFile(uintptr(fd), "/dev/fuse")
if cfg.DebugLogger != nil {
cfg.DebugLogger.Println("Successfully opened the /dev/fuse in blocking mode")
}
// As per libfuse/fusermount.c:847: https://bit.ly/2SgtWYM#L847
data := fmt.Sprintf("fd=%d,rootmode=40000,user_id=%d,group_id=%d",
dev.Fd(), os.Getuid(), os.Getgid())
@ -75,6 +84,7 @@ func directmount(dir string, cfg *MountConfig) (*os.File, error) {
mountflag = fn(mountflag)
delete(opts, k)
}
fsname := opts["fsname"]
delete(opts, "fsname") // handled via fstype mount(2) parameter
fstype := "fuse"
if subtype, ok := opts["subtype"]; ok {
@ -82,12 +92,16 @@ func directmount(dir string, cfg *MountConfig) (*os.File, error) {
}
delete(opts, "subtype")
data += "," + mapToOptionsString(opts)
if cfg.DebugLogger != nil {
cfg.DebugLogger.Println("Starting the unix mounting")
}
if err := unix.Mount(
cfg.FSName, // source
dir, // target
fstype, // fstype
mountflag, // mountflag
data, // data
fsname, // source
dir, // target
fstype, // fstype
mountflag, // mountflag
data, // data
); err != nil {
if err == syscall.EPERM {
return nil, errFallback
@ -95,6 +109,9 @@ func directmount(dir string, cfg *MountConfig) (*os.File, error) {
}
return nil, err
}
if cfg.DebugLogger != nil {
cfg.DebugLogger.Println("Unix mounting completed successfully")
}
return dev, nil
}
@ -106,10 +123,24 @@ func mount(dir string, cfg *MountConfig, ready chan<- error) (*os.File, error) {
// On linux, mounting is never delayed.
ready <- nil
if cfg.DebugLogger != nil {
cfg.DebugLogger.Println("Parsing fuse file descriptor")
}
// If the mountpoint is /dev/fd/N, assume that the file descriptor N is an
// already open FUSE channel. Parse it, cast it to an fd, and don't do any
// other part of the mount dance.
if fd, err := parseFuseFd(dir); err == nil {
dev := os.NewFile(uintptr(fd), "/dev/fuse")
return dev, nil
}
// Try mounting without fusermount(1) first: we might be running as root or
// have the CAP_SYS_ADMIN capability.
dev, err := directmount(dir, cfg)
if err == errFallback {
if cfg.DebugLogger != nil {
cfg.DebugLogger.Println("Directmount failed. Trying fallback.")
}
fusermountPath, err := findFusermount()
if err != nil {
return nil, err
@ -119,7 +150,20 @@ func mount(dir string, cfg *MountConfig, ready chan<- error) (*os.File, error) {
"--",
dir,
}
return fusermount(fusermountPath, argv, []string{}, true)
return fusermount(fusermountPath, argv, []string{}, true, cfg.DebugLogger)
}
return dev, err
}
func parseFuseFd(dir string) (int, error) {
if !strings.HasPrefix(dir, "/dev/fd/") {
return -1, fmt.Errorf("not a /dev/fd path")
}
fd, err := strconv.ParseUint(strings.TrimPrefix(dir, "/dev/fd/"), 10, 32)
if err != nil {
return -1, fmt.Errorf("invalid /dev/fd/N path: N must be a positive integer")
}
return int(fd), nil
}

37
mount_linux_test.go Normal file
View File

@ -0,0 +1,37 @@
package fuse
import (
"testing"
)
func Test_parseFuseFd(t *testing.T) {
t.Run("valid", func(t *testing.T) {
fd, err := parseFuseFd("/dev/fd/42")
if fd != 42 {
t.Errorf("expected 42, got %d", fd)
}
if err != nil {
t.Errorf("expected no error, got %#v", err)
}
})
t.Run("negative", func(t *testing.T) {
fd, err := parseFuseFd("/dev/fd/-42")
if fd != -1 {
t.Errorf("expected an invalid fd, got %d", fd)
}
if err == nil {
t.Errorf("expected an error, nil")
}
})
t.Run("not an int", func(t *testing.T) {
fd, err := parseFuseFd("/dev/fd/3.14159")
if fd != -1 {
t.Errorf("expected an invalid fd, got %d", fd)
}
if err == nil {
t.Errorf("expected an error, nil")
}
})
}

View File

@ -14,7 +14,10 @@
package fuse
import "context"
import (
"context"
"fmt"
)
// MountedFileSystem represents the status of a mount operation, with a method
// that waits for unmounting.
@ -47,3 +50,17 @@ func (mfs *MountedFileSystem) Join(ctx context.Context) error {
return ctx.Err()
}
}
// GetFuseContext implements the equiv. of FUSE-C fuse_get_context() and thus
// returns the UID / GID / PID associated with all FUSE requests send by the kernel.
// ctx parameter must be one of the context from the fuseops handlers (e.g.: CreateFile)
func (mfs *MountedFileSystem) GetFuseContext(ctx context.Context) (uid, gid, pid uint32, err error) {
foo := ctx.Value(contextKey)
state, ok := foo.(opState)
if !ok {
return 0, 0, 0, fmt.Errorf("GetFuseContext called with invalid context: %#v", ctx)
}
inMsg := state.inMsg
header := inMsg.Header()
return header.Uid, header.Gid, header.Pid, nil
}

View File

@ -36,9 +36,9 @@ const (
// A file system with a fixed structure that looks like this:
//
// foo
// dir/
// bar
// foo
// dir/
// bar
//
// The file system is configured with durations that specify how long to allow
// inode entries and attributes to be cached, used when responding to fuse
@ -48,6 +48,11 @@ const (
// Each file responds to reads with random contents. SetKeepCache can be used
// to control whether the response to OpenFileOp tells the kernel to keep the
// file's data in the page cache or not.
//
// Each directory responds to readdir with random entries (different names).
// SetCacheDir and SetKeepDirCache can be used to control whether the response
// to OpenDirOp tells the kernel to cache the next response of ReadDirOp
// in cache, or invalidate the existing cached entry in page cache.
type CachingFS interface {
fuseutil.FileSystem
@ -66,20 +71,27 @@ type CachingFS interface {
// Instruct the file system whether or not to reply to OpenFileOp with
// FOPEN_KEEP_CACHE set.
SetKeepCache(keep bool)
// Instruct the file system whether or not to reply to OpenDirOp with
// FOPEN_KEEP_CACHE set.
SetKeepDirCache(keep bool)
// Instruct the file system whether or not to reply to OpenDirOp with
// FOPEN_CACHE_DIR set.
SetCacheDir(cacheDir bool)
}
// Create a file system that issues cacheable responses according to the
// following rules:
//
// * LookUpInodeResponse.Entry.EntryExpiration is set according to
// - LookUpInodeResponse.Entry.EntryExpiration is set according to
// lookupEntryTimeout.
//
// * GetInodeAttributesResponse.AttributesExpiration is set according to
// - GetInodeAttributesResponse.AttributesExpiration is set according to
// getattrTimeout.
//
// * Nothing else is marked cacheable. (In particular, the attributes
// - Nothing else is marked cacheable. (In particular, the attributes
// returned by LookUpInode are not cacheable.)
//
func NewCachingFS(
lookupEntryTimeout time.Duration,
getattrTimeout time.Duration) (CachingFS, error) {
@ -127,6 +139,10 @@ type cachingFS struct {
// GUARDED_BY(mu)
keepPageCache bool
// GUARDED_BY(mu)
keepDirCache bool
cacheDir bool
// The current ID of the lowest numbered non-root inode.
//
// INVARIANT: baseID > fuseops.RootInodeID
@ -203,6 +219,17 @@ func (fs *cachingFS) barAttrs() fuseops.InodeAttributes {
}
}
func randomString(len int) string {
bytes := make([]byte, len)
nn, err := rand.Read(bytes)
if err != nil || len != nn {
return ""
}
return string(bytes)
}
////////////////////////////////////////////////////////////////////////
// Public interface
////////////////////////////////////////////////////////////////////////
@ -255,6 +282,22 @@ func (fs *cachingFS) SetKeepCache(keep bool) {
fs.keepPageCache = keep
}
// LOCKS_EXCLUDED(fs.mu)
func (fs *cachingFS) SetKeepDirCache(keep bool) {
fs.mu.Lock()
defer fs.mu.Unlock()
fs.keepDirCache = keep
}
// LOCKS_EXCLUDED(fs.mu)
func (fs *cachingFS) SetCacheDir(cacheDir bool) {
fs.mu.Lock()
defer fs.mu.Unlock()
fs.cacheDir = cacheDir
}
////////////////////////////////////////////////////////////////////////
// FileSystem methods
////////////////////////////////////////////////////////////////////////
@ -350,6 +393,51 @@ func (fs *cachingFS) GetInodeAttributes(
func (fs *cachingFS) OpenDir(
ctx context.Context,
op *fuseops.OpenDirOp) error {
fs.mu.Lock()
defer fs.mu.Unlock()
op.CacheDir = fs.cacheDir
op.KeepCache = fs.keepDirCache
return nil
}
func (fs *cachingFS) ReadDir(
ctx context.Context,
op *fuseops.ReadDirOp) error {
entries := []fuseutil.Dirent{
{
Offset: 1,
Inode: 101,
Name: "rdir" + randomString(4),
Type: fuseutil.DT_Directory,
},
{
Offset: 2,
Inode: 102,
Name: "rfoo" + randomString(4),
Type: fuseutil.DT_File,
},
}
// Grab the range of interest.
if op.Offset > fuseops.DirOffset(len(entries)) {
return fuse.EIO
}
entries = entries[op.Offset:]
// Resume at the specified offset into the array.
for _, e := range entries {
n := fuseutil.WriteDirent(op.Dst[op.BytesRead:], e)
if n == 0 {
break
}
op.BytesRead += n
}
return nil
}

View File

@ -723,3 +723,253 @@ func (t *PageCacheTest) TwoFileHandles_KeepCache() {
ExpectTrue(bytes.Equal(c1, c3))
}
////////////////////////////////////////////////////////////////////////
// Dir cache
////////////////////////////////////////////////////////////////////////
type DirCacheTest struct {
cachingFSTest
}
var _ SetUpInterface = &DirCacheTest{}
func init() { RegisterTestSuite(&DirCacheTest{}) }
func (t *DirCacheTest) SetUp(ti *TestInfo) {
const (
lookupEntryTimeout = 0
getattrTimeout = 0
)
t.cachingFSTest.setUp(ti, lookupEntryTimeout, getattrTimeout)
}
func (t *DirCacheTest) CacheDirAndKeepDirCache() {
t.fs.SetCacheDir(true)
t.fs.SetKeepDirCache(true)
// First read, kernel will cache the dir response.
f, err := os.Open(path.Join(t.Dir, "dir"))
defer f.Close() // Make sure to close specially required in failure scenario.
AssertEq(nil, err)
names1, err := f.Readdirnames(-1)
AssertEq(nil, err)
AssertEq(2, len(names1))
err = f.Close()
AssertEq(nil, err)
// Second read, will be served from cache.
f, err = os.Open(path.Join(t.Dir, "dir"))
AssertEq(nil, err)
names2, err := f.Readdirnames(-1)
AssertEq(nil, err)
AssertEq(2, len(names2))
err = f.Close()
AssertEq(nil, err)
AssertEq(names1[0], names2[0])
AssertEq(names1[1], names2[1])
}
func (t *DirCacheTest) NoCacheDirAndKeepDirCache() {
t.fs.SetCacheDir(false)
t.fs.SetKeepDirCache(true)
// First read, no caching since NoCacheDir.
f, err := os.Open(path.Join(t.Dir, "dir"))
defer f.Close() // Make sure to close specially required in failure scenario.
AssertEq(nil, err)
names1, err := f.Readdirnames(-1)
AssertEq(nil, err)
AssertEq(2, len(names1))
err = f.Close()
AssertEq(nil, err)
// Second read, will be served from filesystem, hence different name.
f, err = os.Open(path.Join(t.Dir, "dir"))
AssertEq(nil, err)
names2, err := f.Readdirnames(-1)
AssertEq(nil, err)
AssertEq(2, len(names2))
err = f.Close()
AssertEq(nil, err)
AssertNe(names1[0], names2[0])
AssertNe(names1[1], names2[1])
}
func (t *DirCacheTest) CacheDirAndNoKeepDirCache() {
t.fs.SetCacheDir(true)
t.fs.SetKeepDirCache(false)
// First read, kernel will cache the dir response.
f, err := os.Open(path.Join(t.Dir, "dir"))
defer f.Close() // Make sure to close specially required in failure scenario.
AssertEq(nil, err)
names1, err := f.Readdirnames(-1)
AssertEq(nil, err)
AssertEq(2, len(names1))
err = f.Close()
AssertEq(nil, err)
// Second read, cached response will be invalidated since NoKeepDirCache.
// Hence, different names.
f, err = os.Open(path.Join(t.Dir, "dir"))
AssertEq(nil, err)
names2, err := f.Readdirnames(-1)
AssertEq(nil, err)
AssertEq(2, len(names2))
err = f.Close()
AssertEq(nil, err)
AssertNe(names1[0], names2[0])
AssertNe(names1[1], names2[1])
}
func (t *DirCacheTest) NoCacheDirAndNoKeepDirCache() {
t.fs.SetCacheDir(false)
t.fs.SetKeepDirCache(false)
// First read, no caching since NoCacheDir.
f, err := os.Open(path.Join(t.Dir, "dir"))
defer f.Close() // Make sure to close specially required in failure scenario.
AssertEq(nil, err)
names1, err := f.Readdirnames(-1)
AssertEq(nil, err)
AssertEq(2, len(names1))
err = f.Close()
AssertEq(nil, err)
// Second read, will be served from filesystem.
// Since NoCacheDir also NoKeepDirCache. Hence, different names.
f, err = os.Open(path.Join(t.Dir, "dir"))
AssertEq(nil, err)
names2, err := f.Readdirnames(-1)
AssertEq(nil, err)
AssertEq(2, len(names2))
err = f.Close()
AssertEq(nil, err)
AssertNe(names1[0], names2[0])
AssertNe(names1[1], names2[1])
}
func (t *DirCacheTest) CacheDirWithChangingKeepDirCache() {
t.fs.SetCacheDir(true)
t.fs.SetKeepDirCache(false)
// First read, kernel will cache the dir response.
f, err := os.Open(path.Join(t.Dir, "dir"))
defer f.Close() // Make sure to close specially required in failure scenario.
AssertEq(nil, err)
names1, err := f.Readdirnames(-1)
AssertEq(nil, err)
AssertEq(2, len(names1))
err = f.Close()
AssertEq(nil, err)
// Cached response will not be invalidated. Hence, served from kernel cache.
t.fs.SetKeepDirCache(true)
// Second read, will be served from cache
f, err = os.Open(path.Join(t.Dir, "dir"))
AssertEq(nil, err)
names2, err := f.Readdirnames(-1)
AssertEq(nil, err)
AssertEq(2, len(names2))
err = f.Close()
AssertEq(nil, err)
AssertEq(names1[0], names2[0])
AssertEq(names1[1], names2[1])
// Kernel has cached but invalidated due to NoKeepDirCache.
// Hence, third read will be served from filesystem.
t.fs.SetKeepDirCache(false)
// Third read, will be served from filesystem. So, names will be different.
f, err = os.Open(path.Join(t.Dir, "dir"))
AssertEq(nil, err)
names3, err := f.Readdirnames(-1)
AssertEq(nil, err)
AssertEq(2, len(names3))
err = f.Close()
AssertEq(nil, err)
AssertNe(names2[0], names3[0])
AssertNe(names2[1], names3[1])
}
func (t *DirCacheTest) ChangingCacheDirWithKeepDirCache() {
t.fs.SetCacheDir(true)
t.fs.SetKeepDirCache(true)
// First read, kernel will cache the dir response.
f, err := os.Open(path.Join(t.Dir, "dir"))
defer f.Close() // Make sure to close specially required in failure scenario.
AssertEq(nil, err)
names1, err := f.Readdirnames(-1)
AssertEq(nil, err)
AssertEq(2, len(names1))
err = f.Close()
AssertEq(nil, err)
// Cached response will not be invalidated, but also not be served from cache.
// Since NoCacheDir so names will be different.
t.fs.SetCacheDir(false)
// Second read, will be served from filesystem.
f, err = os.Open(path.Join(t.Dir, "dir"))
AssertEq(nil, err)
names2, err := f.Readdirnames(-1)
AssertEq(nil, err)
AssertEq(2, len(names2))
err = f.Close()
AssertEq(nil, err)
AssertNe(names1[0], names2[0])
AssertNe(names1[1], names2[1])
// Third read will be served from cache.
// But first read response is cached, since KeepDirCache.
t.fs.SetCacheDir(true)
// Third read, will be served from filesystem. So, names will be different.
f, err = os.Open(path.Join(t.Dir, "dir"))
AssertEq(nil, err)
names3, err := f.Readdirnames(-1)
AssertEq(nil, err)
AssertEq(2, len(names3))
err = f.Close()
AssertEq(nil, err)
AssertEq(names1[0], names3[0])
AssertEq(names1[1], names3[1])
}

View File

@ -28,9 +28,9 @@ import (
// Create a file system with a fixed structure that looks like this:
//
// hello
// dir/
// world
// hello
// dir/
// world
//
// Each file contains the string "Hello, world!".
func NewHelloFS(clock timeutil.Clock) (fuse.Server, error) {
@ -218,7 +218,7 @@ func (fs *helloFS) ReadDir(
// Grab the range of interest.
if op.Offset > fuseops.DirOffset(len(entries)) {
return fuse.EIO
return nil
}
entries = entries[op.Offset:]

View File

@ -0,0 +1,81 @@
package memfs_test
import (
"os"
"path"
"testing"
"github.com/jacobsa/fuse/samples"
"github.com/jacobsa/fuse/samples/memfs"
. "github.com/jacobsa/ogletest"
)
func TestFuseServerCallbacks(t *testing.T) { RunTests(t) }
type CallbackTest struct {
samples.SampleTest
readFileCallbackInvoked bool
writeFileCallbackInvoked bool
}
func init() { RegisterTestSuite(&CallbackTest{}) }
func (t *CallbackTest) SetUp(ti *TestInfo) {
t.MountConfig.DisableWritebackCaching = true
t.readFileCallbackInvoked = false
t.writeFileCallbackInvoked = false
t.Server = memfs.NewMemFSWithCallbacks(
currentUid(),
currentGid(),
func() { t.readFileCallbackInvoked = true },
func() { t.writeFileCallbackInvoked = true },
)
t.SampleTest.SetUp(ti)
}
// The test suite is torn down during the test to ensure
// that all FUSE operations are complete before checking
// the invocations on the callbacks.
func (t *CallbackTest) TearDown() {}
func (t *CallbackTest) TestCallbacksInvokedForWriteFile() {
AssertEq(t.writeFileCallbackInvoked, false)
AssertEq(t.readFileCallbackInvoked, false)
// Write a file.
fileName := path.Join(t.Dir, memfs.CheckFileOpenFlagsFileName)
const contents = "Hello world"
err := os.WriteFile(fileName, []byte(contents), 0400)
AssertEq(nil, err)
// Tear down the FUSE mount. This ensures that all FUSE operations are complete.
t.SampleTest.TearDown()
// Check that our callback was invoked as expected.
AssertEq(t.writeFileCallbackInvoked, true)
AssertEq(t.readFileCallbackInvoked, false)
}
func (t *CallbackTest) TestCallbacksInvokedForWriteFileAndReadFile() {
AssertEq(t.writeFileCallbackInvoked, false)
AssertEq(t.readFileCallbackInvoked, false)
// Write a file.
fileName := path.Join(t.Dir, memfs.CheckFileOpenFlagsFileName)
const contents = "Hello world"
err := os.WriteFile(fileName, []byte(contents), 0400)
AssertEq(nil, err)
// Read it back.
slice, err := os.ReadFile(fileName)
AssertEq(nil, err)
ExpectEq(contents, string(slice))
// Tear down the FUSE mount. This ensures that all FUSE operations are complete.
t.SampleTest.TearDown()
// Check that our callback was invoked as expected.
AssertEq(t.writeFileCallbackInvoked, true)
AssertEq(t.readFileCallbackInvoked, true)
}

View File

@ -17,6 +17,7 @@ package memfs
import (
"fmt"
"io"
"io/fs"
"os"
"time"
@ -33,6 +34,10 @@ type inode struct {
// Mutable state
/////////////////////////
// Name of the inode, only contains relative path.
// For example, if the full path for an inode is /foo/bar/f1, its name is f1.
name string
// The current attributes of this inode.
//
// INVARIANT: attrs.Mode &^ (os.ModePerm|os.ModeDir|os.ModeSymlink) == 0
@ -73,7 +78,7 @@ type inode struct {
// Create a new inode with the supplied attributes, which need not contain
// time-related information (the inode object will take care of that).
func newInode(attrs fuseops.InodeAttributes) *inode {
func newInode(attrs fuseops.InodeAttributes, name string) *inode {
// Update time info.
now := time.Now()
attrs.Mtime = now
@ -81,6 +86,7 @@ func newInode(attrs fuseops.InodeAttributes) *inode {
// Create the object.
return &inode{
name: name,
attrs: attrs,
xattrs: make(map[string][]byte),
}
@ -372,7 +378,8 @@ func (in *inode) SetAttributes(
// Change mode?
if mode != nil {
in.attrs.Mode = *mode
in.attrs.Mode &= ^fs.ModePerm
in.attrs.Mode |= *mode & fs.ModePerm
}
// Change mtime?

View File

@ -16,6 +16,7 @@ package memfs
import (
"context"
"encoding/binary"
"fmt"
"io"
"os"
@ -29,6 +30,11 @@ import (
"golang.org/x/sys/unix"
)
const (
FileOpenFlagsXattrName = "fileOpenFlagsXattr"
CheckFileOpenFlagsFileName = "checkFileOpenFlags"
)
type memFS struct {
fuseutil.NotImplementedFileSystem
@ -61,6 +67,9 @@ type memFS struct {
// INVARIANT: This is all and only indices i of 'inodes' such that i >
// fuseops.RootInodeID and inodes[i] == nil
freeInodes []fuseops.InodeID // GUARDED_BY(mu)
readFileCallback func()
writeFileCallback func()
}
// Create a file system that stores data and metadata in memory.
@ -71,11 +80,21 @@ type memFS struct {
func NewMemFS(
uid uint32,
gid uint32) fuse.Server {
return NewMemFSWithCallbacks(uid, gid, nil, nil)
}
func NewMemFSWithCallbacks(
uid uint32,
gid uint32,
readFileCallback func(),
writeFileCallback func()) fuse.Server {
// Set up the basic struct.
fs := &memFS{
inodes: make([]*inode, fuseops.RootInodeID+1),
uid: uid,
gid: gid,
inodes: make([]*inode, fuseops.RootInodeID+1),
uid: uid,
gid: gid,
readFileCallback: readFileCallback,
writeFileCallback: writeFileCallback,
}
// Set up the root inode.
@ -85,7 +104,7 @@ func NewMemFS(
Gid: gid,
}
fs.inodes[fuseops.RootInodeID] = newInode(rootAttrs)
fs.inodes[fuseops.RootInodeID] = newInode(rootAttrs, "")
// Set up invariant checking.
fs.mu = syncutil.NewInvariantMutex(fs.checkInvariants)
@ -157,9 +176,9 @@ func (fs *memFS) getInodeOrDie(id fuseops.InodeID) *inode {
//
// LOCKS_REQUIRED(fs.mu)
func (fs *memFS) allocateInode(
attrs fuseops.InodeAttributes) (id fuseops.InodeID, inode *inode) {
attrs fuseops.InodeAttributes, name string) (id fuseops.InodeID, inode *inode) {
// Create the inode.
inode = newInode(attrs)
inode = newInode(attrs, name)
// Re-use a free ID if possible. Otherwise mint a new one.
numFree := len(fs.freeInodes)
@ -194,10 +213,6 @@ func (fs *memFS) StatFS(
func (fs *memFS) LookUpInode(
ctx context.Context,
op *fuseops.LookUpInodeOp) error {
if op.OpContext.Pid == 0 {
return fuse.EINVAL
}
fs.mu.Lock()
defer fs.mu.Unlock()
@ -228,10 +243,6 @@ func (fs *memFS) LookUpInode(
func (fs *memFS) GetInodeAttributes(
ctx context.Context,
op *fuseops.GetInodeAttributesOp) error {
if op.OpContext.Pid == 0 {
return fuse.EINVAL
}
fs.mu.Lock()
defer fs.mu.Unlock()
@ -251,10 +262,6 @@ func (fs *memFS) GetInodeAttributes(
func (fs *memFS) SetInodeAttributes(
ctx context.Context,
op *fuseops.SetInodeAttributesOp) error {
if op.OpContext.Pid == 0 {
return fuse.EINVAL
}
fs.mu.Lock()
defer fs.mu.Unlock()
@ -284,10 +291,6 @@ func (fs *memFS) SetInodeAttributes(
func (fs *memFS) MkDir(
ctx context.Context,
op *fuseops.MkDirOp) error {
if op.OpContext.Pid == 0 {
return fuse.EINVAL
}
fs.mu.Lock()
defer fs.mu.Unlock()
@ -310,7 +313,7 @@ func (fs *memFS) MkDir(
}
// Allocate a child.
childID, child := fs.allocateInode(childAttrs)
childID, child := fs.allocateInode(childAttrs, op.Name)
// Add an entry in the parent.
parent.AddChild(childID, op.Name, fuseutil.DT_Directory)
@ -330,10 +333,6 @@ func (fs *memFS) MkDir(
func (fs *memFS) MkNode(
ctx context.Context,
op *fuseops.MkNodeOp) error {
if op.OpContext.Pid == 0 {
return fuse.EINVAL
}
fs.mu.Lock()
defer fs.mu.Unlock()
@ -371,7 +370,7 @@ func (fs *memFS) createFile(
}
// Allocate a child.
childID, child := fs.allocateInode(childAttrs)
childID, child := fs.allocateInode(childAttrs, name)
// Add an entry in the parent.
parent.AddChild(childID, name, fuseutil.DT_File)
@ -392,11 +391,6 @@ func (fs *memFS) createFile(
func (fs *memFS) CreateFile(
ctx context.Context,
op *fuseops.CreateFileOp) (err error) {
if op.OpContext.Pid == 0 {
// CreateFileOp should have a valid pid in context.
return fuse.EINVAL
}
fs.mu.Lock()
defer fs.mu.Unlock()
@ -407,10 +401,6 @@ func (fs *memFS) CreateFile(
func (fs *memFS) CreateSymlink(
ctx context.Context,
op *fuseops.CreateSymlinkOp) error {
if op.OpContext.Pid == 0 {
return fuse.EINVAL
}
fs.mu.Lock()
defer fs.mu.Unlock()
@ -438,7 +428,7 @@ func (fs *memFS) CreateSymlink(
}
// Allocate a child.
childID, child := fs.allocateInode(childAttrs)
childID, child := fs.allocateInode(childAttrs, op.Name)
// Set up its target.
child.target = op.Target
@ -461,10 +451,6 @@ func (fs *memFS) CreateSymlink(
func (fs *memFS) CreateLink(
ctx context.Context,
op *fuseops.CreateLinkOp) error {
if op.OpContext.Pid == 0 {
return fuse.EINVAL
}
fs.mu.Lock()
defer fs.mu.Unlock()
@ -504,10 +490,6 @@ func (fs *memFS) CreateLink(
func (fs *memFS) Rename(
ctx context.Context,
op *fuseops.RenameOp) error {
if op.OpContext.Pid == 0 {
return fuse.EINVAL
}
fs.mu.Lock()
defer fs.mu.Unlock()
@ -549,10 +531,6 @@ func (fs *memFS) Rename(
func (fs *memFS) RmDir(
ctx context.Context,
op *fuseops.RmDirOp) error {
if op.OpContext.Pid == 0 {
return fuse.EINVAL
}
fs.mu.Lock()
defer fs.mu.Unlock()
@ -585,10 +563,6 @@ func (fs *memFS) RmDir(
func (fs *memFS) Unlink(
ctx context.Context,
op *fuseops.UnlinkOp) error {
if op.OpContext.Pid == 0 {
return fuse.EINVAL
}
fs.mu.Lock()
defer fs.mu.Unlock()
@ -616,10 +590,6 @@ func (fs *memFS) Unlink(
func (fs *memFS) OpenDir(
ctx context.Context,
op *fuseops.OpenDirOp) error {
if op.OpContext.Pid == 0 {
return fuse.EINVAL
}
fs.mu.Lock()
defer fs.mu.Unlock()
@ -638,10 +608,6 @@ func (fs *memFS) OpenDir(
func (fs *memFS) ReadDir(
ctx context.Context,
op *fuseops.ReadDirOp) error {
if op.OpContext.Pid == 0 {
return fuse.EINVAL
}
fs.mu.Lock()
defer fs.mu.Unlock()
@ -657,11 +623,6 @@ func (fs *memFS) ReadDir(
func (fs *memFS) OpenFile(
ctx context.Context,
op *fuseops.OpenFileOp) error {
if op.OpContext.Pid == 0 {
// OpenFileOp should have a valid pid in context.
return fuse.EINVAL
}
fs.mu.Lock()
defer fs.mu.Unlock()
@ -674,16 +635,27 @@ func (fs *memFS) OpenFile(
panic("Found non-file.")
}
if inode.name == CheckFileOpenFlagsFileName {
// For testing purpose only.
// Set attribute (name=fileOpenFlagsXattr, value=OpenFlags) to test whether
// we set OpenFlags correctly. The value is checked in test with getXattr.
value := make([]byte, 4)
binary.LittleEndian.PutUint32(value, uint32(op.OpenFlags)&syscall.O_ACCMODE)
err := fs.setXattrHelper(inode, &fuseops.SetXattrOp{
Name: FileOpenFlagsXattrName,
Value: value,
})
if err != nil {
panic("unable to set fileOpenFlagsXattr")
}
}
return nil
}
func (fs *memFS) ReadFile(
ctx context.Context,
op *fuseops.ReadFileOp) error {
if op.OpContext.Pid == 0 {
return fuse.EINVAL
}
fs.mu.Lock()
defer fs.mu.Unlock()
@ -694,6 +666,8 @@ func (fs *memFS) ReadFile(
var err error
op.BytesRead, err = inode.ReadAt(op.Dst, op.Offset)
op.Callback = fs.readFileCallback
// Don't return EOF errors; we just indicate EOF to fuse using a short read.
if err == io.EOF {
return nil
@ -705,10 +679,6 @@ func (fs *memFS) ReadFile(
func (fs *memFS) WriteFile(
ctx context.Context,
op *fuseops.WriteFileOp) error {
if op.OpContext.Pid == 0 {
return fuse.EINVAL
}
fs.mu.Lock()
defer fs.mu.Unlock()
@ -718,26 +688,20 @@ func (fs *memFS) WriteFile(
// Serve the request.
_, err := inode.WriteAt(op.Data, op.Offset)
op.Callback = fs.writeFileCallback
return err
}
func (fs *memFS) FlushFile(
ctx context.Context,
op *fuseops.FlushFileOp) (err error) {
if op.OpContext.Pid == 0 {
// FlushFileOp should have a valid pid in context.
return fuse.EINVAL
}
return
}
func (fs *memFS) ReadSymlink(
ctx context.Context,
op *fuseops.ReadSymlinkOp) error {
if op.OpContext.Pid == 0 {
return fuse.EINVAL
}
fs.mu.Lock()
defer fs.mu.Unlock()
@ -752,10 +716,6 @@ func (fs *memFS) ReadSymlink(
func (fs *memFS) GetXattr(ctx context.Context,
op *fuseops.GetXattrOp) error {
if op.OpContext.Pid == 0 {
return fuse.EINVAL
}
fs.mu.Lock()
defer fs.mu.Unlock()
@ -776,10 +736,6 @@ func (fs *memFS) GetXattr(ctx context.Context,
func (fs *memFS) ListXattr(ctx context.Context,
op *fuseops.ListXattrOp) error {
if op.OpContext.Pid == 0 {
return fuse.EINVAL
}
fs.mu.Lock()
defer fs.mu.Unlock()
@ -803,10 +759,6 @@ func (fs *memFS) ListXattr(ctx context.Context,
func (fs *memFS) RemoveXattr(ctx context.Context,
op *fuseops.RemoveXattrOp) error {
if op.OpContext.Pid == 0 {
return fuse.EINVAL
}
fs.mu.Lock()
defer fs.mu.Unlock()
inode := fs.getInodeOrDie(op.Inode)
@ -821,14 +773,15 @@ func (fs *memFS) RemoveXattr(ctx context.Context,
func (fs *memFS) SetXattr(ctx context.Context,
op *fuseops.SetXattrOp) error {
if op.OpContext.Pid == 0 {
return fuse.EINVAL
}
fs.mu.Lock()
defer fs.mu.Unlock()
inode := fs.getInodeOrDie(op.Inode)
return fs.setXattrHelper(inode, op)
}
// Required to hold fs.mu
func (fs *memFS) setXattrHelper(inode *inode, op *fuseops.SetXattrOp) error {
_, ok := inode.xattrs[op.Name]
switch op.Flags {
@ -850,10 +803,6 @@ func (fs *memFS) SetXattr(ctx context.Context,
func (fs *memFS) Fallocate(ctx context.Context,
op *fuseops.FallocateOp) error {
if op.OpContext.Pid == 0 {
return fuse.EINVAL
}
fs.mu.Lock()
defer fs.mu.Unlock()
inode := fs.getInodeOrDie(op.Inode)

View File

@ -1,3 +1,4 @@
//go:build go1.8
// +build go1.8
package memfs_test

View File

@ -1,3 +1,4 @@
//go:build !go1.8
// +build !go1.8
package memfs_test

View File

@ -16,6 +16,7 @@ package memfs_test
import (
"bytes"
"encoding/binary"
"io"
"io/ioutil"
"os"
@ -31,6 +32,7 @@ import (
fallocate "github.com/detailyang/go-fallocate"
"github.com/jacobsa/fuse"
"github.com/jacobsa/fuse/fusetesting"
"github.com/jacobsa/fuse/internal/fusekernel"
"github.com/jacobsa/fuse/samples"
"github.com/jacobsa/fuse/samples/memfs"
. "github.com/jacobsa/oglematchers"
@ -88,6 +90,15 @@ func applyUmask(m os.FileMode) os.FileMode {
return m &^ os.FileMode(umask)
}
func (t *MemFSTest) checkOpenFlagsXattr(
fileName string, expectedOpenFlags fusekernel.OpenFlags) {
dest := make([]byte, 4)
_, err := unix.Getxattr(fileName, memfs.FileOpenFlagsXattrName, dest)
AssertEq(nil, err)
openFlags := binary.LittleEndian.Uint32(dest)
AssertEq(openFlags, uint32(expectedOpenFlags))
}
////////////////////////////////////////////////////////////////////////
// Boilerplate
////////////////////////////////////////////////////////////////////////
@ -292,7 +303,7 @@ func (t *MemFSTest) CreateNewFile_InRoot() {
var stat *syscall.Stat_t
// Write a file.
fileName := path.Join(t.Dir, "foo")
fileName := path.Join(t.Dir, memfs.CheckFileOpenFlagsFileName)
const contents = "Hello\x00world"
createTime := time.Now()
@ -304,7 +315,7 @@ func (t *MemFSTest) CreateNewFile_InRoot() {
stat = fi.Sys().(*syscall.Stat_t)
AssertEq(nil, err)
ExpectEq("foo", fi.Name())
ExpectEq(memfs.CheckFileOpenFlagsFileName, fi.Name())
ExpectEq(len(contents), fi.Size())
ExpectEq(applyUmask(0400), fi.Mode())
ExpectThat(fi, fusetesting.MtimeIsWithin(createTime, timeSlop))
@ -321,6 +332,7 @@ func (t *MemFSTest) CreateNewFile_InRoot() {
slice, err := ioutil.ReadFile(fileName)
AssertEq(nil, err)
ExpectEq(contents, string(slice))
t.checkOpenFlagsXattr(fileName, fusekernel.OpenReadOnly)
}
func (t *MemFSTest) CreateNewFile_InSubDir() {
@ -372,7 +384,7 @@ func (t *MemFSTest) ModifyExistingFile_InRoot() {
var stat *syscall.Stat_t
// Write a file.
fileName := path.Join(t.Dir, "foo")
fileName := path.Join(t.Dir, memfs.CheckFileOpenFlagsFileName)
createTime := time.Now()
err = ioutil.WriteFile(fileName, []byte("Hello, world!"), 0600)
@ -382,6 +394,7 @@ func (t *MemFSTest) ModifyExistingFile_InRoot() {
f, err := os.OpenFile(fileName, os.O_WRONLY, 0400)
t.ToClose = append(t.ToClose, f)
AssertEq(nil, err)
t.checkOpenFlagsXattr(fileName, fusekernel.OpenWriteOnly)
modifyTime := time.Now()
n, err = f.WriteAt([]byte("H"), 0)
@ -393,7 +406,7 @@ func (t *MemFSTest) ModifyExistingFile_InRoot() {
stat = fi.Sys().(*syscall.Stat_t)
AssertEq(nil, err)
ExpectEq("foo", fi.Name())
ExpectEq(memfs.CheckFileOpenFlagsFileName, fi.Name())
ExpectEq(len("Hello, world!"), fi.Size())
ExpectEq(applyUmask(0600), fi.Mode())
ExpectThat(fi, fusetesting.MtimeIsWithin(modifyTime, timeSlop))
@ -410,6 +423,7 @@ func (t *MemFSTest) ModifyExistingFile_InRoot() {
slice, err := ioutil.ReadFile(fileName)
AssertEq(nil, err)
ExpectEq("Hello, world!", string(slice))
t.checkOpenFlagsXattr(fileName, fusekernel.OpenReadOnly)
}
func (t *MemFSTest) ModifyExistingFile_InSubDir() {
@ -832,7 +846,7 @@ func (t *MemFSTest) AppendMode() {
buf := make([]byte, 1024)
// Create a file with some contents.
fileName := path.Join(t.Dir, "foo")
fileName := path.Join(t.Dir, memfs.CheckFileOpenFlagsFileName)
err = ioutil.WriteFile(fileName, []byte("Jello, "), 0600)
AssertEq(nil, err)
@ -840,6 +854,7 @@ func (t *MemFSTest) AppendMode() {
f, err := os.OpenFile(fileName, os.O_RDWR|os.O_APPEND, 0600)
t.ToClose = append(t.ToClose, f)
AssertEq(nil, err)
t.checkOpenFlagsXattr(fileName, fusekernel.OpenReadWrite)
// Seek to somewhere silly and then write.
off, err = f.Seek(2, 0)
@ -907,7 +922,7 @@ func (t *MemFSTest) ReadsPastEndOfFile() {
func (t *MemFSTest) Truncate_Smaller() {
var err error
fileName := path.Join(t.Dir, "foo")
fileName := path.Join(t.Dir, memfs.CheckFileOpenFlagsFileName)
// Create a file.
err = ioutil.WriteFile(fileName, []byte("taco"), 0600)
@ -917,6 +932,7 @@ func (t *MemFSTest) Truncate_Smaller() {
f, err := os.OpenFile(fileName, os.O_RDWR, 0)
t.ToClose = append(t.ToClose, f)
AssertEq(nil, err)
t.checkOpenFlagsXattr(fileName, fusekernel.OpenReadWrite)
// Truncate it.
err = f.Truncate(2)
@ -931,6 +947,7 @@ func (t *MemFSTest) Truncate_Smaller() {
contents, err := ioutil.ReadFile(fileName)
AssertEq(nil, err)
ExpectEq("ta", string(contents))
t.checkOpenFlagsXattr(fileName, fusekernel.OpenReadOnly)
}
func (t *MemFSTest) Truncate_SameSize() {

View File

@ -50,6 +50,7 @@ func main() {
cfg := &fuse.MountConfig{
ReadOnly: *fReadOnly,
FuseImpl: fuse.FUSEImplMacFUSE,
}
if *fDebug {

View File

@ -0,0 +1,68 @@
// Copyright 2015 Google Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package main
import (
"context"
"flag"
"log"
"os/user"
"strconv"
"github.com/jacobsa/fuse"
"github.com/jacobsa/fuse/samples/memfs"
)
var fMountPoint = flag.String("mount_point", "", "Path to mount point.")
func main() {
flag.Parse()
if *fMountPoint == "" {
log.Fatalf("You must set --mount_point.")
}
user, err := user.Current()
if err != nil {
panic(err)
}
uid, err := strconv.ParseUint(user.Uid, 10, 32)
if err != nil {
panic(err)
}
gid, err := strconv.ParseUint(user.Gid, 10, 32)
if err != nil {
panic(err)
}
server := memfs.NewMemFS(uint32(uid), uint32(gid))
cfg := &fuse.MountConfig{
// Disable writeback caching so that pid is always available in OpContext
DisableWritebackCaching: true,
}
mfs, err := fuse.Mount(*fMountPoint, server, cfg)
if err != nil {
log.Fatalf("Mount: %v", err)
}
// Wait for it to be unmounted.
if err = mfs.Join(context.Background()); err != nil {
log.Fatalf("Join: %v", err)
}
}

View File

@ -0,0 +1,58 @@
package main
import (
"context"
"flag"
"fmt"
"github.com/jacobsa/fuse"
"github.com/jacobsa/fuse/samples/readbenchfs"
"log"
"net/http"
_ "net/http/pprof"
"os"
)
var fMountPoint = flag.String("mount_point", "", "Path to mount point.")
var fReadOnly = flag.Bool("read_only", false, "Mount in read-only mode.")
var fVectored = flag.Bool("vectored", false, "Use vectored read.")
var fDebug = flag.Bool("debug", false, "Enable debug logging.")
var fPprof = flag.Int("pprof", 0, "Enable pprof profiling on the specified port.")
func main() {
flag.Parse()
if *fPprof != 0 {
go func() {
fmt.Printf("%v", http.ListenAndServe(fmt.Sprintf("localhost:%v", *fPprof), nil))
}()
}
server, err := readbenchfs.NewReadBenchServer(*fVectored)
if err != nil {
log.Fatalf("makeFS: %v", err)
}
// Mount the file system.
if *fMountPoint == "" {
log.Fatalf("You must set --mount_point.")
}
cfg := &fuse.MountConfig{
ReadOnly: *fReadOnly,
UseVectoredRead: *fVectored,
}
if *fDebug {
cfg.DebugLogger = log.New(os.Stderr, "fuse: ", 0)
}
mfs, err := fuse.Mount(*fMountPoint, server, cfg)
if err != nil {
log.Fatalf("Mount: %v", err)
}
// Wait for it to be unmounted.
if err = mfs.Join(context.Background()); err != nil {
log.Fatalf("Join: %v", err)
}
}

View File

@ -0,0 +1,170 @@
// Copyright 2021 Vitaliy Filippov
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package readbenchfs
import (
"golang.org/x/net/context"
"io"
"math/rand"
"os"
"github.com/jacobsa/fuse"
"github.com/jacobsa/fuse/fuseops"
"github.com/jacobsa/fuse/fuseutil"
)
type readBenchFS struct {
fuseutil.NotImplementedFileSystem
buf []byte
useVectoredRead bool
}
// 1 TB
const fileSize = 1024 * 1024 * 1024 * 1024
var _ fuseutil.FileSystem = &readBenchFS{}
func NewReadBenchServer(useVectoredRead bool) (server fuse.Server, err error) {
// 1 GB of random data to exceed CPU cache
buf := make([]byte, 1024*1024*1024)
rand.Read(buf)
server = fuseutil.NewFileSystemServer(&readBenchFS{
buf: buf,
useVectoredRead: useVectoredRead,
})
return
}
func (fs *readBenchFS) StatFS(ctx context.Context, op *fuseops.StatFSOp) error {
return nil
}
func (fs *readBenchFS) LookUpInode(ctx context.Context, op *fuseops.LookUpInodeOp) error {
if op.Name == "test" {
op.Entry = fuseops.ChildInodeEntry{
Child: 2,
Attributes: fuseops.InodeAttributes{
Size: fileSize,
Nlink: 1,
Mode: 0444,
},
}
return nil
}
return fuse.ENOENT
}
func (fs *readBenchFS) GetInodeAttributes(ctx context.Context, op *fuseops.GetInodeAttributesOp) error {
if op.Inode == 1 {
op.Attributes = fuseops.InodeAttributes{
Nlink: 1,
Mode: 0755 | os.ModeDir,
}
return nil
} else if op.Inode == 2 {
op.Attributes = fuseops.InodeAttributes{
Size: fileSize,
Nlink: 1,
Mode: 0444,
}
return nil
}
return fuse.ENOENT
}
func (fs *readBenchFS) OpenDir(ctx context.Context, op *fuseops.OpenDirOp) error {
// Allow opening any directory.
return nil
}
func (fs *readBenchFS) ReadDir(ctx context.Context, op *fuseops.ReadDirOp) error {
if op.Inode != 1 {
return fuse.ENOENT
}
if op.Offset > 0 {
return nil
}
entries := []fuseutil.Dirent{
fuseutil.Dirent{
Offset: 1,
Inode: 2,
Name: "test",
Type: fuseutil.DT_File,
},
}
for _, e := range entries[op.Offset:] {
n := fuseutil.WriteDirent(op.Dst[op.BytesRead:], e)
if n == 0 {
break
}
op.BytesRead += n
}
return nil
}
func (fs *readBenchFS) OpenFile(ctx context.Context, op *fuseops.OpenFileOp) error {
// Allow opening any file.
return nil
}
func (fs *readBenchFS) ReadFile(ctx context.Context, op *fuseops.ReadFileOp) error {
if op.Offset > fileSize {
return io.EOF
}
end := op.Offset + op.Size
if end > fileSize {
end = fileSize
}
buflen := int64(len(fs.buf))
for pos := op.Offset; pos < end; {
s := pos % buflen
e := buflen
if e-s > end-pos {
e = s + end - pos
}
if fs.useVectoredRead {
op.Data = append(op.Data, fs.buf[s:e])
} else {
copy(op.Dst[pos-op.Offset:], fs.buf[s:])
}
pos = op.Offset + e
}
op.BytesRead = int(end - op.Offset)
return nil
}
func (fs *readBenchFS) ReleaseDirHandle(ctx context.Context, op *fuseops.ReleaseDirHandleOp) error {
return nil
}
func (fs *readBenchFS) GetXattr(ctx context.Context, op *fuseops.GetXattrOp) error {
return nil
}
func (fs *readBenchFS) ListXattr(ctx context.Context, op *fuseops.ListXattrOp) error {
return nil
}
func (fs *readBenchFS) ForgetInode(ctx context.Context, op *fuseops.ForgetInodeOp) error {
return nil
}
func (fs *readBenchFS) ReleaseFileHandle(ctx context.Context, op *fuseops.ReleaseFileHandleOp) error {
return nil
}
func (fs *readBenchFS) FlushFile(ctx context.Context, op *fuseops.FlushFileOp) error {
return nil
}

View File

@ -21,7 +21,7 @@ import (
"path/filepath"
"sync"
"sync/atomic"
"time"
"syscall"
"github.com/jacobsa/fuse/fuseops"
"github.com/jacobsa/fuse/fuseutil"
@ -53,21 +53,20 @@ func getOrCreateInode(inodes *sync.Map, parentId fuseops.InodeID, name string) (
return nil, nil
}
parentPath := parent.(Inode).Path()
entries, err := ioutil.ReadDir(parentPath)
path := filepath.Join(parentPath, name)
fileInfo, err := os.Stat(path)
if err != nil {
return nil, err
return nil, nil
}
for _, entry := range entries {
if entry.Name() == name {
inodeEntry := &inodeEntry{
id: nextInodeID(),
path: filepath.Join(parentPath, name),
}
storedEntry, _ := inodes.LoadOrStore(inodeEntry.id, inodeEntry)
return storedEntry.(Inode), nil
}
stat, _ := fileInfo.Sys().(*syscall.Stat_t)
inodeEntry := &inodeEntry{
id: fuseops.InodeID(stat.Ino),
path: path,
}
return nil, nil
storedEntry, _ := inodes.LoadOrStore(inodeEntry.id, inodeEntry)
return storedEntry.(Inode), nil
}
type inodeEntry struct {
@ -101,16 +100,14 @@ func (in *inodeEntry) Attributes() (*fuseops.InodeAttributes, error) {
if err != nil {
return &fuseops.InodeAttributes{}, err
}
return &fuseops.InodeAttributes{
Size: uint64(fileInfo.Size()),
Nlink: 1,
Mode: fileInfo.Mode(),
Atime: fileInfo.ModTime(),
Mtime: fileInfo.ModTime(),
Ctime: time.Now(),
Crtime: time.Now(),
Uid: uid,
Gid: gid,
Size: uint64(fileInfo.Size()),
Nlink: 1,
Mode: fileInfo.Mode(),
Mtime: fileInfo.ModTime(),
Uid: uid,
Gid: gid,
}, nil
}
@ -119,12 +116,12 @@ func (in *inodeEntry) ListChildren(inodes *sync.Map) ([]*fuseutil.Dirent, error)
if err != nil {
return nil, err
}
dirents := make([]*fuseutil.Dirent, len(children))
dirents := []*fuseutil.Dirent{}
for i, child := range children {
childInode, err := getOrCreateInode(inodes, in.id, child.Name())
if err != nil || childInode == nil {
return nil, nil
continue
}
var childType fuseutil.DirentType
@ -136,12 +133,12 @@ func (in *inodeEntry) ListChildren(inodes *sync.Map) ([]*fuseutil.Dirent, error)
childType = fuseutil.DT_File
}
dirents[i] = &fuseutil.Dirent{
dirents = append(dirents, &fuseutil.Dirent{
Offset: fuseops.DirOffset(i + 1),
Inode: childInode.Id(),
Name: child.Name(),
Type: childType,
}
})
}
return dirents, nil
}

View File

@ -121,7 +121,7 @@ func (fs *readonlyLoopbackFs) ReadDir(
}
if op.Offset > fuseops.DirOffset(len(children)) {
return fuse.EIO
return nil
}
children = children[op.Offset:]
@ -139,8 +139,13 @@ func (fs *readonlyLoopbackFs) ReadDir(
func (fs *readonlyLoopbackFs) OpenFile(
ctx context.Context,
op *fuseops.OpenFileOp) error {
// Allow opening any file.
var _, found = fs.inodes.Load(op.Inode)
if !found {
return fuse.ENOENT
}
return nil
}
func (fs *readonlyLoopbackFs) ReadFile(

View File

@ -16,6 +16,7 @@ package statfs_test
import (
"fmt"
"github.com/jacobsa/oglematchers"
"math"
"regexp"
"syscall"
@ -26,9 +27,8 @@ import (
// Sample output:
//
// Filesystem 1024-blocks Used Available Capacity iused ifree %iused Mounted on
// fake@bucket 32 16 16 50% 0 0 100% /Users/jacobsa/tmp/mp
//
// Filesystem 1024-blocks Used Available Capacity iused ifree %iused Mounted on
// fake@bucket 32 16 16 50% 0 0 100% /Users/jacobsa/tmp/mp
var gDfOutputRegexp = regexp.MustCompile(`^\S+\s+(\d+)\s+(\d+)\s+(\d+)\s+\d+%\s+\d+\s+\d+\s+\d+%.*$`)
////////////////////////////////////////////////////////////////////////
@ -69,7 +69,7 @@ func (t *StatFSTest) Syscall_ZeroValues() {
ExpectEq(0, stat.Bavail)
ExpectEq(0, stat.Files)
ExpectEq(0, stat.Ffree)
ExpectEq("osxfuse", convertName(stat.Fstypename[:]))
ExpectThat(convertName(stat.Fstypename[:]), oglematchers.AnyOf(oglematchers.Equals("osxfuse"), oglematchers.Equals("macfuse")))
ExpectEq(t.canonicalDir, convertName(stat.Mntonname[:]))
ExpectEq(fsName, convertName(stat.Mntfromname[:]))
}
@ -104,7 +104,7 @@ func (t *StatFSTest) Syscall_NonZeroValues() {
ExpectEq(canned.BlocksAvailable, stat.Bavail)
ExpectEq(canned.Inodes, stat.Files)
ExpectEq(canned.InodesFree, stat.Ffree)
ExpectEq("osxfuse", convertName(stat.Fstypename[:]))
ExpectThat(convertName(stat.Fstypename[:]), oglematchers.AnyOf(oglematchers.Equals("osxfuse"), oglematchers.Equals("macfuse")))
ExpectEq(t.canonicalDir, convertName(stat.Mntonname[:]))
ExpectEq(fsName, convertName(stat.Mntfromname[:]))
}

View File

@ -26,9 +26,8 @@ import (
// Sample output:
//
// Filesystem 1K-blocks Used Available Use% Mounted on
// some_fuse_file_system 512 64 384 15% /tmp/sample_test001288095
//
// Filesystem 1K-blocks Used Available Use% Mounted on
// some_fuse_file_system 512 64 384 15% /tmp/sample_test001288095
var gDfOutputRegexp = regexp.MustCompile(`^\S+\s+(\d+)\s+(\d+)\s+(\d+)\s+\d+%.*$`)
////////////////////////////////////////////////////////////////////////

View File

@ -1,3 +1,4 @@
//go:build !linux
// +build !linux
package fuse

29
writev.go Normal file
View File

@ -0,0 +1,29 @@
package fuse
import (
"syscall"
"unsafe"
)
func writev(fd int, packet [][]byte) (n int, err error) {
iovecs := make([]syscall.Iovec, 0, len(packet))
for _, v := range packet {
if len(v) == 0 {
continue
}
vec := syscall.Iovec{
Base: &v[0],
}
vec.SetLen(len(v))
iovecs = append(iovecs, vec)
}
n1, _, e1 := syscall.Syscall(
syscall.SYS_WRITEV,
uintptr(fd), uintptr(unsafe.Pointer(&iovecs[0])), uintptr(len(iovecs)),
)
n = int(n1)
if e1 != 0 {
err = syscall.Errno(e1)
}
return
}