Some servers sometimes do not return attrivbutes for files in the RDP
replies. So we need to fallback to using LOOKUPs for these entries
just like we always have to do in the READDIR case.
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
Update the configure to add some sanity -W arguments.
A good start is probably :
-Wall -Werror -Wshadow -Wno-write-strings -Wstrict-prototypes
-Wpointer-arith -Wcast-align -Wno-strict-aliasing
Fixup the paces in the code that triggers.
(one of which is readahead code which is perhaps broken?)
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
- Auto-traverse mounts. With this option (default to on) libnfs will
autodiscover and handle any nested submounts.
- Remove nfs_get_current_offset. Applications should use seek instead of this function.
- Add umask() support.
- Change set_tcp_sockopt() to be static.
- Android fix for nfs-ls
- Make S_IFLNK available on windows.
- Fix a use after free.
- Fix a bug where truncate() treated offset as 32bit.
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
Make the continue_int field we use through the internal callbacks a uint64_t
instead of an int.
This fixes a bug for the truncate function where we pass the offset to truncate
to via this field.
The bug is that otherwise we first truncate the field to an int and then
in the callback we cast this int back to a uint64_t again.
If the user called truncate with an offset that is >= 2^31
then :
IF the 1<<31 bit is cleared then we would truncate to (offset & 0xffffffff)
IF the 1<<31 bit is set, we would instead truncate to (offset | 0xffffffff00000000)
Reported-by: doktorstick at github
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
Fix retrieving the attributes for a submount in nfs_opendir when
the submount is an entry of '/'.
Signed-off-by: Ross Lagerwall <rosslagerwall@gmail.com>
Add a URL argument to enable/disable the use of automatic traversal of nested
mounts for a libnfs context. Default it to enabled.
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
When opendir_cb encounters an entry that refers to a nested mount, then
replace the attributes with those attributes we collected for this export
during the mount and return those instead.
This makes traversing a director that crosses into a different filesystem
on the server transparent to the client.
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
Durng nfs_lookuppath_async, check if the requested path traverses into
a nested mountpoint and if so, skip resolving that part of thre path and just
use the filehandle for the nested mount.
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
During the mount process, once we have connected to NFSd we need to collect
the file attributes for all the filehandles that are associated with
nested mounts.
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
During the mount process, call MOUNT3 EXPORT and collect a list of all
exports on the server. For any export that is a nested mount to the current
directory we are trying to mount call out to MOUNT3 MNT and collect the
filehandle for those mounts.
Track all nested mounts and their filehandles in a list from the nfs_context.
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
Rename these callback functions to make space for two new functions
nfs_mount_7/8_cb which we will be using to collect information about
nested mounts in a later patch.
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
NFS reports type in a separate to mode, where mode only contains
the protection bits. This makes it inconvenient to use when porting programs
since under posix the type is supposed to be part of the mode bits (S_IFMT)
When unmarshalling the directory entries into a nfsdirent structure, bake
the S_IF* file type into where the S_IFMT bits would be.
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
- Add O_TRUNC support for nfs_create
- Handle OOM during create
- Return more stats fields as part of readdir since we get these for "free"
when we use READDIRPLUS
- Follow symlinks during path resolution
- Add lchown, lstat and lutimes
- Replace all [u_]quad types with [u]int types in our RPC layer
- Solaris build fixes
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
Add nfs_access2(), like nfs_access() but it returns the individual
statuses of R_OK, W_OK and X_OK rather than a single success or failure
status. This saves the latency and overhead of multiple lookups if an
application tries to determine the status of each of R_OK, W_OK and
X_OK.
Signed-off-by: Ross Lagerwall <rosslagerwall@gmail.com>
Map ACCESS3_{MODIFY,EXTEND,DELETE} to W_OK and ACCESS3_{LOOKUP,EXECUTE}
to X_OK so that nfs_access() gives sensible results for directories.
Signed-off-by: Ross Lagerwall <rosslagerwall@gmail.com>
All current platforms have a quad type that maps to a 64bit scalar.
But there are platforms where quad maps to a 64bit non-scalar.
Replace quad with int64 in the protocol definitions and the ZDR layer
so that these fields will map to a 64 bit scalar also on those platforms
where quad can not be used.
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
Include the proper headers to fix warnings like:
libnfs-sync.c:1529:3: warning: implicit declaration of function 'gettimeofday' [-Wimplicit-function-declaration]
libnfs-zdr.c:506:2: warning: implicit declaration of function 'getuid' [-Wimplicit-function-declaration]
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
sys/time.h needs to be protected with an ifdef
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
Since the user callback may perform operations on the nfsfh (e.g. it
might close it), all updates should be done before the user callback is
called.
Signed-off-by: Ross Lagerwall <rosslagerwall@gmail.com>
Add lchmod which is like chmod but operates on the symbolic link itself
if the destination is a symbolic link.
Signed-off-by: Ross Lagerwall <rosslagerwall@gmail.com>
Add lutimes which is like utimes but operates on the symbolic link
itself if the destination is a symbolic link.
Signed-off-by: Ross Lagerwall <rosslagerwall@gmail.com>
Add lstat which is like stat but operates on the symbolic link itself if
the destination is a symbolic link.
Signed-off-by: Ross Lagerwall <rosslagerwall@gmail.com>
Add lchown which is like chown but operates on the symbolic link itself
if the destination is a symbolic link.
Signed-off-by: Ross Lagerwall <rosslagerwall@gmail.com>
Follow symlinks during path resolution. If the symlink points outside
the mount, -ENOENT is returned. This is slightly different behavior
from the in-kernel NFS client where symlinks pointing outside the mount
get resolved to local paths.
The algorithm for symlink resolution is simple and stupid. If a symlink
is encountered, the path is rewritten and path resolution begins again
from the root filehandle. A count is kept to prevent loops. This is
not particularly efficient but it is good enough for now.
Signed-off-by: Ross Lagerwall <rosslagerwall@gmail.com>
Set as much stat information as possible for stat, stat64, fstat and
readdir.
Fill in dev to the given fsid.
Fill in rdev to the given major and minor numbers.
Set the file type bits in the mode from the type returned by the server.
Set the number of blocks used based on the number of bytes used in
blocks of size 512 (which is what stat(2) uses), rounded up.
Fill in the nanosecond timestamps.
Signed-off-by: Ross Lagerwall <rosslagerwall@gmail.com>
This version removes old ONC-RPC symbols and automatically includes the
RPC/ZDR layer include from the raw low level headers.
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
Since rpc_connect_async sets autoreconnect to 0, turn autoreconnect on
after the connection has completed, not before.
This fixes#76.
Signed-off-by: Ross Lagerwall <rosslagerwall@gmail.com>
Add a new family of functions, nfs_create, like nfs_creat but takes an
additional flags argument which allows extra flags like O_SYNC, O_EXCL
and O_APPEND to be specified.
New version of libnfs:
- IPv6 support
- Support for Portmapper version 3 and an example portmap client.
- Directory caching top make repeated opendir() calls faster
- Readahead support
- Build manpages for the utilities (==nfs-ls)
- Support for O_APPEND
- Rename the list macros to avoid collission on *BSD
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
This patch add support for an internal readahead machanism. The maximum readahead
size can be specified via URL parameter readahead. This should significantly
speed up small sequential reads.
Signed-off-by: Peter Lieven <pl@kamp.de>
Always pass a fattr3 structure to the callbacks for the internal function
to perform recursive lookups : nfs_lookuppath_async().
This will allow us to access for example the mtime for an object before we
start performing any expensive functions.
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
Use the macro when removing the pdus in the wait list from the queues.
Also make sure to remove them from the right queue, from waitqueue and not
the outqueue for PDUs we have already sent out.
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
commit 1c1e09a completely broke connects for non broadcast
traffic since it forgot to copy the server address into
the socket_storage struct.
Signed-off-by: Peter Lieven <pl@kamp.de>
Since the interface is modelled after the libc calls we should try to match
their behaviour to avoid unpleasant surprises:
* read / write (sync and async flavours) update the file position
* pread / pwrite (sync and async flavours) do not update the file position
.
Signed-off-by: Arne Redlich <arne.redlich@googlemail.com>
Previously nfs_read and nfs_write used to use nfs_pread and nfs_pwrite respectively.
In preparation of getting the file position handling right this has to be detangled.
Signed-off-by: Arne Redlich <arne.redlich@googlemail.com>
NFS servers can respond to requests in any order, and they do. In our
tests there is also some clustering to the responses; it could be
because eg. requests are served synchronously if the data is in the cache.
Introduce a hash table so that we are able to find the pdu quickly in
all cases, assuming random distribution of the responses.
Fixes a bug where the next pointer was not being explicitly set. We
were ok much of the time due to zero-filled memory, and also we need
this if the same pdu is re-queued.
When making many concurrent requests (as is likely in any performance
criticial application), the use of SLIST_REMOVE and SLIST_ADD_END are
a severe bottleneck because of their linear search.
I considered using a double-linked list but it was unnecessary to
allocate the additional memory for each list entry.
Instead, continue to use a single-linked list but retain:
* a pointer to the end of the list; and
* a pointer to the previous entry during a linear search.
The former would makes append operations O(1) time, and the latter
does the same for removal. We can do this because removal only happens
within the linear search, and there is no random access to the queue.
Otherwise end up with a null string which is not permitted (RFC 1813, 3.2;
the code checks for it right after the now fixed nullification of "/").
Signed-off-by: Arne Redlich <arne.redlich@googlemail.com>
This adds basic IPv6 support to libnfs.
Since libnfs currently only support PORTMAPPER protocol up to version 2
the IPv6 support only works if the server runs Both MOUNT and NFS protocols
on the same ports for IPv6 as for IPv4.
To get full IPv6 support we need to add support for PORTMAPPER version 3
and use it for discovery when using IPv6
Wed Mar 19 2014: Version 1.9.3
- Add O_TRUNC support to nfs_open()
- Add a simple but incomplete LD_PRELOAD tool
- Fixes for some memory leaks and C++ compile support
- Make ANDROID default uid/gid to 65534
- Allow the READDIRPLUS emulation to still work if some objects
in the direcotry can not be lookedup (NFSv4 ACL denying READ-ATTRIBUTES)
- Have libnfs retry any read/write operations where the server responds
with a short read/write. Some servers do this when they are overloaded?
if there are already requests in flight we cannot return with an error immediately
from the functions since the caller will likely tidy up his data structures directly
and later on we call his callback with private_data that has likely already
been freed.
Signed-off-by: Peter Lieven <pl@kamp.de>
the RFC allows the server to read less bytes than requested even
if not at the EOF.
this patch implements a reissue logic for the reminder of the
read request(s).
Signed-off-by: Peter Lieven <pl@kamp.de>
the RFC allows the server to write less bytes than requested.
this patch implements a reissue logic for the reminder of the
write request(s).
Signed-off-by: Peter Lieven <pl@kamp.de>
when calculation the max_offset the (unsigned) leads to a cast
to a 32-bit unsigned integer depending on the platform. as a result
we update the max_offset everytime when it grows beyond 2^32.
this leads to a wrong return max_offset value if the callbacks
are received out of order.
Signed-off-by: Peter Lieven <pl@kamp.de>
On servers with extended attributes, a server copuld be set up to
deny READ-ATTRIBUTES for the libnfs user.
This means that READDIRPLUS will no longer work since it will need to
stat() and thus READ-ATTRIBUTE in order to prepare the response.
Libnfs has READDIRPLUS emulation for the cases where this command fails
by switching to old READDIR to scan all the file names and then a LOOKUP loop for getting the file attributes.
Most of the time the purpose for this emulation is to handle the case where the server simply does not support READDIRPLUS at all, which sometimes is the case for embedded systems with userspace nfs servers.
In this case, where files just have READ-ATTRIBUTE deny for the libnfs user,
this will also fail and trigger the fallback to READDIR + LOOKUP-loop.
If the LOOKUP fails for this loop, then just ignore trying to update the attributes we have for this object, but do not fail the actual READDIRPLUS emulation.
This addresses a permissions issue reported by a XBMC user in issue #60
Spotted by clang analyzer.
This also introduces asserts to help clang analyzer avoid reporting
false positives.
Signed-off-by: Arne Redlich <arne.redlich@googlemail.com>
Spotted by clang analyzer.
This also introduces asserts to help clang analyzer avoid
reporting false positives.
Signed-off-by: Arne Redlich <arne.redlich@googlemail.com>
These were uncovered by the previously added __attribute__((format(printf))).
Emacs also removed trailing whitespace while at it.
Signed-off-by: Arne Redlich <arne.redlich@googlemail.com>
Spotted by clang analyzer.
This introduces another allocation to create a copy of the target path
of a rename in case it needs to be reported via rpc_set_error - it might
be a better idea to avoid the allocation and have a slightly less informative
error message?
Signed-off-by: Arne Redlich <arne.redlich@googlemail.com>
free() can cope with nullptrs and there's no point in null-ing free'd members
as the containing struct is free'd as well.
Signed-off-by: Arne Redlich <arne.redlich@googlemail.com>
Non-debug builds will trip over the nullptr too - do we rather
want to leak the memory there?
Signed-off-by: Arne Redlich <arne.redlich@googlemail.com>
nfs_chown_data is hooked up under nfs_cb_data->continue_data but
no ->free_continue_data is configured, so once free_nfs_cb_data is
invoked it will trip over a nullptr.
Signed-off-by: Arne Redlich <arne.redlich@googlemail.com>
Spotted by clang:
../../libnfs.git/lib/libnfs.c:1002:13: warning: using the result of an assignment as a condition without parentheses
[-Wparentheses]
while (str = strstr(path, "//")) {
~~~~^~~~~~~~~~~~~~~~~~~~
../../libnfs.git/lib/libnfs.c:1002:13: note: place parentheses around the assignment to silence this warning
while (str = strstr(path, "//")) {
^
( )
../../libnfs.git/lib/libnfs.c:1002:13: note: use '==' to turn this assignment into an equality comparison
while (str = strstr(path, "//")) {
^
==
Make the intent clear by adding extra parentheses, and also
remove trailing whitespace from libnfs.c while at it.
Signed-off-by: Arne Redlich <arne.redlich@googlemail.com>
O_TRUNC will attempt to truncate the file when opened with O_RDWR
or O_WRONLY.
Normal posix open(O_RDONLY|O_TRUNC) is undefined.
libnfs nfs_open() only uses the O_TRUNC flag when used in combination with either O_RDWR or O_WRONLY.
When O_TRUNC is used together with O_RDONLY libnfs will silently ignore the O_TRUNC flag.
Libnfs nfs_open(O_RDONLY|O_TRUNC) is thus the same as nfs_open(O_RDONLY)
This is mainly needed when having to track and control the file descriptors that are used by libnfs, for example when trying to emulate dup2() ontop
of libnfs.
Add chdir and getcwd and store cwd in the nfs_context.
Add functions to process the paths specified and normalize them
by performing the transforms :
// -> /
/./ -> /
^/../ -> error
^[^/] -> error
/string/../ -> /
/$ -> \0
/.$ -> \0
^/..$ -> error
/string/..$ -> /
Update the path lookup function to allow specifying relative paths based on
cwd for all functions.
Add chdir and getcwd and store cwd in the nfs_context.
Add functions to process the paths specified and normalize them
by performing the transforms :
// -> /
/./ -> /
^/../ -> error
^[^/] -> error
/string/../ -> /
/$ -> \0
/.$ -> \0
^/..$ -> error
/string/..$ -> /
Update the path lookup function to allow specifying relative paths based on
cwd for all functions.
- Use _stat64 on windows so file sizes become 64bit always.
- Increase default marshalling buffer so we can marshall large PDUs.
- RPC layer support for NFSv2
- Win32 updates and fixes
- Add URL parsing functions and URL argument support.
- New utility: nfs-io
- nfs-ls enhancements
- RPC layer support for NSM
- Add example FUSE filesystem.
- Minor fixes.
We also get uid/gid for free when using READDIRPLU3 (and READDIRPLUS3 emulation)
so store these too so applications that needs to look at the uid/gid can avoid
the extra call to nfs_stat()
It only makes sense to pass url arguments to an nfs URL and thus a nfs context.
While some arguments may apply to the underlying raw rpc context (uid, gid,
tcp settings etc) some future arguments may need to apply to the nfs context
instead, such as selecting nfs version, whether or not to force sync writes
etc.
This allows to connect with an alternate uid or gid than that
of the current user.
Example:
examples/nfs-ls nfs://10.0.0.1/export?uid=1000&gid=33
Signed-off-by: Peter Lieven <pl@kamp.de>
This allows indirect support for a configurable connect timeout.
Linux uses a exponential backoff for SYN retries starting
with 1 second.
This means for a value n for TCP_SYNCNT, the connect will
effectively timeout after 2^(n+1)-1 seconds.
Example:
examples/nfs-ls nfs://10.0.0.1/export?tcp-syncnt=1
Signed-off-by: Peter Lieven <pl@kamp.de>
This makes it possible for multiple processes/contexts to use the same
target and (with some synchronization) avoid XID collissions across processes/contexts.
This helps for users which rapidly fork a lot of processes that then
immediately create a new context (I am looking at you dbench)
to awoid having lots of processes starting and using overlapping xid values.
We use our own XDR and RPC layer nowadays and do not have
any external dependencies to XDR or RPC.
As such we no longer need to clamp the write size to max 32kb
since we never link to the system rpc/xdr libraries.
(and thus dont have to clamp in case the system library is broken for
pdu's > 32k)
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
- Allow nested eventloops so that a sync function can be called from a callback.
- Fix a bug in unmarshalling a uint64.
- Add PATHCONF support.
- WIN32/64 updates
- AROS updates
Add AROS/Amiga support
Chose initial XID value better to reduce probability for collissions
Fix bug in the initial default credentials and use getgid() instead of -1
use getgid() as the group instead of -1.
Recent linux knfsd do not allow grp==-1
On windows there are no uid/gids in the traditional sense so there I still specify a default credential of uid==gid==-1 :
rpc->auth = authunix_create("LibNFS", 65535, 65535, 0, NULL);
This is I think the sanest/safest thing to do since most servers will have
special handing of -1 meaning 'nobody' or similar.
This should work on many/most servers and give the user the minimum available
access allowed for 'nobody'.
I think on windows (or AROS for that matter) applications will probably have
to invoke and set the credentials themself explicitely.
Those apps probably, unfortunately, also need to have a configuration
setting to select which uid/gid to use when talking to the server.
(or they could hardcode it)
rpc_set_auth(rpc, libnfs_authunix_create("hostname", uid, gid, 0, NULL))
should do the trick if they call immediately after creating the rpc/nfs context.
But dont set it to 0,0 root/root for uid/gid.
First of all, most servers have root-squash so they will re-map this uid/gid
to 'nobody' internally.
But, if the user uses a server that does not do root-squash, then setting this to 0,0 would mean that your app now access the nfs share as root which is probably not what you want.
want to create an object straight under the root directory of what we
mounted.
As always, the actual object to create is then a string starting after the \0 byte
- Add trackig of freed context and assert on using a context after it has been
freed.
- Windows x64 support and fixes.
- Switch to using our own version of xdr_int64() since the one in libtirpc
crashes on some platforms.
- Fix memory leak in an error path for addrinfo.
- Fix bug dereferencing a null pointer in the mount callback on error.
If status is non-zero in the mount callback, then the datapointer might be NULL
so we should not dereference it into an exports pointer before we have checked the status.
This would otherwise cause crashes.
If status is non-zero in the mount callback, then the datapointer might be NULL
so we should not dereference it into an exports pointer before we have checked the status.
This would otherwise cause crashes.
This patch switches libnfs over to use precompiled rpcgen files
and using ZDR. ZDR is a trivial reimplementation of XDR that is built in
into libnfs.
This removes the dependencies of rpc/xdr completely and allow us to build on any
system, even systems where rpcgen and librpc/libxdr are not generally available.
Peter Lieven