fcae6ff0b9
S3: Fine tuning, supporting bucket per file/dir and single bucket.
2020-06-30 20:02:36 +01:00
87c9906d70
Error checking for NULL create/opens
2020-06-30 19:41:49 +01:00
154cf2cde7
LibS3 version that stores fragments as one object each.
2020-06-30 18:43:44 +01:00
2a3838c360
S3: Partial implementation.
2020-06-30 16:50:13 +01:00
81a7a3ab81
Renamed existing S3 impl. which uses (apparently) outdated library. Added dummy support for libS3 with library search.
2020-06-30 13:41:59 +01:00
3a9bd7828d
Ported PMDK API.
2020-06-30 13:12:44 +01:00
82417128cd
Extracted file size routine that is replicated in each module into IOR removing the MPI dependency from AIORI.
2020-06-30 12:33:56 +01:00
3395fc621c
Moved GetFileSize into CheckFileSize, starting to simplify get_file_size (too much replication in modules).
2020-06-30 12:03:05 +01:00
5a5b90cdaa
Run existing check to allow only supported backends in mdtest.
2020-06-30 11:17:46 +01:00
3e6bfd2db7
Added missing header info.
2020-06-29 20:58:45 +01:00
03dbb20594
MDTest changed verification pattern. Read now always checks the first byte/8 bytes for the signature (item number).
...
Added also --verify-write option which performs a read immediately after a write. Supports #206
2020-06-29 20:15:14 +01:00
df3f7082d8
MDTest: Added warnings/errors for errorneous read pattern #206
2020-06-28 17:21:24 +01:00
145c71f7c3
Trivial cleanup: Extracted debug-related stuff into new header.
2020-06-28 17:16:56 +01:00
38b1752bda
JSON issue with results. Fix for #233 .
2020-06-25 14:37:52 +01:00
0bffd14de7
Added --warningAsErrors option to IOR and MDTest and refactored WARNINGs in IOR. #174
2020-06-24 11:13:12 +01:00
c55848f14d
Bugfix based on the pull request for fix-189. Should fix #189 .
2020-06-24 10:15:31 +01:00
d5de9bcc77
Merge branch 'master' into master
2020-06-24 09:50:03 +01:00
6a88b37171
Merge branch 'master' into fix-194
2020-06-24 09:43:37 +01:00
b4a54c34e8
Merge branch 'master' into fix-modules
...
Conflicts:
src/aiori-DAOS.c
src/aiori-DFS.c
2020-06-11 19:55:23 +00:00
e08314f571
update DFS and DAOS drivers to new aiori module interface.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2020-06-11 17:26:58 +00:00
eb2cd27fac
Removed offset from IOR structure.
2020-06-10 17:47:07 +01:00
c828a0f32c
mdtest: use correct number of bytes for posix_memalign of read buffer
...
Fixes a typo in initial posix_memalign commit for DirectIO support.
2020-06-03 01:22:44 +03:00
3eb488939d
mdtest: allocate aligned buffers to support DirectIO
2020-06-03 00:30:38 +03:00
3623f3b8bd
DFS: allow multiple IO phases when ior_run() is called
...
- init/fini can be made multiple times
- reset tunables on fini so they can change on a re-init
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2020-05-31 16:20:58 +00:00
8fa8ef0c02
Call backend statfs() instead of only POSIX-specific variant.
2020-05-31 13:30:31 +01:00
5663593919
Moved generic IOR information to "hint" structure. Backends should only use the "hints" if set.
2020-05-31 12:50:03 +01:00
8098c3740a
Refactored fd into aiori_fd_t for type safety.
2020-05-31 12:12:49 +01:00
c4c19d3245
Fixed type typo in name.
2020-05-31 11:58:34 +01:00
294b8891e6
Created dummy option type to provide type safety. Fixed various call issues for options.
2020-05-31 11:50:15 +01:00
e91b79cea0
Moved Lustre/GPFS/BeeGFS options into POSIX backend. Needs testing. Additional minor fixes.
2020-05-30 20:09:37 +01:00
930ccdc68d
Updated HDF5 to new module specification.
2020-05-30 19:01:20 +01:00
aa9e2ad815
MDTest updated for AIORI changes.
2020-05-30 18:30:26 +01:00
1890aaaeb0
Started to move IOR specific knowledge out of AIORI.
...
For now, and compatibility, IOR options can still be set/internally accessed using the backends init_xfer_options.
This should be removed in the long run to strip away this dependency.
2020-05-30 18:19:48 +01:00
2de4210311
Provide a string option parser function.
2020-05-29 16:05:42 +01:00
edba70e99c
aiori-CEPHFS: don't just unmount on close, release cmount too.
...
Signed-off-by: Mark Nelson <mnelson@redhat.com>
2020-05-28 20:42:26 -05:00
efc6d49832
src/aiori-CEPHFS: Add option checks before init
...
Signed-off-by: Mark Nelson <mnelson@redhat.com>
2020-05-28 19:52:45 -05:00
8abb7a62dc
src/aiori-CEPHFS: Add sync
...
Signed-off-by: Mark Nelson <mnelson@redhat.com>
2020-05-28 22:28:16 +00:00
153b7aa60f
Critical bugfix for ior API (used e.g. in IO500)
...
TODO: Check potential issue when using IOR scripts with different backends.
2020-05-28 21:18:44 +01:00
4eb0ebec55
Add dummy sync to dummy backend.
2020-05-28 21:13:45 +01:00
8c92a1b2ff
Merge branch 'master' of github.com:hpc/ior
2020-05-28 20:12:06 +01:00
f97090a86b
Typo fix.
2020-05-28 20:11:52 +01:00
28fae361f4
dfs: add DFS sync call
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2020-05-28 18:34:31 +00:00
4b417ef1f3
IOR shouldn't fail if it cannot determine file system sizes but warn. Functionality should presumably be moved into AIORI backends.
2020-05-28 19:05:41 +01:00
a536649abb
Clarified some issues in the test framework and added example test. #219
2020-05-20 17:46:35 +01:00
48eb1880e9
For better testing: Fixed fake tasks per node to work with MPI3, added environment variable to fake num nodes.
2020-04-21 10:35:53 +01:00
8bc9a12f80
Merge pull request #221 from daos-stack/mschaara/dfs_l2g
...
DFS driver update to share DFS mount on init
2020-04-14 10:13:55 +01:00
1b8848ccbc
Use l2g-g2l to share the dfs mount instead of opening by each rank.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2020-04-13 19:59:56 +00:00
a21137be3e
Display outlier host names
...
Display host names when using the -j argument (outlierThreshold) so
that IOR could help to better identify slow nodes:
WARNING: for client02, task 6, write elapsed transfer time is 65.738326
2020-03-17 16:31:56 +01:00
657ff8ad8f
Merge pull request #217 from markhpc/wip-aiori-cephfs
...
[RFC] src/aiori-CEPHFS: New libcephfs backend
2020-03-10 12:29:24 -07:00
9649a0c520
src/aiori-CEPHFS: New libcephfs backend
...
Signed-off-by: Mark Nelson <mnelson@redhat.com>
2020-03-10 10:19:41 -05:00
9726aa0ce2
Bugfix API which doesn't set the AIORI backend.
2020-03-07 14:40:59 +00:00
f5ecb9d974
Merge pull request #215 from daos-stack/daos-devel
...
DAOS Driver: Update License & Align with 1.0 API changes
2020-02-26 19:13:48 +00:00
50a4ba0735
DAOS Driver: Update License & Align with 1.0 API changes
...
- License should inherit from IOR License.
- remove checksums from Array API
- add DFS dir oclass option
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2020-02-26 18:53:51 +00:00
ec8d246b33
Merge pull request #214 from markhpc/wip-ior-rados-mark
...
src/aiori-RADOS: Remove Direct IO option check.
2020-02-26 09:06:54 +00:00
e363713c07
src/aiori-RADOS: Remove Direct IO option check.
...
Signed-off-by: Mark Nelson <mnelson@redhat.com>
2020-02-25 18:13:29 -06:00
c702a98376
Strict prototypes for AIORI
2019-12-22 11:21:40 +00:00
6de5cdc6f9
Fixed wrong usage of platform, fixed printf output.
2019-12-21 14:14:27 +00:00
240d5fae79
Trivial fix to ensure all functions are prototypes.
2019-12-21 11:19:02 +00:00
73c5c0efc2
Fix for collective operations by Wolfgang Szoecs #210 .
...
It does have performance implications but is only active if needed.
2019-12-19 13:43:38 +00:00
0666875df3
aiori-IME: Add support of sync and mknod in IME backend
...
This patch adds the support of sync (flush client page cache
for all opened files) and mknod in the IME backend.
2019-12-17 14:51:21 +01:00
7a2d307c74
Merge pull request #208 from jyvet/mknod
...
Fix confusing output of mknod
2019-12-16 12:38:13 -08:00
9da36abaca
Fix confusing output of mknod
...
Context: mknod does not return a file descriptor.
2019-12-16 14:55:26 +01:00
be48206e5f
Fix compilation warnings
2019-12-16 14:51:10 +01:00
3a34b2efb7
Merge pull request #200 from daos-stack/daos-devel
...
DAOS driver update
2019-11-13 10:38:28 +00:00
acfd4673cc
We only need to add dirs to the Hash in the DFS driver.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-11-13 00:09:03 +00:00
ddac5c8dbf
Merge pull request #3 from hpc/master
...
Bring fork up to date
2019-11-12 14:20:53 +00:00
47199e28dd
Not sure why iops and latency are left out of the JSON job output. It is really useful information.
...
Signed-off by: Robert LeBlanc <robert@leblancnet.us>
2019-11-11 23:16:01 +00:00
9c0926ef2a
Moving fsync pmdk functionalty from persist to drain on the advice of the PMDK developers
2019-11-07 20:17:55 +00:00
fbff9b6fb2
add DFS prefix option
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-11-07 18:01:07 +00:00
c24b2f156a
Merge pull request #2 from hpc/master
...
Merging in latest master changes
2019-11-05 15:00:47 +00:00
b4c85798bd
Tidying up PMDK backend
2019-11-05 14:39:31 +00:00
46c5d4e78d
Adding in dual mount functionality for NVRAM within nodes
2019-11-05 14:37:54 +00:00
817603f5d1
Updating copyright statement
2019-10-31 15:31:28 +00:00
fa25c31994
Adding PMDK functionality
2019-10-31 15:29:09 +00:00
3c7ac2df23
Merge pull request #196 from daos-stack/daos-devel
...
update DFS driver with new DAOS API changes.
2019-10-28 16:56:29 -07:00
4409082064
Merge pull request #188 from pkoutoupis/feature-global_default_dir_layout
...
Enable global default dir layout for subdirs in Lustre
2019-10-28 10:43:56 -07:00
92180e022d
update DFS read/write APIs with new DAOS API changes
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-10-28 16:42:44 +00:00
ccba3b11e5
Merge branch 'master' into fix-194
2019-10-23 16:46:00 -06:00
61333af822
Fix #194 .
...
On previous versions of IOR (and most Linux CLI applications in general)
multiple flags can be combined such as -vvv to get verbose=3 rather than
having to specify each separately as -v -v -v. This patch fixes this behavior.
It also fixes issues around handling of the verbose flag where the
output would never be printed as we were comparing verbose < VERBOSE_0
where verbose defaults to 0 and can only increase.
2019-10-23 16:17:37 -06:00
b334c3f6fd
Merge pull request #192 from daos-stack/daos-devel
...
fix -R option to do proper data verification
2019-10-16 16:20:43 +01:00
dc82a1bf2f
ior -R should memset the buffer being read, otherwise a read that does nothing
...
will report success in the data verification phase.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-10-16 14:54:00 +00:00
a69a5916cf
Hotfix for ior -F -u
2019-10-16 09:42:10 +01:00
f40b0744ff
Merge pull request #187 from daos-stack/daos-devel
...
IOR: Add latency and IOPS numbers to each iteration.
2019-10-02 13:09:26 -07:00
b3ecd756ce
Implemented AC_CHECK_HEADERS fix in style of #190
2019-09-30 04:25:59 -07:00
2aea04c8cb
fix some compiler warnings.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-09-21 22:43:20 +00:00
fca0a62ccb
update to new DAOS API for creating dfs containers.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-09-21 22:05:54 +00:00
604598ab2f
Merge pull request #182 from jschwartz-cray/fix-181
...
Fix #181 .
2019-09-19 16:53:41 +01:00
12284ae04a
Add latency and iops numbers to each iteration.
...
- Latency reported is computed by taking the average latency of
all ops from a single task, then taking the minimum of that
between all tasks.
- IOPS is computed by taking the total number of ops across all
tasks divided by the total access time to execute those ops.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-09-10 18:49:32 +00:00
c58ba8ffb5
add break that was accidentally removed
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-09-10 14:11:55 +00:00
fe9d76ddf3
fix segfault when no API is specified to mdtest
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-09-09 22:03:55 +00:00
73dbda09c6
remove printf added
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-09-09 20:19:42 +00:00
5622aabf46
missed Makefile.am update
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-09-09 20:14:12 +00:00
d332d586bf
remove uneeded code and doc
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-09-09 20:07:55 +00:00
e9dd5fe15e
Merge branch 'master' into daos-devel
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Conflicts:
configure.ac
src/Makefile.am
src/aiori-MPIIO.c
src/aiori.c
src/aiori.h
src/ior.c
src/mdtest.c
src/option.c
2019-09-09 19:55:56 +00:00
57a16ddda8
- update debugging error checking in DFS.
...
- remove usleep() before daos_fini()
- fix README
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-09-09 19:14:35 +00:00
0921556456
Merge pull request #184 from hpc/feature-aiori
...
Feature aiori
2019-09-09 15:20:43 +01:00
36d13d3b97
Enable global default dir layout for subdirs in Lustre
2019-09-02 22:26:12 -07:00
c83edfe39b
Extracted check function into aiori. #24 . #177
2019-09-01 15:59:52 +01:00
ced9b0183f
Merge pull request #172 from hpc/fix-168
...
Fix #168
2019-09-01 15:51:44 +01:00
151102ff8f
Merge pull request #176 from otatebe/master
...
abstract I/O interface for Gfarm file system
2019-09-01 15:51:16 +01:00
e3db1759b2
Moded sync() to aiori backend.
2019-09-01 15:47:42 +01:00
6de1ea3176
Merge branch 'feature-sync' into feature-aiori
2019-09-01 15:39:49 +01:00
60a641f911
sync help corrected.
2019-09-01 15:29:12 +01:00
10d3db1dc8
MDTest: fixing the memset() to account for the number of iterations.
2019-08-31 17:28:08 +01:00
47695aea49
Merge pull request #180 from jschwartz-cray/fix-179
...
Fix #179 .
2019-08-31 10:32:41 +01:00
0e952f0f8c
Fix #181 .
...
On systems where numTasks is not evenly divisible by 'tasksPerNode' we were
seeing some nodes reading multiple files while others read none after
reordering.
Commonly all nodes have the same number of tasks but there is nothing
requiring that to be the case. Imagine having 64 tasks running against 4
nodes which can run 20 tasks each. Here you get three groups of 20 and one
group of 4. On this sytem nodes running in the group of 4 were previously
getting tasksPerNode of 4 which meant they reordered tasks differently than
the nodes which got tasksPerNode of 20.
The key to fixing this is ensuring that every node reorders tasks the same
way, which means ensuring they all use the same input values. Obviously on
systems where the number of tasks per node is inconsistent the reordering will
also be inconsistent (some tasks may end up on the same node, or not as far
separated as desired, etc.) but at least this way you'll always end up with a
1:1 reordering.
- Renamed nodes/nodeCount to numNodes
- Renamed tasksPerNode to numTasksOnNode0
- Ensured that numTasksOnNode0 will always have the same value regardless of
which node you're on
- Removed inconsistently used globals numTasksWorld and tasksPerNode and
replaced with per-test params equivalents
- Added utility functions for setting these values:
- numNodes -> GetNumNodes
- numTasks -> GetNumTasks
- numTasksOnNode0 -> GetNumNodesOnTask0
- Improved MPI_VERSION < 3 logic for GetNumNodes so it works when numTasks is
not evenly divisible by numTasksOnNode0
- Left 'nodes' and 'tasksPerNode' in output alone to not break compatibility
- Allowed command-line params to override numTasks, numNodes, and
numTasksOnNode0 but default to using the MPI-calculated values
2019-08-30 16:45:03 -06:00
4c3d96bfed
Fix #179 .
...
-u (uniqueDir) will once again use the full file path specified by the
client instead of truncating it. This was caused by a broken sprintf
which was trying to read/write overlapping buffers.
From the glibc sprintf() documentation:
"The behavior of this function is undefined if copying takes place
between objects that overlap"
2019-08-30 15:31:23 -06:00
0bd979637e
Added some extra debug including ERRF, WARNF, and MPI_CHECKF format string macros.
2019-08-30 15:11:25 -06:00
32db1cd902
add timing for container close.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-08-29 03:24:48 +00:00
93730771fd
add some verbose messages on finalize routines for DAOS and DFS drivers.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-08-28 17:01:51 +00:00
4df051bf28
New option -Y to invoke the sync command.
2019-08-26 18:57:14 +01:00
a4068be551
Improved help for fsync.
2019-08-26 15:57:13 +01:00
0d9f46e980
MDTest re-added the -Z option for compatibility (for now) and switched back behavior.
2019-08-15 16:49:46 +01:00
de3baf8861
MDTest: Document choice of 42.
2019-08-15 16:21:30 +01:00
b3663bd29a
add sleep depending on MPI rank to avoid all ranks calling daos_fini()
...
at once (issue with PSM2).
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-08-14 13:32:51 +00:00
1320aa279c
add some barriers before cont close and destroy to make sure all
...
ranks are done.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-08-12 16:04:20 +00:00
8cb878507e
Add dfs chunk_size and oclass options.
...
update dfs_remove for API change.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-08-12 14:25:05 +00:00
70e8b13d1d
Fix Last Free
2019-08-04 18:27:20 -05:00
da5f091afc
Merge pull request #169 from ax3l/fix-singleRankHeapBufferOverflow
...
Fix: Heap Buffer Overflow
2019-08-03 09:20:15 +01:00
5e6a03442f
Merge pull request #170 from ax3l/fix-someMemleaks
...
Fix Some Memory Leaks; Thanks.
2019-08-03 09:19:08 +01:00
c4ff3d7c4e
Trivial fix for #168
2019-08-03 09:12:48 +01:00
f16ef9ace5
update dfs_lookup() call for extra parameter.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-08-03 05:07:13 +00:00
b72b51be48
Fix: Heap Buffer Overflow
...
Fix a memory violation when run in serial.
2019-08-02 23:34:13 -05:00
bfff0df8fd
Fix Some Memory Leaks
...
Fixing some memory leaks :)
2019-08-02 23:33:01 -05:00
b1b66962ac
incorrect warning
2019-08-02 13:03:59 +09:00
cf56715a5a
Make sure that each read buffer contains an invalid first byte.
2019-08-01 18:33:44 +01:00
ce1ae750f6
MDtest: Support to verify the read operation with a default pattern.
2019-08-01 18:29:32 +01:00
df8355a9bc
Added output of mdtest stonewall timer.
2019-08-01 17:57:45 +01:00
061b5a860f
Backmerged: New option: print rate AND time; improves debugging.
2019-08-01 17:54:11 +01:00
6c0fadc2a9
Include performance when stonewall is hit to output.
2019-08-01 17:20:01 +01:00
3890b71b54
Fixed issues and followed suggestions from Glenn's review of the PR
2019-08-01 09:42:03 +09:00
92939e4fbd
update for DAOS API changes
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-07-31 17:22:20 +00:00
a3c37808da
Made FAIL take variable args so we can pass printf like args to it
2019-07-28 11:17:11 -06:00
0ffec67d2b
Following Julian's suggestion about better naming
2019-07-28 10:25:42 -06:00
b2d486f749
Followed Andreas suggestion to replace escape double quotes within printf's with single quotes
2019-07-28 10:07:03 -06:00
168a407793
Fixed inconsistent spacing that Andreas commented upon
2019-07-28 09:55:00 -06:00
629ff810b7
Got IOR shifting to work regardless of whether node/task mapping is round-robin or contiguous
2019-07-27 15:27:20 -06:00
d69957e55b
Final changes cleaning up the output messages
2019-07-27 14:31:49 -06:00
e767ef3de9
Remove extraneous print_help function that was causing people to have to edit the same string in two different locations
2019-07-27 13:26:39 -06:00
f6491fcd37
Cleaned up the verbose messages by creating a macro and a function
2019-07-27 13:22:15 -06:00
9d6480d46e
Fixed bug in the nstride calculation where only 0 was computing it correctly. Had to bcast it out
2019-07-27 11:30:07 -06:00
945487f743
Better debug message when stat fails
2019-07-27 09:20:20 -06:00
524d053be1
Making shift work in mdtest as well as it works in IOR and on a per-node basis.
...
Also added printing the nodemap so we can check the allocation.
2019-07-26 08:55:24 -06:00
7981fc70b1
Changed the behavior of shift in mdtest to mimic that in IOR so that it shifts by _entire nodes_ not by ranks.
2019-07-25 19:18:52 -06:00
76f6bb6e42
Abort when invalid options are set.
2019-07-24 00:29:58 +01:00
1c640716c7
Use MPI Info object for all MPI_File_open calls.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-07-08 21:31:00 +00:00
ad61fdd89c
add proper check for DAOS object class and exit if unknown.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-07-08 18:18:24 +00:00
04e3df0d42
use the daos provided function for name to object class
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-07-08 17:02:30 +00:00
52ed766df4
update DAOS and DFS drivers to new DAOS API changes.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-07-08 15:33:06 +00:00
3eb0f86640
Function ncmpi_get_file_info was first introduced in
...
PnetCDF version 1.2.0. Check PnetCDF library version,
so printing MPI-IO hints can be enabled for NCMPI.
2019-06-28 17:40:35 -05:00
9f9730d6fd
should abort on daos init if pool/cont/svcl not specified.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-06-28 16:20:56 +00:00
2ae7c29de2
cache the parent dir in a hashtable so we don't have to do a lookup
...
on every access.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-06-27 21:31:07 +00:00
960005351d
update DAOS and DFS markings.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-06-25 22:41:30 +00:00
262d35d87e
- Merge branch 'master' into daos-devel
...
- fix bugs with cont destroy.
- add destroy option to DFS driver
- share pool and container handle with DFS driver, and allow multi rank access
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Conflicts:
src/ior.c
src/mdtest.c
src/option.c
src/parse_options.c
2019-06-24 21:26:15 +00:00
a822363ee0
style fix
2019-06-22 10:48:41 +09:00
6cae44b512
abstract I/O interface for Gfarm file system
2019-06-21 21:05:41 +09:00
915283bae8
update array_g2l usage with DAOS API change
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-06-17 20:38:02 +00:00
af1aa627eb
use d_ version of function instead of daos_ since some of those are
...
going to be removed from the DAOS API.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-06-07 16:59:11 +00:00
cdaca2c591
Merge pull request #160 from hpc/fix-ime-157
...
Moving IME ODirect option into IME. #157
2019-05-28 08:41:26 +01:00
e4969b5af7
Fix for #159 (also slightly improved parsing for options, for portability between scripts also allow comment at the end of an option).
2019-05-23 12:47:43 +01:00
1b3182904d
Moving IME ODirect option into IME. #157
2019-05-23 09:54:10 +01:00
ed4c1d88d0
Fix missing initialization in MMAP options.
2019-05-23 09:31:05 +01:00
a6f8388f9f
add mknod support
...
Add new option '-k' to support file creation test with mknod,
which is widely used in lustre.
Signed-off-by: Gu Zheng <gzheng@ddn.com>
2019-05-23 13:41:46 +08:00
3d8893e507
Fix #155 . Fixup 3a873ea fixing #150 .
2019-05-22 15:25:27 +02:00
cf197d5e38
Moved broken collectiveMetadata option into HDF5 backend module. #154
2019-05-22 10:21:18 +01:00
c5c43c4f3c
Changed the version name to alpha to differentiate. Now mdtest has the version number from IOR.
2019-05-16 21:25:34 +01:00
3a873ea2b0
Fix #150 .
2019-05-16 19:35:00 +01:00
6abdcc363a
Trivial fix for #149
2019-05-16 12:56:56 +01:00
9ead56c24e
Merge pull request #148 from hpc/fix-146
...
Moving options to submodules + fixes #146 + fixes #144
2019-05-14 14:32:37 -07:00
30a1ca187b
Bugfix by rmn1 for #147 and #112 .
2019-05-14 14:55:11 +01:00
54522315ec
Fix #146 ; handling of hints. Why are GPFS hints special, remains unclear.
2019-05-08 14:02:40 +01:00
a9b0bf35bc
Support MDTest verbosity >=5 prints behavior for all ranks instead of just rank 0 (which is enabled with verbosity level 3 as before)
2019-05-02 09:58:35 +01:00
e3afdb5ed5
beegfs: don't fail if output exists
...
Use existing output file if it has the desired stripe settings
(number of targets, chunk size)
2019-04-29 15:52:26 +02:00
8fd755147a
Fix for #144
2019-04-19 10:20:04 +01:00
5797624d56
fix bug in file delete and access where only rank 0 is calling those functions.
...
remove the bcasts for the error code since the other ranks are not calling it.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-04-15 22:14:33 +00:00
27ac1b88af
Removed verbose output.
2019-04-02 12:34:35 +01:00
07217c93aa
Bugfix for the parsing of flags. Backwards compatibility for direct IO.
2019-04-02 12:08:31 +01:00
e295a4e45f
remove printf of chunk size.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-04-01 22:58:00 +00:00
4a788a0c23
re-write the DAOS IOR driver to use the Array API
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-03-30 19:11:44 +00:00
21405ed924
Updated MMap module to option parser.
2019-03-27 22:32:59 +00:00
b4fb80a69e
Allowed option parser to reset flags (needed in scriptfiles).
2019-03-27 22:32:38 +00:00
07ca5247d7
Checked proper IOR behavior for -f option.
2019-03-27 20:37:46 +00:00
20e960d020
Support the setting of module-specific options per test.
2019-03-27 20:04:48 +00:00
8b23c50389
update dfs_open for API change to expose chunk size selection
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-03-22 16:53:34 +00:00
39eca1bb08
add DAOS_Access routine to check if a container exists before deleting it.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-03-14 21:17:37 +00:00
d49d345007
Merge pull request #137 from DDNStorage/memoryPerNode
...
Fix segfault with option memoryPerNode
2019-03-14 18:33:24 +00:00
fabb1f4d3b
Fix segfault with option memoryPerNode
...
optarg contains a NULL address, which causes a segfault in
NodeMemoryStringToBytes().
2019-03-14 19:20:18 +01:00
c35f64237f
initialize data and akey checksum to NULL since they are not being used.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-03-14 16:55:54 +00:00
db19141c19
remove usage of GURT's D_GOTO from DFS plugin.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-03-01 20:41:01 +00:00
110153cf3a
mdtest: Enable IME native interface
...
This patch makes mdtest allow IME native interface.
It also defines a generic way to enable mdtest with each
backend.
2019-02-12 15:50:51 +01:00
c6abf85378
aiori-IME: Add support of statfs in IME backend
...
This patch adds the support of statfs in the IME backend.
It uses ime_native_statvfs call to fill the ior_aiori_statfs_t
structure.
2019-02-11 15:00:18 +01:00
29090df283
update DAOS and DFS drivers with new DAOS API changes.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-02-08 18:00:33 +00:00
2c87b5e0f5
Merge remote-tracking branch 'upstream/master' into daos
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Conflicts:
src/aiori.c
src/aiori.h
src/ior.c
src/mdtest-main.c
src/mdtest.c
src/option.c
2019-01-24 00:31:12 +00:00
4c1c73ecae
add malloc check to fix
2019-01-23 16:02:58 -08:00
20bb99e1b7
Potential fix for #127 => Segmentation fault due to memory protection.
2019-01-19 17:00:49 +00:00
334bf73797
remove transaction commit and options to do that.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-01-17 20:58:58 +00:00
d2a0023765
remove epoch usage and rebase with latest DAOS API that switches to transactions.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-01-07 18:28:59 +00:00
e71cded6c3
Merge pull request #124 from hpc/fix-hdf5
...
Bugfix for HDF5 plugin; readded collective_md.
2019-01-02 10:01:56 -08:00
1dbca5c293
Bugfix for HDF5 plugin; readded collective_md.
2018-12-23 13:07:23 +00:00
ed421bcc7b
Fixes that issue. Hopefully works for all combinations.
2018-12-20 21:08:55 +00:00
Julian M. Kunkel