Julian M. Kunkel
ae8a11b42f
MDTest updated stonewall check.
2020-11-27 15:35:32 +00:00
Julian M. Kunkel
7542e75c82
MDTest ignore verbose potential format overflows to be able to spot real errors.
2020-11-27 15:23:32 +00:00
Julian M. Kunkel
ad6dfc5e63
Allow MDTest to print per proc.
2020-11-25 09:50:26 +00:00
Julian M. Kunkel
b5891141d8
Move checks before inititalization.
...
Add simple validation for S3.
2020-11-22 11:43:42 +00:00
Julian M. Kunkel
277f380139
Remove duplicated functionality between mdtest and IOR.
...
Refactored the ShowFileSystemSize function.
2020-11-09 16:23:34 +00:00
Julian M. Kunkel
f275671cc9
AIORI POSIX AIO support. Collect ops until granularity is reached, then submit pending IOs. Synchronize latest on close. Doesn't work with data verification and reuses the existing buffer. The implementation shows the potential AIO may have.
...
Extract also the POSIX header from AIORI to allow better reuse. #240
2020-07-21 16:18:50 +01:00
Julian M. Kunkel
4258e14c11
mdtest: Bugfix of changed behavior, can only remove dir if all children were deleted.
2020-07-21 09:31:19 +01:00
Julian M. Kunkel
8ca388fc78
Merge branch 'feature-cleanup-aiori'
2020-07-21 09:26:03 +01:00
Karsten Weiss
f280123d0b
Spelling fixes (found by codespell)
2020-07-03 09:16:30 +02:00
Julian M. Kunkel
4f7350dc6a
MDTest remove testdir if created by MDTest.
2020-07-02 16:26:05 +01:00
Julian M. Kunkel
fb8f43fcfe
Reduce verbosity for verification
2020-07-01 10:05:09 +01:00
Julian M. Kunkel
5a5b90cdaa
Run existing check to allow only supported backends in mdtest.
2020-06-30 11:17:46 +01:00
Julian M. Kunkel
03dbb20594
MDTest changed verification pattern. Read now always checks the first byte/8 bytes for the signature (item number).
...
Added also --verify-write option which performs a read immediately after a write. Supports #206
2020-06-29 20:15:14 +01:00
Julian M. Kunkel
df3f7082d8
MDTest: Added warnings/errors for errorneous read pattern #206
2020-06-28 17:21:24 +01:00
Julian M. Kunkel
0bffd14de7
Added --warningAsErrors option to IOR and MDTest and refactored WARNINGs in IOR. #174
2020-06-24 11:13:12 +01:00
Mohamad Chaarawi
b4a54c34e8
Merge branch 'master' into fix-modules
...
Conflicts:
src/aiori-DAOS.c
src/aiori-DFS.c
2020-06-11 19:55:23 +00:00
Julian M. Kunkel
eb2cd27fac
Removed offset from IOR structure.
2020-06-10 17:47:07 +01:00
Sven Breuner
c828a0f32c
mdtest: use correct number of bytes for posix_memalign of read buffer
...
Fixes a typo in initial posix_memalign commit for DirectIO support.
2020-06-03 01:22:44 +03:00
Sven Breuner
3eb488939d
mdtest: allocate aligned buffers to support DirectIO
2020-06-03 00:30:38 +03:00
Julian M. Kunkel
5663593919
Moved generic IOR information to "hint" structure. Backends should only use the "hints" if set.
2020-05-31 12:50:03 +01:00
Julian M. Kunkel
8098c3740a
Refactored fd into aiori_fd_t for type safety.
2020-05-31 12:12:49 +01:00
Julian M. Kunkel
294b8891e6
Created dummy option type to provide type safety. Fixed various call issues for options.
2020-05-31 11:50:15 +01:00
Julian M. Kunkel
e91b79cea0
Moved Lustre/GPFS/BeeGFS options into POSIX backend. Needs testing. Additional minor fixes.
2020-05-30 20:09:37 +01:00
Julian M. Kunkel
aa9e2ad815
MDTest updated for AIORI changes.
2020-05-30 18:30:26 +01:00
Julian M. Kunkel
1890aaaeb0
Started to move IOR specific knowledge out of AIORI.
...
For now, and compatibility, IOR options can still be set/internally accessed using the backends init_xfer_options.
This should be removed in the long run to strip away this dependency.
2020-05-30 18:19:48 +01:00
Julian M. Kunkel
153b7aa60f
Critical bugfix for ior API (used e.g. in IO500)
...
TODO: Check potential issue when using IOR scripts with different backends.
2020-05-28 21:18:44 +01:00
Julian M. Kunkel
f97090a86b
Typo fix.
2020-05-28 20:11:52 +01:00
Julian M. Kunkel
4b417ef1f3
IOR shouldn't fail if it cannot determine file system sizes but warn. Functionality should presumably be moved into AIORI backends.
2020-05-28 19:05:41 +01:00
Julian M. Kunkel
c702a98376
Strict prototypes for AIORI
2019-12-22 11:21:40 +00:00
Jean-Yves VET
9da36abaca
Fix confusing output of mknod
...
Context: mknod does not return a file descriptor.
2019-12-16 14:55:26 +01:00
Glenn K. Lockwood
4409082064
Merge pull request #188 from pkoutoupis/feature-global_default_dir_layout
...
Enable global default dir layout for subdirs in Lustre
2019-10-28 10:43:56 -07:00
Petros Koutoupis
b3ecd756ce
Implemented AC_CHECK_HEADERS fix in style of #190
2019-09-30 04:25:59 -07:00
Julian Kunkel
604598ab2f
Merge pull request #182 from jschwartz-cray/fix-181
...
Fix #181 .
2019-09-19 16:53:41 +01:00
Mohamad Chaarawi
fe9d76ddf3
fix segfault when no API is specified to mdtest
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-09-09 22:03:55 +00:00
Mohamad Chaarawi
e9dd5fe15e
Merge branch 'master' into daos-devel
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Conflicts:
configure.ac
src/Makefile.am
src/aiori-MPIIO.c
src/aiori.c
src/aiori.h
src/ior.c
src/mdtest.c
src/option.c
2019-09-09 19:55:56 +00:00
Julian Kunkel
0921556456
Merge pull request #184 from hpc/feature-aiori
...
Feature aiori
2019-09-09 15:20:43 +01:00
Petros Koutoupis
36d13d3b97
Enable global default dir layout for subdirs in Lustre
2019-09-02 22:26:12 -07:00
Julian Kunkel
ced9b0183f
Merge pull request #172 from hpc/fix-168
...
Fix #168
2019-09-01 15:51:44 +01:00
Julian M. Kunkel
e3db1759b2
Moded sync() to aiori backend.
2019-09-01 15:47:42 +01:00
Julian M. Kunkel
6de1ea3176
Merge branch 'feature-sync' into feature-aiori
2019-09-01 15:39:49 +01:00
Julian M. Kunkel
10d3db1dc8
MDTest: fixing the memset() to account for the number of iterations.
2019-08-31 17:28:08 +01:00
Josh Schwartz
0e952f0f8c
Fix #181 .
...
On systems where numTasks is not evenly divisible by 'tasksPerNode' we were
seeing some nodes reading multiple files while others read none after
reordering.
Commonly all nodes have the same number of tasks but there is nothing
requiring that to be the case. Imagine having 64 tasks running against 4
nodes which can run 20 tasks each. Here you get three groups of 20 and one
group of 4. On this sytem nodes running in the group of 4 were previously
getting tasksPerNode of 4 which meant they reordered tasks differently than
the nodes which got tasksPerNode of 20.
The key to fixing this is ensuring that every node reorders tasks the same
way, which means ensuring they all use the same input values. Obviously on
systems where the number of tasks per node is inconsistent the reordering will
also be inconsistent (some tasks may end up on the same node, or not as far
separated as desired, etc.) but at least this way you'll always end up with a
1:1 reordering.
- Renamed nodes/nodeCount to numNodes
- Renamed tasksPerNode to numTasksOnNode0
- Ensured that numTasksOnNode0 will always have the same value regardless of
which node you're on
- Removed inconsistently used globals numTasksWorld and tasksPerNode and
replaced with per-test params equivalents
- Added utility functions for setting these values:
- numNodes -> GetNumNodes
- numTasks -> GetNumTasks
- numTasksOnNode0 -> GetNumNodesOnTask0
- Improved MPI_VERSION < 3 logic for GetNumNodes so it works when numTasks is
not evenly divisible by numTasksOnNode0
- Left 'nodes' and 'tasksPerNode' in output alone to not break compatibility
- Allowed command-line params to override numTasks, numNodes, and
numTasksOnNode0 but default to using the MPI-calculated values
2019-08-30 16:45:03 -06:00
Julian M. Kunkel
4df051bf28
New option -Y to invoke the sync command.
2019-08-26 18:57:14 +01:00
Julian M. Kunkel
0d9f46e980
MDTest re-added the -Z option for compatibility (for now) and switched back behavior.
2019-08-15 16:49:46 +01:00
Julian M. Kunkel
de3baf8861
MDTest: Document choice of 42.
2019-08-15 16:21:30 +01:00
Julian Kunkel
5e6a03442f
Merge pull request #170 from ax3l/fix-someMemleaks
...
Fix Some Memory Leaks; Thanks.
2019-08-03 09:19:08 +01:00
Julian M. Kunkel
c4ff3d7c4e
Trivial fix for #168
2019-08-03 09:12:48 +01:00
Axel Huebl
bfff0df8fd
Fix Some Memory Leaks
...
Fixing some memory leaks :)
2019-08-02 23:33:01 -05:00
Julian M. Kunkel
cf56715a5a
Make sure that each read buffer contains an invalid first byte.
2019-08-01 18:33:44 +01:00
Julian M. Kunkel
ce1ae750f6
MDtest: Support to verify the read operation with a default pattern.
2019-08-01 18:29:32 +01:00