0bffd14de7
Added --warningAsErrors option to IOR and MDTest and refactored WARNINGs in IOR. #174
2020-06-24 11:13:12 +01:00
b4a54c34e8
Merge branch 'master' into fix-modules
...
Conflicts:
src/aiori-DAOS.c
src/aiori-DFS.c
2020-06-11 19:55:23 +00:00
eb2cd27fac
Removed offset from IOR structure.
2020-06-10 17:47:07 +01:00
c828a0f32c
mdtest: use correct number of bytes for posix_memalign of read buffer
...
Fixes a typo in initial posix_memalign commit for DirectIO support.
2020-06-03 01:22:44 +03:00
3eb488939d
mdtest: allocate aligned buffers to support DirectIO
2020-06-03 00:30:38 +03:00
5663593919
Moved generic IOR information to "hint" structure. Backends should only use the "hints" if set.
2020-05-31 12:50:03 +01:00
8098c3740a
Refactored fd into aiori_fd_t for type safety.
2020-05-31 12:12:49 +01:00
294b8891e6
Created dummy option type to provide type safety. Fixed various call issues for options.
2020-05-31 11:50:15 +01:00
e91b79cea0
Moved Lustre/GPFS/BeeGFS options into POSIX backend. Needs testing. Additional minor fixes.
2020-05-30 20:09:37 +01:00
aa9e2ad815
MDTest updated for AIORI changes.
2020-05-30 18:30:26 +01:00
1890aaaeb0
Started to move IOR specific knowledge out of AIORI.
...
For now, and compatibility, IOR options can still be set/internally accessed using the backends init_xfer_options.
This should be removed in the long run to strip away this dependency.
2020-05-30 18:19:48 +01:00
153b7aa60f
Critical bugfix for ior API (used e.g. in IO500)
...
TODO: Check potential issue when using IOR scripts with different backends.
2020-05-28 21:18:44 +01:00
f97090a86b
Typo fix.
2020-05-28 20:11:52 +01:00
4b417ef1f3
IOR shouldn't fail if it cannot determine file system sizes but warn. Functionality should presumably be moved into AIORI backends.
2020-05-28 19:05:41 +01:00
c702a98376
Strict prototypes for AIORI
2019-12-22 11:21:40 +00:00
9da36abaca
Fix confusing output of mknod
...
Context: mknod does not return a file descriptor.
2019-12-16 14:55:26 +01:00
4409082064
Merge pull request #188 from pkoutoupis/feature-global_default_dir_layout
...
Enable global default dir layout for subdirs in Lustre
2019-10-28 10:43:56 -07:00
b3ecd756ce
Implemented AC_CHECK_HEADERS fix in style of #190
2019-09-30 04:25:59 -07:00
604598ab2f
Merge pull request #182 from jschwartz-cray/fix-181
...
Fix #181 .
2019-09-19 16:53:41 +01:00
fe9d76ddf3
fix segfault when no API is specified to mdtest
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-09-09 22:03:55 +00:00
e9dd5fe15e
Merge branch 'master' into daos-devel
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Conflicts:
configure.ac
src/Makefile.am
src/aiori-MPIIO.c
src/aiori.c
src/aiori.h
src/ior.c
src/mdtest.c
src/option.c
2019-09-09 19:55:56 +00:00
0921556456
Merge pull request #184 from hpc/feature-aiori
...
Feature aiori
2019-09-09 15:20:43 +01:00
36d13d3b97
Enable global default dir layout for subdirs in Lustre
2019-09-02 22:26:12 -07:00
ced9b0183f
Merge pull request #172 from hpc/fix-168
...
Fix #168
2019-09-01 15:51:44 +01:00
e3db1759b2
Moded sync() to aiori backend.
2019-09-01 15:47:42 +01:00
6de1ea3176
Merge branch 'feature-sync' into feature-aiori
2019-09-01 15:39:49 +01:00
10d3db1dc8
MDTest: fixing the memset() to account for the number of iterations.
2019-08-31 17:28:08 +01:00
0e952f0f8c
Fix #181 .
...
On systems where numTasks is not evenly divisible by 'tasksPerNode' we were
seeing some nodes reading multiple files while others read none after
reordering.
Commonly all nodes have the same number of tasks but there is nothing
requiring that to be the case. Imagine having 64 tasks running against 4
nodes which can run 20 tasks each. Here you get three groups of 20 and one
group of 4. On this sytem nodes running in the group of 4 were previously
getting tasksPerNode of 4 which meant they reordered tasks differently than
the nodes which got tasksPerNode of 20.
The key to fixing this is ensuring that every node reorders tasks the same
way, which means ensuring they all use the same input values. Obviously on
systems where the number of tasks per node is inconsistent the reordering will
also be inconsistent (some tasks may end up on the same node, or not as far
separated as desired, etc.) but at least this way you'll always end up with a
1:1 reordering.
- Renamed nodes/nodeCount to numNodes
- Renamed tasksPerNode to numTasksOnNode0
- Ensured that numTasksOnNode0 will always have the same value regardless of
which node you're on
- Removed inconsistently used globals numTasksWorld and tasksPerNode and
replaced with per-test params equivalents
- Added utility functions for setting these values:
- numNodes -> GetNumNodes
- numTasks -> GetNumTasks
- numTasksOnNode0 -> GetNumNodesOnTask0
- Improved MPI_VERSION < 3 logic for GetNumNodes so it works when numTasks is
not evenly divisible by numTasksOnNode0
- Left 'nodes' and 'tasksPerNode' in output alone to not break compatibility
- Allowed command-line params to override numTasks, numNodes, and
numTasksOnNode0 but default to using the MPI-calculated values
2019-08-30 16:45:03 -06:00
4df051bf28
New option -Y to invoke the sync command.
2019-08-26 18:57:14 +01:00
0d9f46e980
MDTest re-added the -Z option for compatibility (for now) and switched back behavior.
2019-08-15 16:49:46 +01:00
de3baf8861
MDTest: Document choice of 42.
2019-08-15 16:21:30 +01:00
5e6a03442f
Merge pull request #170 from ax3l/fix-someMemleaks
...
Fix Some Memory Leaks; Thanks.
2019-08-03 09:19:08 +01:00
c4ff3d7c4e
Trivial fix for #168
2019-08-03 09:12:48 +01:00
bfff0df8fd
Fix Some Memory Leaks
...
Fixing some memory leaks :)
2019-08-02 23:33:01 -05:00
cf56715a5a
Make sure that each read buffer contains an invalid first byte.
2019-08-01 18:33:44 +01:00
ce1ae750f6
MDtest: Support to verify the read operation with a default pattern.
2019-08-01 18:29:32 +01:00
df8355a9bc
Added output of mdtest stonewall timer.
2019-08-01 17:57:45 +01:00
061b5a860f
Backmerged: New option: print rate AND time; improves debugging.
2019-08-01 17:54:11 +01:00
3890b71b54
Fixed issues and followed suggestions from Glenn's review of the PR
2019-08-01 09:42:03 +09:00
a3c37808da
Made FAIL take variable args so we can pass printf like args to it
2019-07-28 11:17:11 -06:00
0ffec67d2b
Following Julian's suggestion about better naming
2019-07-28 10:25:42 -06:00
b2d486f749
Followed Andreas suggestion to replace escape double quotes within printf's with single quotes
2019-07-28 10:07:03 -06:00
168a407793
Fixed inconsistent spacing that Andreas commented upon
2019-07-28 09:55:00 -06:00
d69957e55b
Final changes cleaning up the output messages
2019-07-27 14:31:49 -06:00
e767ef3de9
Remove extraneous print_help function that was causing people to have to edit the same string in two different locations
2019-07-27 13:26:39 -06:00
f6491fcd37
Cleaned up the verbose messages by creating a macro and a function
2019-07-27 13:22:15 -06:00
9d6480d46e
Fixed bug in the nstride calculation where only 0 was computing it correctly. Had to bcast it out
2019-07-27 11:30:07 -06:00
945487f743
Better debug message when stat fails
2019-07-27 09:20:20 -06:00
524d053be1
Making shift work in mdtest as well as it works in IOR and on a per-node basis.
...
Also added printing the nodemap so we can check the allocation.
2019-07-26 08:55:24 -06:00
7981fc70b1
Changed the behavior of shift in mdtest to mimic that in IOR so that it shifts by _entire nodes_ not by ranks.
2019-07-25 19:18:52 -06:00
262d35d87e
- Merge branch 'master' into daos-devel
...
- fix bugs with cont destroy.
- add destroy option to DFS driver
- share pool and container handle with DFS driver, and allow multi rank access
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Conflicts:
src/ior.c
src/mdtest.c
src/option.c
src/parse_options.c
2019-06-24 21:26:15 +00:00
a6f8388f9f
add mknod support
...
Add new option '-k' to support file creation test with mknod,
which is widely used in lustre.
Signed-off-by: Gu Zheng <gzheng@ddn.com>
2019-05-23 13:41:46 +08:00
c5c43c4f3c
Changed the version name to alpha to differentiate. Now mdtest has the version number from IOR.
2019-05-16 21:25:34 +01:00
9ead56c24e
Merge pull request #148 from hpc/fix-146
...
Moving options to submodules + fixes #146 + fixes #144
2019-05-14 14:32:37 -07:00
30a1ca187b
Bugfix by rmn1 for #147 and #112 .
2019-05-14 14:55:11 +01:00
a9b0bf35bc
Support MDTest verbosity >=5 prints behavior for all ranks instead of just rank 0 (which is enabled with verbosity level 3 as before)
2019-05-02 09:58:35 +01:00
20e960d020
Support the setting of module-specific options per test.
2019-03-27 20:04:48 +00:00
110153cf3a
mdtest: Enable IME native interface
...
This patch makes mdtest allow IME native interface.
It also defines a generic way to enable mdtest with each
backend.
2019-02-12 15:50:51 +01:00
2c87b5e0f5
Merge remote-tracking branch 'upstream/master' into daos
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Conflicts:
src/aiori.c
src/aiori.h
src/ior.c
src/mdtest-main.c
src/mdtest.c
src/option.c
2019-01-24 00:31:12 +00:00
4c1c73ecae
add malloc check to fix
2019-01-23 16:02:58 -08:00
20bb99e1b7
Potential fix for #127 => Segmentation fault due to memory protection.
2019-01-19 17:00:49 +00:00
ed421bcc7b
Fixes that issue. Hopefully works for all combinations.
2018-12-20 21:08:55 +00:00
7b603bc5a9
Reset stonewall timer to make it work again with running all phases in one execution; i.e. not using multiple runs specifying: -C, -r
2018-12-19 21:37:37 +00:00
4297ccd7f2
Bugfix for stonewall computation.
2018-12-19 20:29:47 +00:00
5e2e1cc96f
Merge pull request #104 from hpc/feature-hdf5-backend
...
Changed the parser to fix #98 . Does _not_ yet have the HDF5 backend args updated. They were reverted to allow this to make the 3.2 release and should be re-applied as a separate commit.
2018-12-04 14:41:31 -06:00
5c847f552a
Merge pull request #110 from hpc/fix-gnu99-101
...
Moved _XOPEN_SOURCE to config.h; resolves #93
2018-12-04 14:13:16 -06:00
8902bc8722
mdtest: fix calculation for barriers
...
Currently, calculation for barriers case looks a bit
suprising, for example, Min stats is from min of
Max value of number of mpi proc, this makes Min
and Max very similar in our testing thus we get
a small Std value too.
I am not a MPI expert, but question is why we
don't use more nature calcuation which calculate
all results from different iterations and MPI proc?
Signed-off-by: Wang Shilong <wshilong@ddn.com>
2018-11-22 19:57:32 +08:00
5a8071fbd8
GNU99 extracted _XOPEN_SOURCE to config.h.
2018-10-28 18:34:22 +00:00
3a4f2ea33e
Merge pull request #101 from glennklockwood/issue93-squashed
...
compile correctly with -std=c99
2018-10-28 18:16:52 +00:00
52182afed6
Merge pull request #108 from hpc/fix-106
...
Fix 106
2018-10-19 11:06:05 +01:00
ea3aeea964
Bugfix initialize the variable to zero for proper API usage.
2018-10-17 16:16:46 +01:00
6b0b1a9830
Fix computation when using branching factor with -n and -i.
2018-10-17 16:04:49 +01:00
093113313d
mdtest: Use GetTimeStamp() instead of MPI_Wtime()
...
IOR uses the GetTimeStamp() wrapper to allow gettimeofday() to be used for timings instead of MPI_Wtime() to calculate throughput. This commits adds the same logic to mdtest timings.
2018-10-17 16:38:35 +02:00
f4afa63ebf
Bugfix #106 mostly.
2018-10-17 13:50:04 +01:00
6f7576aa8a
Changed the parser.
...
The parser now supports concurrent parsing of all plugin options.
Moved HDF5 collective_md option into the backend as an example.
Example: ./src/ior -a dummy --dummy.delay-xfer=50000
2018-10-11 18:31:21 +01:00
87c5c9ef04
compile correctly with -std=c99
...
This patch enables correct compilation on both MacOS and Linux using only
POSIX.1-2008-compliant C with XSI extensions.
Specifically, POSIX.1-2008 is the minimum version because we use strdup(3);
explicit XSI is required to expose putenv from glibc.
2018-10-08 13:47:28 -07:00
f472162784
Support old legacy name in backends
...
Context: Some backends may have used different names in
the past (like IME backend use to be IM). Legacy scripts
may break.
This patch adds a legacy name option in the aiori structure.
Both name and legacy name work to select the interface.
But the following warning is printed if the legacy name is used:
ior WARNING: [legacy name] backend is deprecated use [name] instead.
2018-09-24 11:51:00 +02:00
1a4fd7d095
Fix -i option #81
2018-09-05 09:14:22 +01:00
46ff4e26be
Merge remote branch 'daos-stack/daos'
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Conflicts:
configure.ac
src/Makefile.am
src/aiori.c
src/aiori.h
src/ior.c
src/mdtest.c
src/parse_options.c
2018-08-29 21:14:19 +00:00
5fb850c810
- update the DFS driver to latest DFS API.
...
- update cmd line options to add DAOS Pool and Container uuid and SVCL
- Add init/finalize backend functions.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-08-23 21:58:53 +00:00
4dcf36c1a4
Help text now outputs *compiled* APIs.
2018-08-15 17:07:17 +01:00
0ca5f26e84
cleanup RADOS func names and docs
2018-08-14 16:18:35 -05:00
df34f024ba
Fix warnings reported at compilation time
...
This patch fixes most of the warnings caused
by unused variables and assignments from incompatible
type.
2018-08-09 13:04:52 +02:00
0870ad78b3
Support shared large runs with limited number of files per directory by creating one directory for a bunch of files.
...
Therefore, it uses the -I argument (number of files per directory) and divides it by -n (items).
This will be a number of sub directories that are created on the top-level.
2018-07-26 10:38:06 +01:00
bb6d4fb2a2
Update create for stonewall.
2018-07-15 10:02:45 +01:00
d4ecc76ee8
Stonewall bugfix for delete/stat.
2018-07-14 16:46:24 +01:00
c0657866fd
Minor fixes for stonewalling options.
2018-07-14 12:22:36 +01:00
54e47cf729
IOR: use new option parser.
...
TODO: Parse "-O" options
2018-07-14 08:41:35 +01:00
0835191372
Option parser replaced.
...
Benefits: shows currently set options when used with -h
Supports options for child modules, example for aiori-dummy works, RADOS untested.
2018-07-12 18:09:13 +01:00
fd4c306b98
MDTest stonewalling with status file.
...
Appears to work; stonewall timer is honored only in the creation phase.
This implies a limit to the objects that then are used for other phases.
2018-07-07 23:56:39 +01:00
ed5787811c
Extracted max pathlen variable.
2018-07-07 22:39:14 +01:00
48c17b4d81
Extracted (nearly identical) function DelaySeconds.
2018-07-07 22:26:57 +01:00
6f8d2e8845
Some simplification / unification between IOR and MDTest.
...
MDTest needs more refactoring to be really maintainable.
2018-07-07 22:19:42 +01:00
7bc868d5cf
Dummy AIORI supporting MDtest ops.
2018-07-07 20:01:03 +01:00
812b798f05
IOR allows to store actual performaned stonewalling count into a status file.
2018-07-07 14:01:11 +01:00
e10bebe653
MDTest supports only POSIX at the moment. Added command line check.
2018-07-07 12:14:34 +01:00
a6bfa0f94b
MDTest library option.
...
Moved core functionality / variables used by both implementations to utilities
2018-07-07 10:42:48 +01:00
bcaea2a39f
Replaced getopt with an alternative version (on some machines optind was broken).
2018-07-07 08:41:33 +01:00
8d0cddd21e
Add DFS ior/mdtest driver
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-06-13 18:37:37 +00:00
3ef16c3030
initialize path variable correctly
2018-01-24 19:26:53 +01:00
Julian M. Kunkel