Commit Graph

447 Commits (f280123d0b08c7f54987a33f3209c20d8e288b1f)

Author SHA1 Message Date
Josh Schwartz 0bd979637e Added some extra debug including ERRF, WARNF, and MPI_CHECKF format string macros. 2019-08-30 15:11:25 -06:00
Mohamad Chaarawi 32db1cd902 add timing for container close.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-08-29 03:24:48 +00:00
Mohamad Chaarawi 93730771fd add some verbose messages on finalize routines for DAOS and DFS drivers.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-08-28 17:01:51 +00:00
Julian M. Kunkel 4df051bf28 New option -Y to invoke the sync command. 2019-08-26 18:57:14 +01:00
Julian M. Kunkel a4068be551 Improved help for fsync. 2019-08-26 15:57:13 +01:00
Julian M. Kunkel 0d9f46e980 MDTest re-added the -Z option for compatibility (for now) and switched back behavior. 2019-08-15 16:49:46 +01:00
Julian M. Kunkel de3baf8861 MDTest: Document choice of 42. 2019-08-15 16:21:30 +01:00
Mohamad Chaarawi b3663bd29a add sleep depending on MPI rank to avoid all ranks calling daos_fini()
at once (issue with PSM2).

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-08-14 13:32:51 +00:00
Mohamad Chaarawi 1320aa279c add some barriers before cont close and destroy to make sure all
ranks are done.

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-08-12 16:04:20 +00:00
Mohamad Chaarawi 8cb878507e Add dfs chunk_size and oclass options.
update dfs_remove for API change.

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-08-12 14:25:05 +00:00
Axel Huebl 70e8b13d1d
Fix Last Free 2019-08-04 18:27:20 -05:00
Julian Kunkel da5f091afc
Merge pull request #169 from ax3l/fix-singleRankHeapBufferOverflow
Fix: Heap Buffer Overflow
2019-08-03 09:20:15 +01:00
Julian Kunkel 5e6a03442f
Merge pull request #170 from ax3l/fix-someMemleaks
Fix Some Memory Leaks; Thanks.
2019-08-03 09:19:08 +01:00
Julian M. Kunkel c4ff3d7c4e Trivial fix for #168 2019-08-03 09:12:48 +01:00
Mohamad Chaarawi f16ef9ace5 update dfs_lookup() call for extra parameter.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-08-03 05:07:13 +00:00
Axel Huebl b72b51be48
Fix: Heap Buffer Overflow
Fix a memory violation when run in serial.
2019-08-02 23:34:13 -05:00
Axel Huebl bfff0df8fd
Fix Some Memory Leaks
Fixing some memory leaks :)
2019-08-02 23:33:01 -05:00
Osamu Tatebe b1b66962ac incorrect warning 2019-08-02 13:03:59 +09:00
Julian M. Kunkel cf56715a5a Make sure that each read buffer contains an invalid first byte. 2019-08-01 18:33:44 +01:00
Julian M. Kunkel ce1ae750f6 MDtest: Support to verify the read operation with a default pattern. 2019-08-01 18:29:32 +01:00
Julian M. Kunkel df8355a9bc Added output of mdtest stonewall timer. 2019-08-01 17:57:45 +01:00
Julian M. Kunkel 061b5a860f Backmerged: New option: print rate AND time; improves debugging. 2019-08-01 17:54:11 +01:00
Julian M. Kunkel 6c0fadc2a9 Include performance when stonewall is hit to output. 2019-08-01 17:20:01 +01:00
John Bent 3890b71b54 Fixed issues and followed suggestions from Glenn's review of the PR 2019-08-01 09:42:03 +09:00
Mohamad Chaarawi 92939e4fbd update for DAOS API changes
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-07-31 17:22:20 +00:00
John Bent a3c37808da Made FAIL take variable args so we can pass printf like args to it 2019-07-28 11:17:11 -06:00
John Bent 0ffec67d2b Following Julian's suggestion about better naming 2019-07-28 10:25:42 -06:00
John Bent b2d486f749 Followed Andreas suggestion to replace escape double quotes within printf's with single quotes 2019-07-28 10:07:03 -06:00
John Bent 168a407793 Fixed inconsistent spacing that Andreas commented upon 2019-07-28 09:55:00 -06:00
John Bent 629ff810b7 Got IOR shifting to work regardless of whether node/task mapping is round-robin or contiguous 2019-07-27 15:27:20 -06:00
John Bent d69957e55b Final changes cleaning up the output messages 2019-07-27 14:31:49 -06:00
John Bent e767ef3de9 Remove extraneous print_help function that was causing people to have to edit the same string in two different locations 2019-07-27 13:26:39 -06:00
John Bent f6491fcd37 Cleaned up the verbose messages by creating a macro and a function 2019-07-27 13:22:15 -06:00
John Bent 9d6480d46e Fixed bug in the nstride calculation where only 0 was computing it correctly. Had to bcast it out 2019-07-27 11:30:07 -06:00
John Bent 945487f743 Better debug message when stat fails 2019-07-27 09:20:20 -06:00
John Bent 524d053be1 Making shift work in mdtest as well as it works in IOR and on a per-node basis.
Also added printing the nodemap so we can check the allocation.
2019-07-26 08:55:24 -06:00
John Bent 7981fc70b1 Changed the behavior of shift in mdtest to mimic that in IOR so that it shifts by _entire nodes_ not by ranks. 2019-07-25 19:18:52 -06:00
Julian M. Kunkel 76f6bb6e42 Abort when invalid options are set. 2019-07-24 00:29:58 +01:00
Mohamad Chaarawi 1c640716c7 Use MPI Info object for all MPI_File_open calls.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-07-08 21:31:00 +00:00
Mohamad Chaarawi ad61fdd89c add proper check for DAOS object class and exit if unknown.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-07-08 18:18:24 +00:00
Mohamad Chaarawi 04e3df0d42 use the daos provided function for name to object class
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-07-08 17:02:30 +00:00
Mohamad Chaarawi 52ed766df4 update DAOS and DFS drivers to new DAOS API changes.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-07-08 15:33:06 +00:00
Wei-keng Liao 3eb0f86640 Function ncmpi_get_file_info was first introduced in
PnetCDF version 1.2.0. Check PnetCDF library version,
so printing MPI-IO hints can be enabled for NCMPI.
2019-06-28 17:40:35 -05:00
Mohamad Chaarawi 9f9730d6fd should abort on daos init if pool/cont/svcl not specified.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-06-28 16:20:56 +00:00
Mohamad Chaarawi 2ae7c29de2 cache the parent dir in a hashtable so we don't have to do a lookup
on every access.

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-06-27 21:31:07 +00:00
Mohamad Chaarawi 960005351d update DAOS and DFS markings.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-06-25 22:41:30 +00:00
Mohamad Chaarawi 262d35d87e - Merge branch 'master' into daos-devel
- fix bugs with cont destroy.
- add destroy option to DFS driver
- share pool and container handle with DFS driver, and allow multi rank access

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>

Conflicts:
	src/ior.c
	src/mdtest.c
	src/option.c
	src/parse_options.c
2019-06-24 21:26:15 +00:00
Osamu Tatebe a822363ee0 style fix 2019-06-22 10:48:41 +09:00
Osamu Tatebe 6cae44b512 abstract I/O interface for Gfarm file system 2019-06-21 21:05:41 +09:00
Mohamad Chaarawi 915283bae8 update array_g2l usage with DAOS API change
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-06-17 20:38:02 +00:00
Mohamad Chaarawi af1aa627eb use d_ version of function instead of daos_ since some of those are
going to be removed from the DAOS API.

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-06-07 16:59:11 +00:00
Julian Kunkel cdaca2c591
Merge pull request #160 from hpc/fix-ime-157
Moving IME ODirect option into IME. #157
2019-05-28 08:41:26 +01:00
Julian M. Kunkel e4969b5af7 Fix for #159 (also slightly improved parsing for options, for portability between scripts also allow comment at the end of an option). 2019-05-23 12:47:43 +01:00
Julian M. Kunkel 1b3182904d Moving IME ODirect option into IME. #157 2019-05-23 09:54:10 +01:00
Julian M. Kunkel ed4c1d88d0 Fix missing initialization in MMAP options. 2019-05-23 09:31:05 +01:00
Gu Zheng a6f8388f9f add mknod support
Add new option '-k' to support file creation test with mknod,
which is widely used in lustre.

Signed-off-by: Gu Zheng <gzheng@ddn.com>
2019-05-23 13:41:46 +08:00
Vaclav Hapla 3d8893e507 Fix #155. Fixup 3a873ea fixing #150. 2019-05-22 15:25:27 +02:00
Julian M. Kunkel cf197d5e38 Moved broken collectiveMetadata option into HDF5 backend module. #154 2019-05-22 10:21:18 +01:00
Julian M. Kunkel c5c43c4f3c Changed the version name to alpha to differentiate. Now mdtest has the version number from IOR. 2019-05-16 21:25:34 +01:00
Julian M. Kunkel 3a873ea2b0 Fix #150. 2019-05-16 19:35:00 +01:00
Julian M. Kunkel 6abdcc363a Trivial fix for #149 2019-05-16 12:56:56 +01:00
Glenn K. Lockwood 9ead56c24e
Merge pull request #148 from hpc/fix-146
Moving options to submodules + fixes #146 + fixes #144
2019-05-14 14:32:37 -07:00
Julian M. Kunkel 30a1ca187b Bugfix by rmn1 for #147 and #112. 2019-05-14 14:55:11 +01:00
Julian M. Kunkel 54522315ec Fix #146; handling of hints. Why are GPFS hints special, remains unclear. 2019-05-08 14:02:40 +01:00
Julian M. Kunkel a9b0bf35bc Support MDTest verbosity >=5 prints behavior for all ranks instead of just rank 0 (which is enabled with verbosity level 3 as before) 2019-05-02 09:58:35 +01:00
Oliver Steffen e3afdb5ed5 beegfs: don't fail if output exists
Use existing output file if it has the desired stripe settings
(number of targets, chunk size)
2019-04-29 15:52:26 +02:00
Julian M. Kunkel 8fd755147a Fix for #144 2019-04-19 10:20:04 +01:00
Mohamad Chaarawi 5797624d56 fix bug in file delete and access where only rank 0 is calling those functions.
remove the bcasts for the error code since the other ranks are not calling it.

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-04-15 22:14:33 +00:00
Julian M. Kunkel 27ac1b88af Removed verbose output. 2019-04-02 12:34:35 +01:00
Julian M. Kunkel 07217c93aa Bugfix for the parsing of flags. Backwards compatibility for direct IO. 2019-04-02 12:08:31 +01:00
Mohamad Chaarawi e295a4e45f remove printf of chunk size.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-04-01 22:58:00 +00:00
Mohamad Chaarawi 4a788a0c23 re-write the DAOS IOR driver to use the Array API
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-03-30 19:11:44 +00:00
Julian M. Kunkel 21405ed924 Updated MMap module to option parser. 2019-03-27 22:32:59 +00:00
Julian M. Kunkel b4fb80a69e Allowed option parser to reset flags (needed in scriptfiles). 2019-03-27 22:32:38 +00:00
Julian M. Kunkel 07ca5247d7 Checked proper IOR behavior for -f option. 2019-03-27 20:37:46 +00:00
Julian M. Kunkel 20e960d020 Support the setting of module-specific options per test. 2019-03-27 20:04:48 +00:00
Mohamad Chaarawi 8b23c50389 update dfs_open for API change to expose chunk size selection
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-03-22 16:53:34 +00:00
Mohamad Chaarawi 39eca1bb08 add DAOS_Access routine to check if a container exists before deleting it.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-03-14 21:17:37 +00:00
Julian Kunkel d49d345007
Merge pull request #137 from DDNStorage/memoryPerNode
Fix segfault with option memoryPerNode
2019-03-14 18:33:24 +00:00
Sylvain Didelot fabb1f4d3b Fix segfault with option memoryPerNode
optarg contains a NULL address, which causes a segfault in
NodeMemoryStringToBytes().
2019-03-14 19:20:18 +01:00
Mohamad Chaarawi c35f64237f initialize data and akey checksum to NULL since they are not being used.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-03-14 16:55:54 +00:00
Mohamad Chaarawi db19141c19 remove usage of GURT's D_GOTO from DFS plugin.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-03-01 20:41:01 +00:00
Jean-Yves VET 110153cf3a mdtest: Enable IME native interface
This patch makes mdtest allow IME native interface.
It also defines a generic way to enable mdtest with each
backend.
2019-02-12 15:50:51 +01:00
Jean-Yves VET c6abf85378 aiori-IME: Add support of statfs in IME backend
This patch adds the support of statfs in the IME backend.
It uses ime_native_statvfs call to fill the ior_aiori_statfs_t
structure.
2019-02-11 15:00:18 +01:00
Mohamad Chaarawi 29090df283 update DAOS and DFS drivers with new DAOS API changes.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-02-08 18:00:33 +00:00
Mohamad Chaarawi 2c87b5e0f5 Merge remote-tracking branch 'upstream/master' into daos
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>

Conflicts:
	src/aiori.c
	src/aiori.h
	src/ior.c
	src/mdtest-main.c
	src/mdtest.c
	src/option.c
2019-01-24 00:31:12 +00:00
Glenn K. Lockwood 4c1c73ecae add malloc check to fix 2019-01-23 16:02:58 -08:00
Julian M. Kunkel 20bb99e1b7 Potential fix for #127 => Segmentation fault due to memory protection. 2019-01-19 17:00:49 +00:00
Mohamad Chaarawi 334bf73797 remove transaction commit and options to do that.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-01-17 20:58:58 +00:00
Mohamad Chaarawi d2a0023765 remove epoch usage and rebase with latest DAOS API that switches to transactions.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-01-07 18:28:59 +00:00
Glenn K. Lockwood e71cded6c3
Merge pull request #124 from hpc/fix-hdf5
Bugfix for HDF5 plugin; readded collective_md.
2019-01-02 10:01:56 -08:00
Julian M. Kunkel 1dbca5c293 Bugfix for HDF5 plugin; readded collective_md. 2018-12-23 13:07:23 +00:00
Julian M. Kunkel ed421bcc7b Fixes that issue. Hopefully works for all combinations. 2018-12-20 21:08:55 +00:00
Julian M. Kunkel 7b603bc5a9 Reset stonewall timer to make it work again with running all phases in one execution; i.e. not using multiple runs specifying: -C, -r 2018-12-19 21:37:37 +00:00
Julian M. Kunkel 4297ccd7f2 Bugfix for stonewall computation. 2018-12-19 20:29:47 +00:00
Andreas Dilger 1f217ae48c lustre: fix checks for lustre_user.h header
Fix the configure check if --with-lustre is specified, but
the linux/lustre/lustre_user.h header is not present.  Only
one of the headers needs to be included if both are found.

In some cases, FASYNC is not defined, but forms part of the
O_LOV_DELAY_CREATE value, add a #define in that case.

Fixes #115.

Fixes: cb88c4c19a831d94b864c49a162e2635730540e5
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
2018-12-14 16:45:43 -07:00
Mohamad Chaarawi 72c487013a rebase with latest DAOS
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-12-04 22:59:15 +00:00
Glenn K. Lockwood e1158daaf2
Merge pull request #105 from hpc/feature-test-scripts-85
Dry-run option to test the configuration / scripts.
2018-12-04 15:06:47 -06:00
Glenn K. Lockwood 5e2e1cc96f
Merge pull request #104 from hpc/feature-hdf5-backend
Changed the parser to fix #98.  Does _not_ yet have the HDF5 backend args updated.  They were reverted to allow this to make the 3.2 release and should be re-applied as a separate commit.
2018-12-04 14:41:31 -06:00
Glenn K. Lockwood 5c847f552a
Merge pull request #110 from hpc/fix-gnu99-101
Moved _XOPEN_SOURCE to config.h; resolves #93
2018-12-04 14:13:16 -06:00
Julian M. Kunkel 7db8ffd8ab Reverted the change to HDF5 module back. 2018-12-04 19:16:41 +00:00
Wang Shilong 8902bc8722 mdtest: fix calculation for barriers
Currently, calculation for barriers case looks a bit
suprising, for example, Min stats is from min of
Max value of number of mpi proc, this makes Min
and Max very similar in our testing thus we get
a small Std value too.

I am not a MPI expert, but question is why we
don't use more nature calcuation which calculate
all results from different iterations and MPI proc?

Signed-off-by: Wang Shilong <wshilong@ddn.com>
2018-11-22 19:57:32 +08:00
Julian M. Kunkel 5a8071fbd8 GNU99 extracted _XOPEN_SOURCE to config.h. 2018-10-28 18:34:22 +00:00
Julian Kunkel 3a4f2ea33e
Merge pull request #101 from glennklockwood/issue93-squashed
compile correctly with -std=c99
2018-10-28 18:16:52 +00:00
Julian Kunkel 52182afed6
Merge pull request #108 from hpc/fix-106
Fix 106
2018-10-19 11:06:05 +01:00
Julian M. Kunkel ea3aeea964 Bugfix initialize the variable to zero for proper API usage. 2018-10-17 16:16:46 +01:00
Julian M. Kunkel 6b0b1a9830 Fix computation when using branching factor with -n and -i. 2018-10-17 16:04:49 +01:00
Marc Vef 093113313d mdtest: Use GetTimeStamp() instead of MPI_Wtime()
IOR uses the GetTimeStamp() wrapper to allow gettimeofday() to be used for timings instead of MPI_Wtime() to calculate throughput. This commits adds the same logic to mdtest timings.
2018-10-17 16:38:35 +02:00
Julian M. Kunkel f4afa63ebf Bugfix #106 mostly. 2018-10-17 13:50:04 +01:00
Julian Kunkel f7dc084f32
Merge pull request #103 from DDNStorage/ime_mkdir_rmdir
IME: add support of mkdir/rmdir into the IME backend
2018-10-16 14:34:26 +01:00
Julian M. Kunkel 6c5952906b Dry-run option to test the configuration / scripts. 2018-10-11 19:58:30 +01:00
Julian M. Kunkel 3b5f4fc002 Bugfix JSON output for multiple processes 2018-10-11 18:59:12 +01:00
Julian M. Kunkel fa38cb7992 Bugfix JSON header (allows parsing at line 0). 2018-10-11 18:49:25 +01:00
Julian M. Kunkel 6f7576aa8a Changed the parser.
The parser now supports concurrent parsing of all plugin options.
Moved HDF5 collective_md option into the backend as an example.
Example: ./src/ior -a dummy --dummy.delay-xfer=50000
2018-10-11 18:31:21 +01:00
Sylvain Didelot 912d93e94f aiori-IME: add support of mkdir/rmdir to the backup
ime_native_mkdir() and ime_native_rmdir() are available starting
from IME native API version 1.3.
2018-10-09 17:56:08 +02:00
Sylvain Didelot e6bd9ad3b3 common: call mkdir/access syscalls defined in backends
mkdir/access syscalls used in PrependDir() should call their equivalent
implemented by the backend.
2018-10-09 17:55:58 +02:00
Glenn K. Lockwood 87c5c9ef04 compile correctly with -std=c99
This patch enables correct compilation on both MacOS and Linux using only
POSIX.1-2008-compliant C with XSI extensions.

Specifically, POSIX.1-2008 is the minimum version because we use strdup(3);
explicit XSI is required to expose putenv from glibc.
2018-10-08 13:47:28 -07:00
Julian M. Kunkel 7045295a86 Replaced usleep to nanosleep. 2018-10-06 17:30:00 +01:00
Glenn K. Lockwood 9f56d8b8dd
Merge pull request #89 from DDNStorage/fixwritecheck
Make write verification work even without read test
2018-10-06 09:03:32 -07:00
Julian Kunkel 4f1be703b1
Merge pull request #97 from glennklockwood/issue96
fixes #96
2018-10-06 16:55:05 +01:00
Julian Kunkel c6580dc93b
Merge pull request #99 from glennklockwood/issue87
fixes #87
2018-10-06 16:54:23 +01:00
Glenn K. Lockwood a0ef65e61f stop the segfault when an invalid api is specified in configfile 2018-10-05 23:29:21 -07:00
Glenn K. Lockwood d8d4742523 fixes #87
Leading whitespace is stripped from each line of the ior input file.  This
allows indented comments to be treated as comments.  However it does NOT allow
one to specify excessive whitespace inside of the `ior start` and `ior stop`
magic phrases.

Also added a test to catch regressions in this functionality.
2018-10-05 22:50:34 -07:00
Glenn K. Lockwood 3aa3ff10f5 fixes #96
Call MPI_Abort() if MPI is initialized; otherwise just exit()
2018-10-05 22:20:18 -07:00
Julian Kunkel acee84bedc
Merge pull request #88 from roblatham00/hdf5-coll-md
HDF5: option to enable collective metadata
2018-10-02 16:13:05 +01:00
Glenn K. Lockwood 60820da874
Merge pull request #91 from glennklockwood/makedist-fix
fix various issues related to release packages
2018-10-01 21:30:58 -07:00
Julian M. Kunkel 1f1e56c363 Usleep feature added according to Nathan's suggestions. 2018-09-30 10:01:21 +01:00
Julian Kunkel 0a103147ab
Merge pull request #92 from DDNStorage/backend-legacy
Support old legacy name in backends
2018-09-24 11:38:13 +01:00
Jean-Yves VET f472162784 Support old legacy name in backends
Context: Some backends may have used different names in
the past (like IME backend use to be IM). Legacy scripts
may break.

This patch adds a legacy name option in the aiori structure.
Both name and legacy name work to select the interface.
But the following warning is printed if the legacy name is used:

ior WARNING: [legacy name] backend is deprecated use [name] instead.
2018-09-24 11:51:00 +02:00
Glenn K. Lockwood e448ff266d fix various issues related to release packages
- travis now tests the packaged source to detect missing source/headers
- basic tests are less sensitive to the directory from where they are run
- fixed some missing files from the `make dist` manifest
- updated the format of NEWS to work with `make dist`
2018-09-21 18:35:48 -04:00
Jean-Yves VET 7a7655e959 Fix erroneous write bandwidth with stonewalling
Context: write and read results from the same iteration
use the same length value in Bytes. When stonewalling is
used the size variates depending on the performance of
the access. This leads to wrong max bandwidths reported
for writes as shown in the following example:

    write     10052      ...
    read      9910       ...
    write     10022      ...
    read      9880       ...
    write     10052      ...
    read      9894       ...
    Max Write: 9371.43 MiB/sec (9826.66 MB/sec)
    Max Read:  9910.48 MiB/sec (10391.89 MB/sec)

This patch makes IOR separate variables used for read
and write tests.
2018-09-21 11:43:33 +02:00
Jean-Yves VET 8c727fa99c Simplify and factorize a few functions
This patch factorizes some duplicated code. It also
simplifies the way times are stored to prepare next
patch.
2018-09-21 11:38:44 +02:00
Jean-Yves VET f03e7c6778 Make write verification work even without read test
Context: IOR outputs errors when the '-w -W' flags are used
without '-r'. Write a file using with check write option
should be possible even without setting read.

This patch fixes a condition which was introduced for
HDFS to remove RDWR flag in some particular cases.
Write check was set with the write only flag but it
requieres the read flag.
2018-09-20 18:18:48 +02:00
Rob Latham 7d2464f733 HDF5: option to enable collective metadata
In HDF5-1.10 the library has an option to read and write the metadata of a file collectively.  One has to opt-in to this optimization with a property list but particularly at large scale it can have tremendous performance implications.

Further information:
- https://support.hdfgroup.org/HDF5/docNewFeatures/CollMDataIO/RFC-CollectiveMetadataReads.pdf
- https://support.hdfgroup.org/HDF5/doc/RM/RM_H5P.html#Property-SetAllCollMetadataOps
2018-09-20 11:12:08 -05:00
Julian Kunkel e5c8a11769
Merge pull request #83 from DDNStorage/initbackends
Initialize only the required backends
2018-09-19 10:27:11 +01:00
Jean-Yves VET 2bcaea564e Initialize only the required backends
Context: IOR initializes all available backends. If one
backend fails to initialize IOR cannot be used.

This patch makes IOR initialize only the backends
which will be used. The initialization is done after
that the parameters are checked so that the help message
can still be dispayed is something goes wrong.
2018-09-18 19:24:29 +02:00
Jean-Yves VET 44e8f03917 Fix bug in computing max write and read bandwidths
This patch fixes a bug when computing the maximum
bandwidths in short summary output.
2018-09-18 19:23:36 +02:00
Julian M. Kunkel fd34b08e05 Added patch fix O_DIRECT again. 2018-09-18 16:48:59 +01:00
Mohamad Chaarawi 0a7b1e38ea fix verbose output in DAOS & DFS drivers
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-09-13 13:32:02 +00:00
Mohamad Chaarawi 2f713e947d fix bug in driver name check for DAOS.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-09-12 22:53:42 +00:00
Jean-Yves VET 151a9daa52 Check provided IO API is valid
Context: IOR gets a segfault if an unsupported API string
is provided.

This patch checks that the I/O backend is supported, otherwise
IOR stops with an error.
2018-09-10 19:16:28 +02:00
Julian M. Kunkel 1a4fd7d095 Fix -i option #81 2018-09-05 09:14:22 +01:00
Mohamad Chaarawi 2668363fb1 - update README for DAOS drivers.
- update flag options in DAOS API
- make svcl option in DFS API the same as DAOS API

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-08-30 14:47:17 +00:00
Julian M. Kunkel 2d42c25f69 Fix issue #79 for the last run in a configuration file. 2018-08-30 15:10:38 +01:00
Mohamad Chaarawi 14d67c19d9 fix DAOS plugin options passing.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-08-29 22:25:48 +00:00
Mohamad Chaarawi 46ff4e26be Merge remote branch 'daos-stack/daos'
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>

Conflicts:
	configure.ac
	src/Makefile.am
	src/aiori.c
	src/aiori.h
	src/ior.c
	src/mdtest.c
	src/parse_options.c
2018-08-29 21:14:19 +00:00
Julian Kunkel ba2f0330b7
Merge pull request #80 from roblatham00/more-fixes
We are getting there ^-^
2018-08-29 21:09:31 +01:00
Julian M. Kunkel c1f3f485ec Bugfix #79 2018-08-29 21:07:34 +01:00
Rob Latham cb88c4c19a including lustre_user now deprecated
put in a configure test (same one used in ROMIO) in case someone's using
old Lustre client headers
2018-08-28 16:27:21 -05:00
Rob Latham 4a32a2cbb3 fix "number of arguments doesn’t match prototype" error 2018-08-28 16:24:34 -05:00