Commit Graph

822 Commits (master)

Author SHA1 Message Date
Sylvain Didelot e6bd9ad3b3 common: call mkdir/access syscalls defined in backends
mkdir/access syscalls used in PrependDir() should call their equivalent
implemented by the backend.
2018-10-09 17:55:58 +02:00
Glenn K. Lockwood 87c5c9ef04 compile correctly with -std=c99
This patch enables correct compilation on both MacOS and Linux using only
POSIX.1-2008-compliant C with XSI extensions.

Specifically, POSIX.1-2008 is the minimum version because we use strdup(3);
explicit XSI is required to expose putenv from glibc.
2018-10-08 13:47:28 -07:00
Glenn K. Lockwood bd53ce959d update known issues of 3.2.0 release 2018-10-06 09:31:53 -07:00
Julian M. Kunkel 7045295a86 Replaced usleep to nanosleep. 2018-10-06 17:30:00 +01:00
Glenn K. Lockwood b21ab32ba9 update known issues of 3.2.0 release 2018-10-06 09:27:41 -07:00
Glenn K. Lockwood 9b4c5b93b6 fixes #96
Call MPI_Abort() if MPI is initialized; otherwise just exit()
2018-10-06 09:05:31 -07:00
Glenn K. Lockwood 47b3200521 stop the segfault when an invalid api is specified in configfile 2018-10-06 09:04:43 -07:00
Glenn K. Lockwood 9f56d8b8dd
Merge pull request #89 from DDNStorage/fixwritecheck
Make write verification work even without read test
2018-10-06 09:03:32 -07:00
Julian Kunkel 4f1be703b1
Merge pull request #97 from glennklockwood/issue96
fixes #96
2018-10-06 16:55:05 +01:00
Julian Kunkel c6580dc93b
Merge pull request #99 from glennklockwood/issue87
fixes #87
2018-10-06 16:54:23 +01:00
Julian Kunkel 86726030d3
Merge pull request #100 from glennklockwood/issue95
stop the segfault when an invalid api is specified in configfile
2018-10-06 16:50:55 +01:00
Glenn K. Lockwood a0ef65e61f stop the segfault when an invalid api is specified in configfile 2018-10-05 23:29:21 -07:00
Glenn K. Lockwood d8d4742523 fixes #87
Leading whitespace is stripped from each line of the ior input file.  This
allows indented comments to be treated as comments.  However it does NOT allow
one to specify excessive whitespace inside of the `ior start` and `ior stop`
magic phrases.

Also added a test to catch regressions in this functionality.
2018-10-05 22:50:34 -07:00
Glenn K. Lockwood 3aa3ff10f5 fixes #96
Call MPI_Abort() if MPI is initialized; otherwise just exit()
2018-10-05 22:20:18 -07:00
Julian Kunkel acee84bedc
Merge pull request #88 from roblatham00/hdf5-coll-md
HDF5: option to enable collective metadata
2018-10-02 16:13:05 +01:00
Glenn K. Lockwood d2cc804ba9 fix various issues related to release packages
- travis now tests the packaged source to detect missing source/headers
- basic tests are less sensitive to the directory from where they are run
- fixed some missing files from the `make dist` manifest
- updated the format of NEWS to work with `make dist`
2018-10-01 21:33:31 -07:00
Glenn K. Lockwood 60820da874
Merge pull request #91 from glennklockwood/makedist-fix
fix various issues related to release packages
2018-10-01 21:30:58 -07:00
Julian M. Kunkel 1f1e56c363 Usleep feature added according to Nathan's suggestions. 2018-09-30 10:01:21 +01:00
Julian Kunkel 0a103147ab
Merge pull request #92 from DDNStorage/backend-legacy
Support old legacy name in backends
2018-09-24 11:38:13 +01:00
Jean-Yves VET f472162784 Support old legacy name in backends
Context: Some backends may have used different names in
the past (like IME backend use to be IM). Legacy scripts
may break.

This patch adds a legacy name option in the aiori structure.
Both name and legacy name work to select the interface.
But the following warning is printed if the legacy name is used:

ior WARNING: [legacy name] backend is deprecated use [name] instead.
2018-09-24 11:51:00 +02:00
Glenn K. Lockwood e448ff266d fix various issues related to release packages
- travis now tests the packaged source to detect missing source/headers
- basic tests are less sensitive to the directory from where they are run
- fixed some missing files from the `make dist` manifest
- updated the format of NEWS to work with `make dist`
2018-09-21 18:35:48 -04:00
Julian Kunkel 78fd92267c
Merge pull request #90 from DDNStorage/fixstonewalling
Fix erroneous reported write bandwidth with stonewalling.
2018-09-21 13:20:55 +01:00
Jean-Yves VET 7a7655e959 Fix erroneous write bandwidth with stonewalling
Context: write and read results from the same iteration
use the same length value in Bytes. When stonewalling is
used the size variates depending on the performance of
the access. This leads to wrong max bandwidths reported
for writes as shown in the following example:

    write     10052      ...
    read      9910       ...
    write     10022      ...
    read      9880       ...
    write     10052      ...
    read      9894       ...
    Max Write: 9371.43 MiB/sec (9826.66 MB/sec)
    Max Read:  9910.48 MiB/sec (10391.89 MB/sec)

This patch makes IOR separate variables used for read
and write tests.
2018-09-21 11:43:33 +02:00
Jean-Yves VET 8c727fa99c Simplify and factorize a few functions
This patch factorizes some duplicated code. It also
simplifies the way times are stored to prepare next
patch.
2018-09-21 11:38:44 +02:00
Glenn K. Lockwood 87091dde4c updated metadata for 3.2 2018-09-20 20:38:34 -04:00
Jean-Yves VET f03e7c6778 Make write verification work even without read test
Context: IOR outputs errors when the '-w -W' flags are used
without '-r'. Write a file using with check write option
should be possible even without setting read.

This patch fixes a condition which was introduced for
HDFS to remove RDWR flag in some particular cases.
Write check was set with the write only flag but it
requieres the read flag.
2018-09-20 18:18:48 +02:00
Rob Latham 7d2464f733 HDF5: option to enable collective metadata
In HDF5-1.10 the library has an option to read and write the metadata of a file collectively.  One has to opt-in to this optimization with a property list but particularly at large scale it can have tremendous performance implications.

Further information:
- https://support.hdfgroup.org/HDF5/docNewFeatures/CollMDataIO/RFC-CollectiveMetadataReads.pdf
- https://support.hdfgroup.org/HDF5/doc/RM/RM_H5P.html#Property-SetAllCollMetadataOps
2018-09-20 11:12:08 -05:00
Julian Kunkel 4d82960fe9
Merge pull request #86 from glennklockwood/3.2rc1
3.2rc1 candidate
2018-09-20 09:51:38 +01:00
Jean-Yves VET dface9804d Fix bug in computing max write and read bandwidths
This patch fixes a bug when computing the maximum
bandwidths in short summary output.
2018-09-19 21:01:24 -04:00
Julian M. Kunkel a7b58487b0 Added patch fix O_DIRECT again. 2018-09-19 21:01:16 -04:00
Glenn K. Lockwood ccfa4daa63 updated metadata for 3.2 2018-09-19 20:56:51 -04:00
Julian Kunkel e5c8a11769
Merge pull request #83 from DDNStorage/initbackends
Initialize only the required backends
2018-09-19 10:27:11 +01:00
Julian Kunkel 6881138edb
Merge pull request #84 from DDNStorage/fixshortsummary
Fix bug in computing max write and read bandwidths
2018-09-19 10:25:29 +01:00
Jean-Yves VET 2bcaea564e Initialize only the required backends
Context: IOR initializes all available backends. If one
backend fails to initialize IOR cannot be used.

This patch makes IOR initialize only the backends
which will be used. The initialization is done after
that the parameters are checked so that the help message
can still be dispayed is something goes wrong.
2018-09-18 19:24:29 +02:00
Jean-Yves VET 44e8f03917 Fix bug in computing max write and read bandwidths
This patch fixes a bug when computing the maximum
bandwidths in short summary output.
2018-09-18 19:23:36 +02:00
Julian M. Kunkel fd34b08e05 Added patch fix O_DIRECT again. 2018-09-18 16:48:59 +01:00
Mohamad Chaarawi ab71b4fc2a update DAOS README
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-09-13 13:36:36 +00:00
Mohamad Chaarawi 0a7b1e38ea fix verbose output in DAOS & DFS drivers
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-09-13 13:32:02 +00:00
Mohamad Chaarawi 2f713e947d fix bug in driver name check for DAOS.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-09-12 22:53:42 +00:00
Julian Kunkel 4c1c121fab
Merge pull request #82 from DDNStorage/backend-issue
Check provided IO API is valid
2018-09-11 09:26:21 +01:00
Jean-Yves VET 151a9daa52 Check provided IO API is valid
Context: IOR gets a segfault if an unsupported API string
is provided.

This patch checks that the I/O backend is supported, otherwise
IOR stops with an error.
2018-09-10 19:16:28 +02:00
Julian M. Kunkel 1a4fd7d095 Fix -i option #81 2018-09-05 09:14:22 +01:00
Mohamad Chaarawi 2668363fb1 - update README for DAOS drivers.
- update flag options in DAOS API
- make svcl option in DFS API the same as DAOS API

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-08-30 14:47:17 +00:00
Julian M. Kunkel 2d42c25f69 Fix issue #79 for the last run in a configuration file. 2018-08-30 15:10:38 +01:00
Vishwanath Venkatesan 0adc246adc
Merge pull request #1 from mchaarawi/master
add DFS plugin and update DAOS plugin
2018-08-29 15:32:59 -07:00
Mohamad Chaarawi ced745ae4f Merge branch 'master' of https://github.com/mchaarawi/ior 2018-08-29 22:27:23 +00:00
Mohamad Chaarawi 14d67c19d9 fix DAOS plugin options passing.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-08-29 22:25:48 +00:00
Mohamad Chaarawi 8b48870d38
Merge branch 'daos' into master 2018-08-29 16:24:05 -05:00
Mohamad Chaarawi 46ff4e26be Merge remote branch 'daos-stack/daos'
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>

Conflicts:
	configure.ac
	src/Makefile.am
	src/aiori.c
	src/aiori.h
	src/ior.c
	src/mdtest.c
	src/parse_options.c
2018-08-29 21:14:19 +00:00
Julian Kunkel ba2f0330b7
Merge pull request #80 from roblatham00/more-fixes
We are getting there ^-^
2018-08-29 21:09:31 +01:00