Mohamad Chaarawi
af1aa627eb
use d_ version of function instead of daos_ since some of those are
...
going to be removed from the DAOS API.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-06-07 16:59:11 +00:00
Julian Kunkel
cdaca2c591
Merge pull request #160 from hpc/fix-ime-157
...
Moving IME ODirect option into IME. #157
2019-05-28 08:41:26 +01:00
Julian M. Kunkel
e4969b5af7
Fix for #159 (also slightly improved parsing for options, for portability between scripts also allow comment at the end of an option).
2019-05-23 12:47:43 +01:00
Julian M. Kunkel
1b3182904d
Moving IME ODirect option into IME. #157
2019-05-23 09:54:10 +01:00
Julian M. Kunkel
ed4c1d88d0
Fix missing initialization in MMAP options.
2019-05-23 09:31:05 +01:00
Gu Zheng
a6f8388f9f
add mknod support
...
Add new option '-k' to support file creation test with mknod,
which is widely used in lustre.
Signed-off-by: Gu Zheng <gzheng@ddn.com>
2019-05-23 13:41:46 +08:00
Vaclav Hapla
3d8893e507
Fix #155 . Fixup 3a873ea
fixing #150 .
2019-05-22 15:25:27 +02:00
Julian M. Kunkel
cf197d5e38
Moved broken collectiveMetadata option into HDF5 backend module. #154
2019-05-22 10:21:18 +01:00
Julian M. Kunkel
c5c43c4f3c
Changed the version name to alpha to differentiate. Now mdtest has the version number from IOR.
2019-05-16 21:25:34 +01:00
Julian M. Kunkel
3a873ea2b0
Fix #150 .
2019-05-16 19:35:00 +01:00
Julian M. Kunkel
6abdcc363a
Trivial fix for #149
2019-05-16 12:56:56 +01:00
Glenn K. Lockwood
9ead56c24e
Merge pull request #148 from hpc/fix-146
...
Moving options to submodules + fixes #146 + fixes #144
2019-05-14 14:32:37 -07:00
Julian M. Kunkel
30a1ca187b
Bugfix by rmn1 for #147 and #112 .
2019-05-14 14:55:11 +01:00
Julian M. Kunkel
54522315ec
Fix #146 ; handling of hints. Why are GPFS hints special, remains unclear.
2019-05-08 14:02:40 +01:00
Julian M. Kunkel
a9b0bf35bc
Support MDTest verbosity >=5 prints behavior for all ranks instead of just rank 0 (which is enabled with verbosity level 3 as before)
2019-05-02 09:58:35 +01:00
Oliver Steffen
e3afdb5ed5
beegfs: don't fail if output exists
...
Use existing output file if it has the desired stripe settings
(number of targets, chunk size)
2019-04-29 15:52:26 +02:00
Julian M. Kunkel
8fd755147a
Fix for #144
2019-04-19 10:20:04 +01:00
Mohamad Chaarawi
5797624d56
fix bug in file delete and access where only rank 0 is calling those functions.
...
remove the bcasts for the error code since the other ranks are not calling it.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-04-15 22:14:33 +00:00
Julian M. Kunkel
27ac1b88af
Removed verbose output.
2019-04-02 12:34:35 +01:00
Julian M. Kunkel
07217c93aa
Bugfix for the parsing of flags. Backwards compatibility for direct IO.
2019-04-02 12:08:31 +01:00
Mohamad Chaarawi
e295a4e45f
remove printf of chunk size.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-04-01 22:58:00 +00:00
Mohamad Chaarawi
4a788a0c23
re-write the DAOS IOR driver to use the Array API
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-03-30 19:11:44 +00:00
Julian M. Kunkel
21405ed924
Updated MMap module to option parser.
2019-03-27 22:32:59 +00:00
Julian M. Kunkel
b4fb80a69e
Allowed option parser to reset flags (needed in scriptfiles).
2019-03-27 22:32:38 +00:00
Julian M. Kunkel
07ca5247d7
Checked proper IOR behavior for -f option.
2019-03-27 20:37:46 +00:00
Julian M. Kunkel
20e960d020
Support the setting of module-specific options per test.
2019-03-27 20:04:48 +00:00
Mohamad Chaarawi
8b23c50389
update dfs_open for API change to expose chunk size selection
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-03-22 16:53:34 +00:00
Mohamad Chaarawi
39eca1bb08
add DAOS_Access routine to check if a container exists before deleting it.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-03-14 21:17:37 +00:00
Julian Kunkel
d49d345007
Merge pull request #137 from DDNStorage/memoryPerNode
...
Fix segfault with option memoryPerNode
2019-03-14 18:33:24 +00:00
Sylvain Didelot
fabb1f4d3b
Fix segfault with option memoryPerNode
...
optarg contains a NULL address, which causes a segfault in
NodeMemoryStringToBytes().
2019-03-14 19:20:18 +01:00
Mohamad Chaarawi
c35f64237f
initialize data and akey checksum to NULL since they are not being used.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-03-14 16:55:54 +00:00
Mohamad Chaarawi
db19141c19
remove usage of GURT's D_GOTO from DFS plugin.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-03-01 20:41:01 +00:00
Jean-Yves VET
110153cf3a
mdtest: Enable IME native interface
...
This patch makes mdtest allow IME native interface.
It also defines a generic way to enable mdtest with each
backend.
2019-02-12 15:50:51 +01:00
Jean-Yves VET
c6abf85378
aiori-IME: Add support of statfs in IME backend
...
This patch adds the support of statfs in the IME backend.
It uses ime_native_statvfs call to fill the ior_aiori_statfs_t
structure.
2019-02-11 15:00:18 +01:00
Mohamad Chaarawi
29090df283
update DAOS and DFS drivers with new DAOS API changes.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-02-08 18:00:33 +00:00
Mohamad Chaarawi
2c87b5e0f5
Merge remote-tracking branch 'upstream/master' into daos
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Conflicts:
src/aiori.c
src/aiori.h
src/ior.c
src/mdtest-main.c
src/mdtest.c
src/option.c
2019-01-24 00:31:12 +00:00
Glenn K. Lockwood
4c1c73ecae
add malloc check to fix
2019-01-23 16:02:58 -08:00
Julian M. Kunkel
20bb99e1b7
Potential fix for #127 => Segmentation fault due to memory protection.
2019-01-19 17:00:49 +00:00
Mohamad Chaarawi
334bf73797
remove transaction commit and options to do that.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-01-17 20:58:58 +00:00
Mohamad Chaarawi
d2a0023765
remove epoch usage and rebase with latest DAOS API that switches to transactions.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-01-07 18:28:59 +00:00
Glenn K. Lockwood
e71cded6c3
Merge pull request #124 from hpc/fix-hdf5
...
Bugfix for HDF5 plugin; readded collective_md.
2019-01-02 10:01:56 -08:00
Julian M. Kunkel
1dbca5c293
Bugfix for HDF5 plugin; readded collective_md.
2018-12-23 13:07:23 +00:00
Julian M. Kunkel
ed421bcc7b
Fixes that issue. Hopefully works for all combinations.
2018-12-20 21:08:55 +00:00
Julian M. Kunkel
7b603bc5a9
Reset stonewall timer to make it work again with running all phases in one execution; i.e. not using multiple runs specifying: -C, -r
2018-12-19 21:37:37 +00:00
Julian M. Kunkel
4297ccd7f2
Bugfix for stonewall computation.
2018-12-19 20:29:47 +00:00
Andreas Dilger
1f217ae48c
lustre: fix checks for lustre_user.h header
...
Fix the configure check if --with-lustre is specified, but
the linux/lustre/lustre_user.h header is not present. Only
one of the headers needs to be included if both are found.
In some cases, FASYNC is not defined, but forms part of the
O_LOV_DELAY_CREATE value, add a #define in that case.
Fixes #115 .
Fixes: cb88c4c19a831d94b864c49a162e2635730540e5
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
2018-12-14 16:45:43 -07:00
Mohamad Chaarawi
72c487013a
rebase with latest DAOS
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-12-04 22:59:15 +00:00
Glenn K. Lockwood
e1158daaf2
Merge pull request #105 from hpc/feature-test-scripts-85
...
Dry-run option to test the configuration / scripts.
2018-12-04 15:06:47 -06:00
Glenn K. Lockwood
5e2e1cc96f
Merge pull request #104 from hpc/feature-hdf5-backend
...
Changed the parser to fix #98 . Does _not_ yet have the HDF5 backend args updated. They were reverted to allow this to make the 3.2 release and should be re-applied as a separate commit.
2018-12-04 14:41:31 -06:00
Glenn K. Lockwood
5c847f552a
Merge pull request #110 from hpc/fix-gnu99-101
...
Moved _XOPEN_SOURCE to config.h; resolves #93
2018-12-04 14:13:16 -06:00
Julian M. Kunkel
7db8ffd8ab
Reverted the change to HDF5 module back.
2018-12-04 19:16:41 +00:00
Wang Shilong
8902bc8722
mdtest: fix calculation for barriers
...
Currently, calculation for barriers case looks a bit
suprising, for example, Min stats is from min of
Max value of number of mpi proc, this makes Min
and Max very similar in our testing thus we get
a small Std value too.
I am not a MPI expert, but question is why we
don't use more nature calcuation which calculate
all results from different iterations and MPI proc?
Signed-off-by: Wang Shilong <wshilong@ddn.com>
2018-11-22 19:57:32 +08:00
Julian M. Kunkel
5a8071fbd8
GNU99 extracted _XOPEN_SOURCE to config.h.
2018-10-28 18:34:22 +00:00
Julian Kunkel
3a4f2ea33e
Merge pull request #101 from glennklockwood/issue93-squashed
...
compile correctly with -std=c99
2018-10-28 18:16:52 +00:00
Julian Kunkel
52182afed6
Merge pull request #108 from hpc/fix-106
...
Fix 106
2018-10-19 11:06:05 +01:00
Julian M. Kunkel
ea3aeea964
Bugfix initialize the variable to zero for proper API usage.
2018-10-17 16:16:46 +01:00
Julian M. Kunkel
6b0b1a9830
Fix computation when using branching factor with -n and -i.
2018-10-17 16:04:49 +01:00
Marc Vef
093113313d
mdtest: Use GetTimeStamp() instead of MPI_Wtime()
...
IOR uses the GetTimeStamp() wrapper to allow gettimeofday() to be used for timings instead of MPI_Wtime() to calculate throughput. This commits adds the same logic to mdtest timings.
2018-10-17 16:38:35 +02:00
Julian M. Kunkel
f4afa63ebf
Bugfix #106 mostly.
2018-10-17 13:50:04 +01:00
Julian Kunkel
f7dc084f32
Merge pull request #103 from DDNStorage/ime_mkdir_rmdir
...
IME: add support of mkdir/rmdir into the IME backend
2018-10-16 14:34:26 +01:00
Julian M. Kunkel
6c5952906b
Dry-run option to test the configuration / scripts.
2018-10-11 19:58:30 +01:00
Julian M. Kunkel
3b5f4fc002
Bugfix JSON output for multiple processes
2018-10-11 18:59:12 +01:00
Julian M. Kunkel
fa38cb7992
Bugfix JSON header (allows parsing at line 0).
2018-10-11 18:49:25 +01:00
Julian M. Kunkel
6f7576aa8a
Changed the parser.
...
The parser now supports concurrent parsing of all plugin options.
Moved HDF5 collective_md option into the backend as an example.
Example: ./src/ior -a dummy --dummy.delay-xfer=50000
2018-10-11 18:31:21 +01:00
Sylvain Didelot
912d93e94f
aiori-IME: add support of mkdir/rmdir to the backup
...
ime_native_mkdir() and ime_native_rmdir() are available starting
from IME native API version 1.3.
2018-10-09 17:56:08 +02:00
Sylvain Didelot
e6bd9ad3b3
common: call mkdir/access syscalls defined in backends
...
mkdir/access syscalls used in PrependDir() should call their equivalent
implemented by the backend.
2018-10-09 17:55:58 +02:00
Glenn K. Lockwood
87c5c9ef04
compile correctly with -std=c99
...
This patch enables correct compilation on both MacOS and Linux using only
POSIX.1-2008-compliant C with XSI extensions.
Specifically, POSIX.1-2008 is the minimum version because we use strdup(3);
explicit XSI is required to expose putenv from glibc.
2018-10-08 13:47:28 -07:00
Julian M. Kunkel
7045295a86
Replaced usleep to nanosleep.
2018-10-06 17:30:00 +01:00
Glenn K. Lockwood
9f56d8b8dd
Merge pull request #89 from DDNStorage/fixwritecheck
...
Make write verification work even without read test
2018-10-06 09:03:32 -07:00
Julian Kunkel
4f1be703b1
Merge pull request #97 from glennklockwood/issue96
...
fixes #96
2018-10-06 16:55:05 +01:00
Julian Kunkel
c6580dc93b
Merge pull request #99 from glennklockwood/issue87
...
fixes #87
2018-10-06 16:54:23 +01:00
Glenn K. Lockwood
a0ef65e61f
stop the segfault when an invalid api is specified in configfile
2018-10-05 23:29:21 -07:00
Glenn K. Lockwood
d8d4742523
fixes #87
...
Leading whitespace is stripped from each line of the ior input file. This
allows indented comments to be treated as comments. However it does NOT allow
one to specify excessive whitespace inside of the `ior start` and `ior stop`
magic phrases.
Also added a test to catch regressions in this functionality.
2018-10-05 22:50:34 -07:00
Glenn K. Lockwood
3aa3ff10f5
fixes #96
...
Call MPI_Abort() if MPI is initialized; otherwise just exit()
2018-10-05 22:20:18 -07:00
Julian Kunkel
acee84bedc
Merge pull request #88 from roblatham00/hdf5-coll-md
...
HDF5: option to enable collective metadata
2018-10-02 16:13:05 +01:00
Glenn K. Lockwood
60820da874
Merge pull request #91 from glennklockwood/makedist-fix
...
fix various issues related to release packages
2018-10-01 21:30:58 -07:00
Julian M. Kunkel
1f1e56c363
Usleep feature added according to Nathan's suggestions.
2018-09-30 10:01:21 +01:00
Julian Kunkel
0a103147ab
Merge pull request #92 from DDNStorage/backend-legacy
...
Support old legacy name in backends
2018-09-24 11:38:13 +01:00
Jean-Yves VET
f472162784
Support old legacy name in backends
...
Context: Some backends may have used different names in
the past (like IME backend use to be IM). Legacy scripts
may break.
This patch adds a legacy name option in the aiori structure.
Both name and legacy name work to select the interface.
But the following warning is printed if the legacy name is used:
ior WARNING: [legacy name] backend is deprecated use [name] instead.
2018-09-24 11:51:00 +02:00
Glenn K. Lockwood
e448ff266d
fix various issues related to release packages
...
- travis now tests the packaged source to detect missing source/headers
- basic tests are less sensitive to the directory from where they are run
- fixed some missing files from the `make dist` manifest
- updated the format of NEWS to work with `make dist`
2018-09-21 18:35:48 -04:00
Jean-Yves VET
7a7655e959
Fix erroneous write bandwidth with stonewalling
...
Context: write and read results from the same iteration
use the same length value in Bytes. When stonewalling is
used the size variates depending on the performance of
the access. This leads to wrong max bandwidths reported
for writes as shown in the following example:
write 10052 ...
read 9910 ...
write 10022 ...
read 9880 ...
write 10052 ...
read 9894 ...
Max Write: 9371.43 MiB/sec (9826.66 MB/sec)
Max Read: 9910.48 MiB/sec (10391.89 MB/sec)
This patch makes IOR separate variables used for read
and write tests.
2018-09-21 11:43:33 +02:00
Jean-Yves VET
8c727fa99c
Simplify and factorize a few functions
...
This patch factorizes some duplicated code. It also
simplifies the way times are stored to prepare next
patch.
2018-09-21 11:38:44 +02:00
Jean-Yves VET
f03e7c6778
Make write verification work even without read test
...
Context: IOR outputs errors when the '-w -W' flags are used
without '-r'. Write a file using with check write option
should be possible even without setting read.
This patch fixes a condition which was introduced for
HDFS to remove RDWR flag in some particular cases.
Write check was set with the write only flag but it
requieres the read flag.
2018-09-20 18:18:48 +02:00
Rob Latham
7d2464f733
HDF5: option to enable collective metadata
...
In HDF5-1.10 the library has an option to read and write the metadata of a file collectively. One has to opt-in to this optimization with a property list but particularly at large scale it can have tremendous performance implications.
Further information:
- https://support.hdfgroup.org/HDF5/docNewFeatures/CollMDataIO/RFC-CollectiveMetadataReads.pdf
- https://support.hdfgroup.org/HDF5/doc/RM/RM_H5P.html#Property-SetAllCollMetadataOps
2018-09-20 11:12:08 -05:00
Julian Kunkel
e5c8a11769
Merge pull request #83 from DDNStorage/initbackends
...
Initialize only the required backends
2018-09-19 10:27:11 +01:00
Jean-Yves VET
2bcaea564e
Initialize only the required backends
...
Context: IOR initializes all available backends. If one
backend fails to initialize IOR cannot be used.
This patch makes IOR initialize only the backends
which will be used. The initialization is done after
that the parameters are checked so that the help message
can still be dispayed is something goes wrong.
2018-09-18 19:24:29 +02:00
Jean-Yves VET
44e8f03917
Fix bug in computing max write and read bandwidths
...
This patch fixes a bug when computing the maximum
bandwidths in short summary output.
2018-09-18 19:23:36 +02:00
Julian M. Kunkel
fd34b08e05
Added patch fix O_DIRECT again.
2018-09-18 16:48:59 +01:00
Mohamad Chaarawi
0a7b1e38ea
fix verbose output in DAOS & DFS drivers
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-09-13 13:32:02 +00:00
Mohamad Chaarawi
2f713e947d
fix bug in driver name check for DAOS.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-09-12 22:53:42 +00:00
Jean-Yves VET
151a9daa52
Check provided IO API is valid
...
Context: IOR gets a segfault if an unsupported API string
is provided.
This patch checks that the I/O backend is supported, otherwise
IOR stops with an error.
2018-09-10 19:16:28 +02:00
Julian M. Kunkel
1a4fd7d095
Fix -i option #81
2018-09-05 09:14:22 +01:00
Mohamad Chaarawi
2668363fb1
- update README for DAOS drivers.
...
- update flag options in DAOS API
- make svcl option in DFS API the same as DAOS API
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-08-30 14:47:17 +00:00
Julian M. Kunkel
2d42c25f69
Fix issue #79 for the last run in a configuration file.
2018-08-30 15:10:38 +01:00
Mohamad Chaarawi
14d67c19d9
fix DAOS plugin options passing.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-08-29 22:25:48 +00:00
Mohamad Chaarawi
46ff4e26be
Merge remote branch 'daos-stack/daos'
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Conflicts:
configure.ac
src/Makefile.am
src/aiori.c
src/aiori.h
src/ior.c
src/mdtest.c
src/parse_options.c
2018-08-29 21:14:19 +00:00
Julian Kunkel
ba2f0330b7
Merge pull request #80 from roblatham00/more-fixes
...
We are getting there ^-^
2018-08-29 21:09:31 +01:00
Julian M. Kunkel
c1f3f485ec
Bugfix #79
2018-08-29 21:07:34 +01:00
Rob Latham
cb88c4c19a
including lustre_user now deprecated
...
put in a configure test (same one used in ROMIO) in case someone's using
old Lustre client headers
2018-08-28 16:27:21 -05:00
Rob Latham
4a32a2cbb3
fix "number of arguments doesn’t match prototype" error
2018-08-28 16:24:34 -05:00
Rob Latham
6b782226c7
fix pnetcdf build issues
...
It does not look like anyone has built the pnetcdf driver in a long time
2018-08-28 14:29:36 -05:00
Rob Latham
982f266a73
clean up hints in HDF5 error cases
2018-08-28 13:30:51 -05:00
Mohamad Chaarawi
f4b03efd72
Add the DAOS ior driver.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-08-27 17:22:38 +00:00
Mohamad Chaarawi
5fb850c810
- update the DFS driver to latest DFS API.
...
- update cmd line options to add DAOS Pool and Container uuid and SVCL
- Add init/finalize backend functions.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-08-23 21:58:53 +00:00
Julian M. Kunkel
4dcf36c1a4
Help text now outputs *compiled* APIs.
2018-08-15 17:07:17 +01:00
Shane Snyder
fefefa002d
make sure to allow fsyncs in RADOS
2018-08-14 17:08:04 -05:00
Shane Snyder
dfc86b60b8
add RADOS backend options for pool, config file
2018-08-14 16:56:19 -05:00
Shane Snyder
0ca5f26e84
cleanup RADOS func names and docs
2018-08-14 16:18:35 -05:00
Julian Kunkel
dc7187794c
Merge pull request #70 from DDNStorage/fix-ime-backend
...
Fix compilation of some aiori backends and add new function calls in IME backend
2018-08-10 18:00:37 +01:00
Glenn K. Lockwood
13b885ce21
Merge pull request #71 from DDNStorage/fixwarnings
...
Fix warnings reported at compilation time
2018-08-09 09:46:24 -07:00
Jean-Yves VET
490a912e6e
Add support for new function calls in the IME backend
...
This patch adds initialize, finalize and stat calls to
the IME backend. statfs, mkdir and rmdir are currently
not supported. This patch also fixes the IME_GetVersion
call.
2018-08-09 13:08:43 +02:00
Jean-Yves VET
9596b4bbe0
Fix compilation of aiori backends requiring external libraries
...
This patchs makes a few adjustements in the automake file
in order to properly add include and library directories.
Those are required to be able to compile some aiori backends.
2018-08-09 13:08:36 +02:00
Jean-Yves VET
df34f024ba
Fix warnings reported at compilation time
...
This patch fixes most of the warnings caused
by unused variables and assignments from incompatible
type.
2018-08-09 13:04:52 +02:00
Julian M. Kunkel
54fa3fa8e9
Fixed wrong output.
2018-08-07 23:28:19 +01:00
Jean-Yves VET
44ce8e5aaf
Add backend for DDN's Infinite Memory Engine (IME)
...
This patch adds the support of IME Native interface
as a new AIORI backend.
2018-08-07 11:41:07 +01:00
Julian M. Kunkel
e708681f19
Option: check for missing arg.
2018-08-06 18:38:34 +01:00
Julian M. Kunkel
0870ad78b3
Support shared large runs with limited number of files per directory by creating one directory for a bunch of files.
...
Therefore, it uses the -I argument (number of files per directory) and divides it by -n (items).
This will be a number of sub directories that are created on the top-level.
2018-07-26 10:38:06 +01:00
Mohamad Chaarawi
07ec65c0d5
remoce rank_list_parse as it is exposed by DAOS API now.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-07-19 21:36:29 +00:00
Julian M. Kunkel
580fb8cdf7
Bugfix offsetof; thanks George for reporting.
2018-07-15 20:23:10 +01:00
Julian M. Kunkel
f55761d5d2
Refactored results structure to AoS; allowed to keep results per repeat.
2018-07-15 19:38:17 +01:00
Julian M. Kunkel
c7a598a435
New option parser: Fixed overflow for blocksize. Added warning.
2018-07-15 18:37:15 +01:00
Julian M. Kunkel
bb6d4fb2a2
Update create for stonewall.
2018-07-15 10:02:45 +01:00
Julian M. Kunkel
d4ecc76ee8
Stonewall bugfix for delete/stat.
2018-07-14 16:46:24 +01:00
Julian M. Kunkel
c0657866fd
Minor fixes for stonewalling options.
2018-07-14 12:22:36 +01:00
Julian M. Kunkel
5d4ac1a0a0
Bugfix parser: wrong datatype
2018-07-14 11:32:39 +01:00
Julian M. Kunkel
04ba846da2
Fix for SetHints (after changing way strings are managed).
2018-07-14 11:11:47 +01:00
Julian M. Kunkel
b613bf9396
AIORI update to initialization and fix HDF5 with changed API.
2018-07-14 10:47:25 +01:00
Julian M. Kunkel
ac7600d662
Unit-test with automake for library version. Minor fix for library version.
2018-07-14 10:27:31 +01:00
Julian M. Kunkel
5ef95675bf
AIORI initialize() and finalize() functions.
2018-07-14 09:29:08 +01:00
Julian M. Kunkel
a0b8659cc8
IOR appears to work with the new option parser.
2018-07-14 09:09:48 +01:00
Julian M. Kunkel
5e98fa0408
Dummy: option for xfer delay.
2018-07-14 08:50:27 +01:00
Julian M. Kunkel
54e47cf729
IOR: use new option parser.
...
TODO: Parse "-O" options
2018-07-14 08:41:35 +01:00
Julian M. Kunkel
0835191372
Option parser replaced.
...
Benefits: shows currently set options when used with -h
Supports options for child modules, example for aiori-dummy works, RADOS untested.
2018-07-12 18:09:13 +01:00
Julian M. Kunkel
205b20f0f4
Bugfix merge
2018-07-12 16:13:40 +01:00
Julian Kunkel
060c47e80a
Merge branch 'testing' into rados-hacks
2018-07-12 15:53:04 +01:00
Shane Snyder
7981691a63
add a new AIORI backend for RADOS
...
inform aiori interface about RADOS backend
stubbed out aiori backend for rados
additions to get RADOS backend compiling/linking
first cut at rados create/open patha
make sure to return RADOS oid on open/create
implement rados xfer path for WRITE
refactor + implement getfilesize and close
remember to use read_op interface for stat
implement RADOS delete function
don't error in RADOS_Delete for now
implement RADOS set_version
handle open/create flags appropriately
cleanup RADOS error handling
implement read/readcheck/writecheck for RADOS
rados doesn't support directio
implement unsupported aiori ops for RADOS
implement RADOS access call
define rados types if no rados support
2018-07-12 09:36:49 -05:00
Julian M. Kunkel
831a5ef823
JSON: Slight output adaption.
2018-07-09 16:25:35 +01:00
Julian M. Kunkel
0f17d4860f
Pretty printing for normal output.
2018-07-08 16:56:04 +01:00
Julian M. Kunkel
2bc30410cc
JSON output works.
2018-07-08 16:47:23 +01:00
Julian M. Kunkel
20ebeb71b8
Further adaptions to generic output, support of JSON output (partially complete).
2018-07-08 14:59:54 +01:00
Julian M. Kunkel
8525f9734c
Extraction of output completed.
2018-07-08 13:50:20 +01:00
Julian M. Kunkel
02883d4a63
Moved most output to ior-output.c
2018-07-08 13:38:05 +01:00
Julian M. Kunkel
0f7a1f14b9
Support IO redirect from main()
2018-07-08 13:12:18 +01:00
Shane Snyder
0052bff3e9
use backend->access in IOR source
...
Signed-off-by: Julian M. Kunkel <juliankunkel@googlemail.com>
2018-07-08 12:13:38 +01:00
Shane Snyder
8069e018e3
make ncmpi/hdf5 use mpiio access/delete
...
Signed-off-by: Julian M. Kunkel <juliankunkel@googlemail.com>
2018-07-08 12:13:38 +01:00
Julian M. Kunkel
1a3c92718b
Fix merge
2018-07-08 12:13:08 +01:00
Shane Snyder
7d3c7e901e
expose generic aiori_ calls for access, etc.
...
Signed-off-by: Julian M. Kunkel <juliankunkel@googlemail.com>
2018-07-08 12:09:44 +01:00
Enno Zickler
9e97a16fe4
fix output to be allinge with column headers
...
Signed-off-by: Julian M. Kunkel <juliankunkel@googlemail.com>
2018-07-08 11:54:49 +01:00
Julian M. Kunkel
d938f0efe1
MPIIO support fsync.
2018-07-08 00:16:30 +01:00
Julian M. Kunkel
fd4c306b98
MDTest stonewalling with status file.
...
Appears to work; stonewall timer is honored only in the creation phase.
This implies a limit to the objects that then are used for other phases.
2018-07-07 23:56:39 +01:00
Julian M. Kunkel
58f13ba352
Bugfix: extraction to count ranks per host used local communicator.
2018-07-07 22:54:40 +01:00
Julian M. Kunkel
31e3cc2b60
Bugfix extraction further.
2018-07-07 22:46:13 +01:00
Julian M. Kunkel
b27b50184a
Bugfix Refactoring MaxLen
2018-07-07 22:42:05 +01:00
Julian M. Kunkel
ed5787811c
Extracted max pathlen variable.
2018-07-07 22:39:14 +01:00
Julian M. Kunkel
48c17b4d81
Extracted (nearly identical) function DelaySeconds.
2018-07-07 22:26:57 +01:00
Julian M. Kunkel
6f8d2e8845
Some simplification / unification between IOR and MDTest.
...
MDTest needs more refactoring to be really maintainable.
2018-07-07 22:19:42 +01:00
Julian M. Kunkel
7bc868d5cf
Dummy AIORI supporting MDtest ops.
2018-07-07 20:01:03 +01:00
Julian M. Kunkel
b4f356a7b2
Bugfix: StoneWallingStatusFile.
2018-07-07 14:14:55 +01:00
Julian M. Kunkel
812b798f05
IOR allows to store actual performaned stonewalling count into a status file.
2018-07-07 14:01:11 +01:00
Julian M. Kunkel
378789737f
Dummy AIORI driver
2018-07-07 13:25:17 +01:00
Julian M. Kunkel
15c383b0d1
Extended description for #44
2018-07-07 12:20:29 +01:00
Julian M. Kunkel
e10bebe653
MDTest supports only POSIX at the moment. Added command line check.
2018-07-07 12:14:34 +01:00
Julian M. Kunkel
e31b2dcec1
Added missing file.
2018-07-07 11:57:04 +01:00
Julian M. Kunkel
21b8cca8cf
IOR library version.
2018-07-07 11:42:21 +01:00
Julian M. Kunkel
a6bfa0f94b
MDTest library option.
...
Moved core functionality / variables used by both implementations to utilities
2018-07-07 10:42:48 +01:00
Julian M. Kunkel
bcaea2a39f
Replaced getopt with an alternative version (on some machines optind was broken).
2018-07-07 08:41:33 +01:00
Mohamad Chaarawi
1768eff552
update DFS plugin
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-06-20 21:25:22 +00:00
Mohamad Chaarawi
8d0cddd21e
Add DFS ior/mdtest driver
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-06-13 18:37:37 +00:00
Adam Moody
156a514c8b
open file in RDWR when using write + checkWrite
2018-05-16 16:52:20 -07:00
Adam Moody
43cc47b7f0
use MPI_COMM_SELF when opening file to get file size in MPIIO
2018-05-16 12:31:23 -07:00
Li Dongyang
6ba22a19e4
Add mmap as a new AIORI backend
...
It shares the create/open/delete/set_version/get_file_size
functions with POSIX backend.
The mmap backend also supports fsync and fsyncPerWrite options,
and it will use msync() instead and fsync().
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
2018-05-11 12:13:42 +10:00
Shane Snyder
92dfb67e5c
allow fsync in POSIX, MPIIO, and HDFS
2018-04-27 12:44:20 -05:00
Shane Snyder
76f3d341b9
implement MPIIO sync
2018-04-27 11:33:52 -05:00
Shane Snyder
9b75f071f4
use backend->access in IOR source
2018-04-25 17:35:10 -05:00
Shane Snyder
7233bee953
make ncmpi/hdf5 use mpiio access/delete
2018-04-25 17:34:25 -05:00
Shane Snyder
60243e0277
expose MPIIO access and delete calls
2018-04-25 17:32:16 -05:00
Shane Snyder
d7fc07163e
expose generic aiori_ calls for access, etc.
2018-04-25 17:21:48 -05:00
Glenn K. Lockwood
7a371cfeda
Merge pull request #46 from lidongyang/iops
...
Add OPs result columns back
2018-04-23 13:12:43 -07:00
Glenn K. Lockwood
a80cc44fb1
fixes #47
2018-04-23 13:10:04 -07:00
Li Dongyang
0d2c9e5451
Add OPs result columns back
...
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
2018-04-18 14:24:42 +10:00
Enno Zickler
16c8ef6a23
Doc: src comments for getOffsetArray functions
2018-03-15 21:52:34 +01:00
Enno Zickler
d5507955af
Doc: Initial setup for Sphinx and Doxygen
2018-03-15 21:52:10 +01:00
Jean-Yves VET
6d3edaffc8
Implement an access function for MPIIO backend
...
Context: Some file systems require a prefix in the path.
The POSIX 'access' call fails and consequently files are
never deleted.
This patch implements an access function in the MPIIO
backend using MPI_File_open. Prefixes can now be parsed
by ROMIO.
2018-03-01 11:34:49 +01:00
Glenn K. Lockwood
9a200097a5
Merge pull request #31 from ezickler/O_DIRECTfix
...
Fix O_DIRECT option
2018-01-25 09:32:00 -08:00
Enno Zickler
87fe8485a7
Fix O_DIRECT option
...
The O_DIRECT option was not working as set_o_direct_flag() were moved to
utilities.c but there the #define _GNU_SOURCE where missing. This lead
to not the Waring "cannot use O_DIRECT".
2018-01-25 10:51:35 +01:00
Alexander Hück
3ef16c3030
initialize path variable correctly
2018-01-24 19:26:53 +01:00
Glenn K. Lockwood
e6e0527d72
Merge pull request #17 from lwh1990/master
...
Fix compatibility issues with pnetcdf > 1.7 (prior versions continue to work as well)
2017-12-12 10:58:14 -08:00
Glenn K. Lockwood
3887668bda
Merge pull request #19 from VI4IO/master
...
Various bug fixes and code cleanup
2017-12-12 10:56:09 -08:00
Julian M. Kunkel
cdc93da3a4
Check for command line slight naming change.
2017-12-09 17:24:31 +01:00
Julian M. Kunkel
93355314de
Adjusted command line checks for recent changes in semantics for readCheck/writeCheck.
2017-12-09 17:22:58 +01:00
Julian M. Kunkel
1f6d062f7f
Bugfix #22
2017-12-09 11:52:13 +01:00
Glenn K. Lockwood
10e2b2efa2
make all references to API name case-insensitive (fix for 6ba323b
)
2017-12-04 15:20:31 -08:00
Julian M. Kunkel
c9d6c2316e
Bugfix for self referencing sprintf #20
...
Backmerge from IOR fork.
2017-11-30 12:01:19 +01:00
Julian M. Kunkel
2c224e0ee8
Bugfix for #14 , storeFileOffset=1
2017-11-30 11:56:26 +01:00
Julian Kunkel
b67c5ce62c
Bugfix for #14
...
WriteCheck did not work properly due to shifting of patterns.
It is possible that one buffer could be saved by refactoring the code properly.
2017-11-29 10:17:02 +01:00
Julian Kunkel
6436165edb
For debugging of issues: added IOR_FAKE_TASK_PER_NODES env variable to set tasksPerNode.
2017-11-29 09:54:22 +01:00
VinsonLeung
d1f9e9aee3
change Pnetcdf IO funtion
...
change Pnetcdf IO function which can avoid incompatible MPI datatype problem
2017-11-22 23:20:22 -06:00
VinsonLeung
f8e6cd033a
change MPI_BYTE to MPI_SIGNED_CHAR
...
From Pnetcdf 1.7, the MPI datatype corresponding to NC_BYTE is change from MPI_BYTE to MPI_SIGNED_CHAR. If not change, running IOR with NCMPI will cause a fatal error as below:
ERROR in aiori-NCMPI.c (line 287): cannot write to data set.
ERROR: NetCDF: Not a valid data type or _FillValue type mismatch.
2017-11-20 13:56:36 -06:00
VinsonLeung
fd0e99e544
fix a bug when compile with ncmpi enabled
...
The file "src/aiori-NCMPI.c" uses numTasksWorld as the process which is declared in "src/ior.h". In "src/mdtest.c", NCMPI backend will be called if IOR configured with ncmpi enabled but numTasksWorld was not defined in "src/mdtest.c". So it will cause a compiler error like below:
mdtest-aiori-NCMPI.o: In function `NCMPI_Xfer':
/home/parallels/Documents/ior/src/aiori-NCMPI.c:272: undefined reference to `numTasksWorld'
2017-11-19 16:38:12 -06:00
Glenn K. Lockwood
3aa1f4c9b7
Merge pull request #11 from VI4IO/master
...
StoneWalling with WearOutPhase
2017-11-15 00:09:22 -07:00