Commit Graph

176 Commits (8d0cddd21e7461a4bc26cac0c9d391dcfcf9a891)

Author SHA1 Message Date
Mohamad Chaarawi 8d0cddd21e Add DFS ior/mdtest driver
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-06-13 18:37:37 +00:00
Glenn K. Lockwood b172cd684f
Merge pull request #56 from adammoody/rdwr
fixes #55: open file in RDWR when using write + checkWrite
2018-05-16 18:19:17 -07:00
Adam Moody 156a514c8b open file in RDWR when using write + checkWrite 2018-05-16 16:52:20 -07:00
Glenn K. Lockwood e5a01edff9
Merge pull request #54 from adammoody/mpiio_getfilesize
use MPI_COMM_SELF when opening file to get file size in MPIIO file per process
2018-05-16 13:30:49 -07:00
Adam Moody 43cc47b7f0 use MPI_COMM_SELF when opening file to get file size in MPIIO 2018-05-16 12:31:23 -07:00
Glenn K. Lockwood 945fba2aa2
Merge pull request #52 from lidongyang/mmap
Add mmap as a new AIORI backend
2018-05-10 20:54:34 -07:00
Li Dongyang 6ba22a19e4 Add mmap as a new AIORI backend
It shares the create/open/delete/set_version/get_file_size
functions with POSIX backend.
The mmap backend also supports fsync and fsyncPerWrite options,
and it will use msync() instead and fsync().

Signed-off-by: Li Dongyang <dongyangli@ddn.com>
2018-05-11 12:13:42 +10:00
Glenn K. Lockwood 7a371cfeda
Merge pull request #46 from lidongyang/iops
Add OPs result columns back
2018-04-23 13:12:43 -07:00
Glenn K. Lockwood a80cc44fb1 fixes #47 2018-04-23 13:10:04 -07:00
Li Dongyang 0d2c9e5451 Add OPs result columns back
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
2018-04-18 14:24:42 +10:00
Glenn K. Lockwood 907cbb806b
Merge pull request #42 from ezickler/wip/new-doc
New documention with sphinx and Doxygen for readthedoc.io
2018-03-16 14:34:05 -07:00
Enno Zickler 16c8ef6a23 Doc: src comments for getOffsetArray functions 2018-03-15 21:52:34 +01:00
Enno Zickler 6267203147 Doc: add tutorial, user guide and CI
This adds a beginners tutorial and splits up the conten of the old user
guide in categories. In the developer section gets a page for the
travis-CI.
2018-03-15 21:52:34 +01:00
Enno Zickler d5507955af Doc: Initial setup for Sphinx and Doxygen 2018-03-15 21:52:10 +01:00
Glenn K. Lockwood a4df0fda55
Merge pull request #37 from ezickler/feature/travis-ci
Add Travis-CI config file
2018-03-09 16:00:47 -08:00
Glenn K. Lockwood c40317f75d
Merge pull request #39 from jyvet/mpiio-access
Implement an access function for MPIIO backend
2018-03-02 09:59:03 -08:00
Jean-Yves VET 6d3edaffc8 Implement an access function for MPIIO backend
Context: Some file systems require a prefix in the path.
The POSIX 'access' call fails and consequently files are
never deleted.

This patch implements an access function in the MPIIO
backend using MPI_File_open. Prefixes can now be parsed
by ROMIO.
2018-03-01 11:34:49 +01:00
Enno Zickler 11d3a929ff Travis-CI: Add config file
This adds a .travis.yaml file to use travis-ci for continuous
integration.
It currently compiles IOR on an ubuntu 14.04 machine with
only the POSIX, MPIIO and HDF5 backends enabled. For Lustre, GPFS,
NCMPI, HDFS and S3 libraries needs to be compiled as they are not
present in the ubuntu repositories. Lustre and NCMPI should be easy when
ubuntu 16.04 is available at travis-ci. S3 could be compiled by source
which shouldn't be hard but needs to be done if one is interested.
Hadoop needs a more complex installation process. GPFS seams not
possible as it is proprietary and needs a license as far as i know.

Repo Owner needs to enable CI on travis-ci.org. It's free for open
source git hub repos.
2018-02-28 14:06:12 +01:00
Glenn K. Lockwood 9a200097a5
Merge pull request #31 from ezickler/O_DIRECTfix
Fix O_DIRECT option
2018-01-25 09:32:00 -08:00
Enno Zickler 87fe8485a7 Fix O_DIRECT option
The O_DIRECT option was not working as set_o_direct_flag() were moved to
utilities.c but there the  #define _GNU_SOURCE where missing. This lead
to not the Waring "cannot use O_DIRECT".
2018-01-25 10:51:35 +01:00
Glenn K. Lockwood 0e7f48d649
Merge pull request #29 from ahueck/fix_realpath
initialize path variable correctly
2018-01-24 10:51:09 -08:00
Alexander Hück 3ef16c3030 initialize path variable correctly 2018-01-24 19:26:53 +01:00
Glenn K. Lockwood e6e0527d72
Merge pull request #17 from lwh1990/master
Fix compatibility issues with pnetcdf > 1.7 (prior versions continue to work as well)
2017-12-12 10:58:14 -08:00
Glenn K. Lockwood 3887668bda
Merge pull request #19 from VI4IO/master
Various bug fixes and code cleanup
2017-12-12 10:56:09 -08:00
Julian M. Kunkel cdc93da3a4 Check for command line slight naming change. 2017-12-09 17:24:31 +01:00
Julian M. Kunkel 93355314de Adjusted command line checks for recent changes in semantics for readCheck/writeCheck. 2017-12-09 17:22:58 +01:00
Julian M. Kunkel 1f6d062f7f Bugfix #22 2017-12-09 11:52:13 +01:00
Glenn K. Lockwood 10e2b2efa2 make all references to API name case-insensitive (fix for 6ba323b) 2017-12-04 15:20:31 -08:00
Julian M. Kunkel c9d6c2316e Bugfix for self referencing sprintf #20
Backmerge from IOR fork.
2017-11-30 12:01:19 +01:00
Julian M. Kunkel 2c224e0ee8 Bugfix for #14, storeFileOffset=1 2017-11-30 11:56:26 +01:00
Julian Kunkel b67c5ce62c Bugfix for #14
WriteCheck did not work properly due to shifting of patterns.
It is possible that one buffer could be saved by refactoring the code properly.
2017-11-29 10:17:02 +01:00
Julian Kunkel 4c5856469b Configuration to trigger #14 2017-11-29 10:02:34 +01:00
Julian Kunkel 6436165edb For debugging of issues: added IOR_FAKE_TASK_PER_NODES env variable to set tasksPerNode. 2017-11-29 09:54:22 +01:00
VinsonLeung d1f9e9aee3
change Pnetcdf IO funtion
change Pnetcdf IO function which can avoid incompatible MPI datatype problem
2017-11-22 23:20:22 -06:00
Glenn K. Lockwood 6faf076f11
Merge pull request #18 from ikirker/master
Hardens test for MPI
2017-11-22 13:25:36 -07:00
Ian Kirker 1dce7ffdb6 Hardens test for MPI
Previously, the configure script could fail to find an MPI
implementation but continue regardless. This alters the behaviour so
that the configure script fails if the check for MPI does.
2017-11-22 20:13:53 +00:00
Glenn K. Lockwood b0bffa0d0e fix for ac < 2.67; untested, but doesn't break newer autoconfs 2017-11-22 11:27:18 -08:00
Glenn K. Lockwood 84c7a18fe9 fix for autoconf < 2.69 2017-11-22 11:27:05 -08:00
VinsonLeung f8e6cd033a
change MPI_BYTE to MPI_SIGNED_CHAR
From Pnetcdf 1.7, the MPI datatype corresponding to NC_BYTE  is change from MPI_BYTE to MPI_SIGNED_CHAR. If not change, running IOR with NCMPI will cause a fatal error as below:
ERROR in aiori-NCMPI.c (line 287): cannot write to data set.
ERROR: NetCDF: Not a valid data type or _FillValue type mismatch.
2017-11-20 13:56:36 -06:00
Glenn K. Lockwood 3995f975c0
Merge pull request #16 from lwh1990/master
compile fixes for macOS (lseek64 and open64; NCMPI build)
2017-11-20 12:29:34 -07:00
VinsonLeung fd0e99e544
fix a bug when compile with ncmpi enabled
The file "src/aiori-NCMPI.c" uses numTasksWorld as the process which is declared in "src/ior.h".   In "src/mdtest.c", NCMPI backend will be called if IOR configured with ncmpi enabled but numTasksWorld was not defined in "src/mdtest.c". So it will cause a compiler error like below:
mdtest-aiori-NCMPI.o: In function `NCMPI_Xfer':
/home/parallels/Documents/ior/src/aiori-NCMPI.c:272: undefined reference to `numTasksWorld'
2017-11-19 16:38:12 -06:00
VinsonLeung 9c4bf51d82
Update cbif.c
fix a bug when compiling IOR in MacOS will cause error "Undefined symbols for _lseek64 and _open64"
2017-11-19 15:27:34 -06:00
Glenn K. Lockwood 3aa1f4c9b7
Merge pull request #11 from VI4IO/master
StoneWalling with WearOutPhase
2017-11-15 00:09:22 -07:00
John Bent 34cb63c1b7 Fixed the bug in the read check code 2017-10-25 07:57:50 -06:00
John Bent 53113536e4 Update README 2017-10-23 14:40:59 -06:00
Glenn K. Lockwood e1968cd4ad Merge pull request #7 from hjelmn/master
Add mdtest to IOR repo and update backends to support mdtest
2017-10-23 12:03:48 -07:00
Julian M. Kunkel 0916d432cd Removed short option parsing. 2017-10-21 12:02:03 +02:00
Julian M. Kunkel 072eb452fd Stonewalling wear-out only with long options. 2017-10-21 11:59:09 +02:00
Julian Kunkel 96e75cc832 stoneWallingWearOutIterations allows to read data back using the same number of iterations as used for the write with stoneWallingWearOut. 2017-10-20 18:14:06 +02:00
Julian Kunkel aa0b8c92d2 New stonewalling option to create a wear out phase.
Once a process hits the stonewall (timelimit), they all figure out the maximum pair read/written.
Each proces continues to read/write until the maximum number of pairs is reached, this simulates the wear out.
2017-10-20 18:02:24 +02:00