8de13884a7
HDFS module ported to current AIORI API and improved ( #302 )
...
* HDFS module ported to current AIORI API and extended
* Added instructions how to build and run with a HDFS.
* Replaced read with pread to support offsets.
* Implemented HDFS MDTest API
* Improved sync semantics
2020-12-23 11:51:31 +00:00
6675cd50bf
aiori-DFS: stat should not be fatal
...
ior now expects stat to not be fatal. update the DFS driver to not
exit if stat fails since the file can simply not exist.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2020-12-16 15:41:58 +00:00
0bd52884e8
added region and location to libs3 ( #295 )
...
* added region and location to libs3
2020-12-11 13:05:24 +00:00
fbee4139d8
Merge pull request #292 from ofaaland/b-fix-mpi-checkf
...
Do not execute functions twice in MPI_CHECKF
2020-12-04 09:20:21 +00:00
69e006dd89
IOR clarify return API.
2020-12-03 21:07:45 +00:00
073da08292
IOR: print overwrite check only if a write operation is requested.
2020-12-03 20:54:51 +00:00
231868505d
Do not execute functions twice in MPI_CHECKF
...
Assigning MPI_STATUS to a local variable and then referring to the
local will ensure that the same value is used in both the conditional
expression and the call to MPI_Error_string.
Otherwise, when MPI_STATUS is a function call, like
MPI_CHECKF(fubar(), "%s", "error in fubar");
fubar() is called twice. If there are underlying intermittent errors,
the error code/message for the first call is lost, with confusing output
like this:
read 2206.18 17.27 145.93 262144 131072
0.272595 291.88 0.290829 292.41
ERROR: cannot access explicit, noncollective, MPI MPI_SUCCESS: no errors, (aiori-MPIIO.c:451)
2020-12-03 12:21:06 -08:00
a8ab766483
Merge pull request #270 from hpc/fix-offset-integration
...
Fix offset integration
2020-12-03 12:08:53 +00:00
09d2c3511b
Merge pull request #289 from hpc/fix-ior-no-warn
...
IOR add warning if file exists #273 . To stop on warnings, use existing flag --warningAsErrors
2020-12-02 10:41:44 +00:00
b417505dfd
Merge branch 'master' into fix-modules
2020-12-02 10:21:59 +00:00
f71a144b19
IOR raise warning for existing file once if not filePerProc.
2020-12-02 10:14:28 +00:00
938cf2771b
DFS: make use of collective open to share file handle
...
- remove un-needed libdaos_common lib since DAOS driver is removed
- use default container object class when it's not specified
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2020-12-01 21:36:25 +00:00
da03ae5c9f
IOR add warning if file exists #273 . To stop on warnings, use existing flag --warningAsErrors
2020-12-01 18:24:09 +00:00
691324f17b
Merge pull request #287 from hpc/fix-warning
...
Fix compiler issue with redefinition via typedef.
2020-12-01 17:44:14 +00:00
9dcf9f79e4
Moved barriers to right location.
2020-12-01 14:47:57 +00:00
546eb05d15
Rudimentary support for stonewall added, addresses #261 .
2020-12-01 14:45:07 +00:00
5825dbae35
Provide new option randomPrefill for random (-z access only) that prefill the file with the specified blocksize, e.g., 2m. See issue #270 .
...
This option works so far only without stonewalling!
2020-12-01 13:58:40 +00:00
41184c69e2
mdtest: fix reference to unique_dir_per_task for builds --with-lustre
2020-12-01 01:22:31 -05:00
ae06908a93
Merge branch 'master' into fix-mdtest-iter
2020-11-30 14:16:41 +00:00
11c784c8bd
Integrate review feedback.
2020-11-28 10:40:41 +00:00
ae23523a70
Integrate review feedback
2020-11-28 10:34:20 +00:00
5799e4ef3a
MDTest remove unnede variable.
2020-11-27 18:02:14 +00:00
fd51654393
Readd check for mdtest for now.
2020-11-27 17:51:31 +00:00
fbf976351a
MDTest refactoring: move all global static variables into the global static structure.
2020-11-27 17:49:45 +00:00
ae8a11b42f
MDTest updated stonewall check.
2020-11-27 15:35:32 +00:00
7542e75c82
MDTest ignore verbose potential format overflows to be able to spot real errors.
2020-11-27 15:23:32 +00:00
980ab1dc97
Lustre stripping: Fix default value such that Lustre striping is only set if anything is changed.
2020-11-26 15:56:34 +00:00
4377aebcf8
Bugfix MDTest calculation of multiple iterations was incorrect.
2020-11-26 12:48:11 +00:00
ad6dfc5e63
Allow MDTest to print per proc.
2020-11-25 09:50:26 +00:00
02a47085ec
Bugfix: suboptimal name mapping was leading to accidential deletes in md-workbench.
2020-11-22 12:01:12 +00:00
b5891141d8
Move checks before inititalization.
...
Add simple validation for S3.
2020-11-22 11:43:42 +00:00
5d489064d8
Merge pull request #271 from hpc/fix-statvfs
...
Remove duplicated functionality between mdtest and IOR.
2020-11-13 09:24:24 +00:00
13f9721dbc
Merge pull request #267 from hpc/feature-md-workbench
...
Feature: md-workbench
2020-11-10 19:05:20 +00:00
277f380139
Remove duplicated functionality between mdtest and IOR.
...
Refactored the ShowFileSystemSize function.
2020-11-09 16:23:34 +00:00
306598db67
Bugfix memory issue in global options.
2020-11-06 11:40:31 +00:00
a59e98d7a6
Add error to indicate changed behavior to users
2020-11-06 10:39:16 +00:00
af0753d8ad
Random: last modifications.
2020-11-06 10:27:11 +00:00
a0e5e297d6
Create random for one block (1 segment) only to be repeated.
2020-11-06 09:52:35 +00:00
ad985af763
Location problem on non Linux systems.
2020-11-06 09:30:59 +00:00
0e91769328
aiori-Gfarm: sequel to #262 - sync interface ( #269 )
...
* aiori-Gfarm: update to the new aiori interface
* aiori-Gfarm: Gfarm_sync
2020-11-06 09:15:39 +00:00
2d79efc0c5
Fix wait issue on MacOS
2020-11-05 19:13:08 +00:00
3ee3e9ad5c
Integrated sequential code into the benchmark main loop.
...
A bit ugly but shows that the code logic hasn't changed.
2020-11-05 14:00:09 +00:00
75c08386a4
Bugfix porting error for performance stats.
2020-11-04 20:47:30 +00:00
c0ffdf44d0
Workbench API improved.
2020-11-04 20:25:27 +00:00
d9c74af8f3
Fix (accidently rename of option)
2020-11-04 19:27:39 +00:00
b5dfeea82a
Remove output for the API.
2020-11-04 18:04:46 +00:00
fb66e77072
Enable random seed to be stored. ( #268 )
2020-11-04 13:47:35 +00:00
4e452766b9
Enable random seed to be stored.
2020-11-04 12:48:46 +00:00
e1dd3103cf
Fix make dist for md-workbench.
2020-11-03 10:52:45 +00:00
cb397242f9
DAOS backend cleanup ( #266 )
...
- remove legacy DAOS driver & update Readme.
- update configure options to remove cart requirement
- Optimize DFS file get_size
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2020-11-03 10:01:09 +00:00
Julian Kunkel