f71a144b19
IOR raise warning for existing file once if not filePerProc.
2020-12-02 10:14:28 +00:00
464f562f44
Merge pull request #290 from mchaarawi/dfs_coll_open
...
DFS: make use of collective open to share file handle
2020-12-02 09:15:37 +00:00
938cf2771b
DFS: make use of collective open to share file handle
...
- remove un-needed libdaos_common lib since DAOS driver is removed
- use default container object class when it's not specified
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2020-12-01 21:36:25 +00:00
da03ae5c9f
IOR add warning if file exists #273 . To stop on warnings, use existing flag --warningAsErrors
2020-12-01 18:24:09 +00:00
691324f17b
Merge pull request #287 from hpc/fix-warning
...
Fix compiler issue with redefinition via typedef.
2020-12-01 17:44:14 +00:00
9dcf9f79e4
Moved barriers to right location.
2020-12-01 14:47:57 +00:00
546eb05d15
Rudimentary support for stonewall added, addresses #261 .
2020-12-01 14:45:07 +00:00
5825dbae35
Provide new option randomPrefill for random (-z access only) that prefill the file with the specified blocksize, e.g., 2m. See issue #270 .
...
This option works so far only without stonewalling!
2020-12-01 13:58:40 +00:00
d1db4cbfe0
Merge pull request #286 from rfmohr/fix-undeclared-unique_dir_per_task
...
mdtest: fix reference to unique_dir_per_task for builds --with-lustre
2020-12-01 09:40:39 +00:00
41184c69e2
mdtest: fix reference to unique_dir_per_task for builds --with-lustre
2020-12-01 01:22:31 -05:00
4a3e4806bd
Merge pull request #281 from hpc/fix-mdtest-iter
...
Bugfix MDTest calculation of multiple iterations was incorrect.
Fix the bug reported by Rick to increase clarity. The previous offset calculation when using multiple iterations was:
for (i = start; i < stop; i++) // i = table position == test number
for (k=0; k < size; k++)
for (j = 0; j < iterations; j++)
value = all[(k * tableSize * iterations) + (j*tableSize) + i];
Note that the mean and min/max was then computed over these values.
But as the values were stored in memory in the order: iteration, rank, table
the correct term is: value = all[j * tableSize * size + k * tableSize + i];
Assume iterations = 2 and size = 3, the value for the test i=0 was computed from:
all[0 * 2 *tbl + 0 * tbl] = 0tbl
all[0 * 2 *tbl + 1 * tbl] = 1tbl
all[1 * 2 *tbl + 0 * tbl] = 2tbl
all[1 * 2 *tbl + 1 * tbl] = 3tbl
all[2 * 2 *tbl + 0 * tbl] = 4tbl
all[2 * 2 *tbl + 1 * tbl] = 5tbl
A more clear traversal would have been:
all[0 * 3 *tbl + 0 * tbl] = 0tbl
all[0 * 3 *tbl + 1 * tbl] = 1tbl
all[0 * 3 *tbl + 2 * tbl] = 2tbl
all[1 * 3 *tbl + 0 * tbl] = 3tbl
all[1 * 3 *tbl + 1 * tbl] = 4tbl
all[1 * 3 *tbl + 2 * tbl] = 5tbl
In that sense, it wasn't a functional bug but it decreased readability and now that we want to print the performance of the individual ranks, it is useful to fix this.
2020-11-30 14:17:42 +00:00
ae06908a93
Merge branch 'master' into fix-mdtest-iter
2020-11-30 14:16:41 +00:00
312edaa99c
Merge pull request #283 from hpc/fix-mdtest-refactor
...
Refactoring: MDTest
This PR refactors the 50+ static variables to be part of a global option structure to remove ambiguity in which scope the variable is.
In the code, there is at the moment too much dependency to global variables.
Ultimately, the goal was to make stonewalling work with the multi-dir approach but it turned out this requires much more refactoring and therefore, doing it piece by piece.
2020-11-30 14:11:22 +00:00
11c784c8bd
Integrate review feedback.
2020-11-28 10:40:41 +00:00
ae23523a70
Integrate review feedback
2020-11-28 10:34:20 +00:00
5799e4ef3a
MDTest remove unnede variable.
2020-11-27 18:02:14 +00:00
fd51654393
Readd check for mdtest for now.
2020-11-27 17:51:31 +00:00
fbf976351a
MDTest refactoring: move all global static variables into the global static structure.
2020-11-27 17:49:45 +00:00
ae8a11b42f
MDTest updated stonewall check.
2020-11-27 15:35:32 +00:00
7542e75c82
MDTest ignore verbose potential format overflows to be able to spot real errors.
2020-11-27 15:23:32 +00:00
b658cfce4d
Merge pull request #282 from hpc/fix-lustre-stripe
...
Lustre stripping: Fix default value such that Lustre striping is only set if anything is changed.
2020-11-26 16:00:14 +00:00
980ab1dc97
Lustre stripping: Fix default value such that Lustre striping is only set if anything is changed.
2020-11-26 15:56:34 +00:00
4377aebcf8
Bugfix MDTest calculation of multiple iterations was incorrect.
2020-11-26 12:48:11 +00:00
9349284b82
Merge pull request #279 from hpc/feature-print-per-rpoc
...
Allow MDTest to print per proc.
2020-11-25 10:49:35 +00:00
ad6dfc5e63
Allow MDTest to print per proc.
2020-11-25 09:50:26 +00:00
2f0d311ed2
Merge pull request #278 from adilger/mdtest-man
...
Add missing options to mdtest man page
2020-11-25 09:19:21 +00:00
76d5a77ac2
Add missing options to mdtest man page
2020-11-25 01:28:07 -07:00
b12742e1ad
Merge pull request #275 from hpc/fix-libs3
...
Bugfix S3: suboptimal name mapping was leading to accidential deletes in md-workbench.
2020-11-22 12:05:51 +00:00
02a47085ec
Bugfix: suboptimal name mapping was leading to accidential deletes in md-workbench.
2020-11-22 12:01:12 +00:00
c7f2870004
Merge pull request #274 from hpc/fix-libs3
...
Move option checks before inititalization.
2020-11-22 11:44:57 +00:00
b5891141d8
Move checks before inititalization.
...
Add simple validation for S3.
2020-11-22 11:43:42 +00:00
5d489064d8
Merge pull request #271 from hpc/fix-statvfs
...
Remove duplicated functionality between mdtest and IOR.
2020-11-13 09:24:24 +00:00
13f9721dbc
Merge pull request #267 from hpc/feature-md-workbench
...
Feature: md-workbench
2020-11-10 19:05:20 +00:00
277f380139
Remove duplicated functionality between mdtest and IOR.
...
Refactored the ShowFileSystemSize function.
2020-11-09 16:23:34 +00:00
65666faf8b
Updated tests to have transfer != blocksize for new random variant.
2020-11-06 11:43:52 +00:00
306598db67
Bugfix memory issue in global options.
2020-11-06 11:40:31 +00:00
a59e98d7a6
Add error to indicate changed behavior to users
2020-11-06 10:39:16 +00:00
af0753d8ad
Random: last modifications.
2020-11-06 10:27:11 +00:00
a0e5e297d6
Create random for one block (1 segment) only to be repeated.
2020-11-06 09:52:35 +00:00
ad985af763
Location problem on non Linux systems.
2020-11-06 09:30:59 +00:00
0e91769328
aiori-Gfarm: sequel to #262 - sync interface ( #269 )
...
* aiori-Gfarm: update to the new aiori interface
* aiori-Gfarm: Gfarm_sync
2020-11-06 09:15:39 +00:00
2d79efc0c5
Fix wait issue on MacOS
2020-11-05 19:13:08 +00:00
3ee3e9ad5c
Integrated sequential code into the benchmark main loop.
...
A bit ugly but shows that the code logic hasn't changed.
2020-11-05 14:00:09 +00:00
75c08386a4
Bugfix porting error for performance stats.
2020-11-04 20:47:30 +00:00
c0ffdf44d0
Workbench API improved.
2020-11-04 20:25:27 +00:00
d9c74af8f3
Fix (accidently rename of option)
2020-11-04 19:27:39 +00:00
b5dfeea82a
Remove output for the API.
2020-11-04 18:04:46 +00:00
fb66e77072
Enable random seed to be stored. ( #268 )
2020-11-04 13:47:35 +00:00
4e452766b9
Enable random seed to be stored.
2020-11-04 12:48:46 +00:00
e1dd3103cf
Fix make dist for md-workbench.
2020-11-03 10:52:45 +00:00
Julian M. Kunkel