f16ef9ace5
update dfs_lookup() call for extra parameter.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-08-03 05:07:13 +00:00
b72b51be48
Fix: Heap Buffer Overflow
...
Fix a memory violation when run in serial.
2019-08-02 23:34:13 -05:00
bfff0df8fd
Fix Some Memory Leaks
...
Fixing some memory leaks :)
2019-08-02 23:33:01 -05:00
b1b66962ac
incorrect warning
2019-08-02 13:03:59 +09:00
cf56715a5a
Make sure that each read buffer contains an invalid first byte.
2019-08-01 18:33:44 +01:00
ce1ae750f6
MDtest: Support to verify the read operation with a default pattern.
2019-08-01 18:29:32 +01:00
df8355a9bc
Added output of mdtest stonewall timer.
2019-08-01 17:57:45 +01:00
061b5a860f
Backmerged: New option: print rate AND time; improves debugging.
2019-08-01 17:54:11 +01:00
6c0fadc2a9
Include performance when stonewall is hit to output.
2019-08-01 17:20:01 +01:00
3890b71b54
Fixed issues and followed suggestions from Glenn's review of the PR
2019-08-01 09:42:03 +09:00
92939e4fbd
update for DAOS API changes
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-07-31 17:22:20 +00:00
a3c37808da
Made FAIL take variable args so we can pass printf like args to it
2019-07-28 11:17:11 -06:00
0ffec67d2b
Following Julian's suggestion about better naming
2019-07-28 10:25:42 -06:00
b2d486f749
Followed Andreas suggestion to replace escape double quotes within printf's with single quotes
2019-07-28 10:07:03 -06:00
168a407793
Fixed inconsistent spacing that Andreas commented upon
2019-07-28 09:55:00 -06:00
629ff810b7
Got IOR shifting to work regardless of whether node/task mapping is round-robin or contiguous
2019-07-27 15:27:20 -06:00
d69957e55b
Final changes cleaning up the output messages
2019-07-27 14:31:49 -06:00
e767ef3de9
Remove extraneous print_help function that was causing people to have to edit the same string in two different locations
2019-07-27 13:26:39 -06:00
f6491fcd37
Cleaned up the verbose messages by creating a macro and a function
2019-07-27 13:22:15 -06:00
9d6480d46e
Fixed bug in the nstride calculation where only 0 was computing it correctly. Had to bcast it out
2019-07-27 11:30:07 -06:00
945487f743
Better debug message when stat fails
2019-07-27 09:20:20 -06:00
524d053be1
Making shift work in mdtest as well as it works in IOR and on a per-node basis.
...
Also added printing the nodemap so we can check the allocation.
2019-07-26 08:55:24 -06:00
7981fc70b1
Changed the behavior of shift in mdtest to mimic that in IOR so that it shifts by _entire nodes_ not by ranks.
2019-07-25 19:18:52 -06:00
76f6bb6e42
Abort when invalid options are set.
2019-07-24 00:29:58 +01:00
1c640716c7
Use MPI Info object for all MPI_File_open calls.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-07-08 21:31:00 +00:00
ad61fdd89c
add proper check for DAOS object class and exit if unknown.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-07-08 18:18:24 +00:00
04e3df0d42
use the daos provided function for name to object class
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-07-08 17:02:30 +00:00
52ed766df4
update DAOS and DFS drivers to new DAOS API changes.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-07-08 15:33:06 +00:00
3eb0f86640
Function ncmpi_get_file_info was first introduced in
...
PnetCDF version 1.2.0. Check PnetCDF library version,
so printing MPI-IO hints can be enabled for NCMPI.
2019-06-28 17:40:35 -05:00
9f9730d6fd
should abort on daos init if pool/cont/svcl not specified.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-06-28 16:20:56 +00:00
2ae7c29de2
cache the parent dir in a hashtable so we don't have to do a lookup
...
on every access.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-06-27 21:31:07 +00:00
960005351d
update DAOS and DFS markings.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-06-25 22:41:30 +00:00
262d35d87e
- Merge branch 'master' into daos-devel
...
- fix bugs with cont destroy.
- add destroy option to DFS driver
- share pool and container handle with DFS driver, and allow multi rank access
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Conflicts:
src/ior.c
src/mdtest.c
src/option.c
src/parse_options.c
2019-06-24 21:26:15 +00:00
a822363ee0
style fix
2019-06-22 10:48:41 +09:00
6cae44b512
abstract I/O interface for Gfarm file system
2019-06-21 21:05:41 +09:00
915283bae8
update array_g2l usage with DAOS API change
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-06-17 20:38:02 +00:00
af1aa627eb
use d_ version of function instead of daos_ since some of those are
...
going to be removed from the DAOS API.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-06-07 16:59:11 +00:00
cdaca2c591
Merge pull request #160 from hpc/fix-ime-157
...
Moving IME ODirect option into IME. #157
2019-05-28 08:41:26 +01:00
e4969b5af7
Fix for #159 (also slightly improved parsing for options, for portability between scripts also allow comment at the end of an option).
2019-05-23 12:47:43 +01:00
1b3182904d
Moving IME ODirect option into IME. #157
2019-05-23 09:54:10 +01:00
ed4c1d88d0
Fix missing initialization in MMAP options.
2019-05-23 09:31:05 +01:00
a6f8388f9f
add mknod support
...
Add new option '-k' to support file creation test with mknod,
which is widely used in lustre.
Signed-off-by: Gu Zheng <gzheng@ddn.com>
2019-05-23 13:41:46 +08:00
3d8893e507
Fix #155 . Fixup 3a873ea fixing #150 .
2019-05-22 15:25:27 +02:00
cf197d5e38
Moved broken collectiveMetadata option into HDF5 backend module. #154
2019-05-22 10:21:18 +01:00
c5c43c4f3c
Changed the version name to alpha to differentiate. Now mdtest has the version number from IOR.
2019-05-16 21:25:34 +01:00
3a873ea2b0
Fix #150 .
2019-05-16 19:35:00 +01:00
6abdcc363a
Trivial fix for #149
2019-05-16 12:56:56 +01:00
9ead56c24e
Merge pull request #148 from hpc/fix-146
...
Moving options to submodules + fixes #146 + fixes #144
2019-05-14 14:32:37 -07:00
30a1ca187b
Bugfix by rmn1 for #147 and #112 .
2019-05-14 14:55:11 +01:00
54522315ec
Fix #146 ; handling of hints. Why are GPFS hints special, remains unclear.
2019-05-08 14:02:40 +01:00
a9b0bf35bc
Support MDTest verbosity >=5 prints behavior for all ranks instead of just rank 0 (which is enabled with verbosity level 3 as before)
2019-05-02 09:58:35 +01:00
e3afdb5ed5
beegfs: don't fail if output exists
...
Use existing output file if it has the desired stripe settings
(number of targets, chunk size)
2019-04-29 15:52:26 +02:00
8fd755147a
Fix for #144
2019-04-19 10:20:04 +01:00
5797624d56
fix bug in file delete and access where only rank 0 is calling those functions.
...
remove the bcasts for the error code since the other ranks are not calling it.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-04-15 22:14:33 +00:00
27ac1b88af
Removed verbose output.
2019-04-02 12:34:35 +01:00
07217c93aa
Bugfix for the parsing of flags. Backwards compatibility for direct IO.
2019-04-02 12:08:31 +01:00
e295a4e45f
remove printf of chunk size.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-04-01 22:58:00 +00:00
4a788a0c23
re-write the DAOS IOR driver to use the Array API
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-03-30 19:11:44 +00:00
21405ed924
Updated MMap module to option parser.
2019-03-27 22:32:59 +00:00
b4fb80a69e
Allowed option parser to reset flags (needed in scriptfiles).
2019-03-27 22:32:38 +00:00
07ca5247d7
Checked proper IOR behavior for -f option.
2019-03-27 20:37:46 +00:00
20e960d020
Support the setting of module-specific options per test.
2019-03-27 20:04:48 +00:00
8b23c50389
update dfs_open for API change to expose chunk size selection
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-03-22 16:53:34 +00:00
39eca1bb08
add DAOS_Access routine to check if a container exists before deleting it.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-03-14 21:17:37 +00:00
d49d345007
Merge pull request #137 from DDNStorage/memoryPerNode
...
Fix segfault with option memoryPerNode
2019-03-14 18:33:24 +00:00
fabb1f4d3b
Fix segfault with option memoryPerNode
...
optarg contains a NULL address, which causes a segfault in
NodeMemoryStringToBytes().
2019-03-14 19:20:18 +01:00
c35f64237f
initialize data and akey checksum to NULL since they are not being used.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-03-14 16:55:54 +00:00
db19141c19
remove usage of GURT's D_GOTO from DFS plugin.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-03-01 20:41:01 +00:00
110153cf3a
mdtest: Enable IME native interface
...
This patch makes mdtest allow IME native interface.
It also defines a generic way to enable mdtest with each
backend.
2019-02-12 15:50:51 +01:00
c6abf85378
aiori-IME: Add support of statfs in IME backend
...
This patch adds the support of statfs in the IME backend.
It uses ime_native_statvfs call to fill the ior_aiori_statfs_t
structure.
2019-02-11 15:00:18 +01:00
29090df283
update DAOS and DFS drivers with new DAOS API changes.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-02-08 18:00:33 +00:00
2c87b5e0f5
Merge remote-tracking branch 'upstream/master' into daos
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Conflicts:
src/aiori.c
src/aiori.h
src/ior.c
src/mdtest-main.c
src/mdtest.c
src/option.c
2019-01-24 00:31:12 +00:00
4c1c73ecae
add malloc check to fix
2019-01-23 16:02:58 -08:00
20bb99e1b7
Potential fix for #127 => Segmentation fault due to memory protection.
2019-01-19 17:00:49 +00:00
334bf73797
remove transaction commit and options to do that.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-01-17 20:58:58 +00:00
d2a0023765
remove epoch usage and rebase with latest DAOS API that switches to transactions.
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-01-07 18:28:59 +00:00
e71cded6c3
Merge pull request #124 from hpc/fix-hdf5
...
Bugfix for HDF5 plugin; readded collective_md.
2019-01-02 10:01:56 -08:00
1dbca5c293
Bugfix for HDF5 plugin; readded collective_md.
2018-12-23 13:07:23 +00:00
ed421bcc7b
Fixes that issue. Hopefully works for all combinations.
2018-12-20 21:08:55 +00:00
7b603bc5a9
Reset stonewall timer to make it work again with running all phases in one execution; i.e. not using multiple runs specifying: -C, -r
2018-12-19 21:37:37 +00:00
4297ccd7f2
Bugfix for stonewall computation.
2018-12-19 20:29:47 +00:00
1f217ae48c
lustre: fix checks for lustre_user.h header
...
Fix the configure check if --with-lustre is specified, but
the linux/lustre/lustre_user.h header is not present. Only
one of the headers needs to be included if both are found.
In some cases, FASYNC is not defined, but forms part of the
O_LOV_DELAY_CREATE value, add a #define in that case.
Fixes #115 .
Fixes: cb88c4c19a831d94b864c49a162e2635730540e5
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
2018-12-14 16:45:43 -07:00
72c487013a
rebase with latest DAOS
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2018-12-04 22:59:15 +00:00
e1158daaf2
Merge pull request #105 from hpc/feature-test-scripts-85
...
Dry-run option to test the configuration / scripts.
2018-12-04 15:06:47 -06:00
5e2e1cc96f
Merge pull request #104 from hpc/feature-hdf5-backend
...
Changed the parser to fix #98 . Does _not_ yet have the HDF5 backend args updated. They were reverted to allow this to make the 3.2 release and should be re-applied as a separate commit.
2018-12-04 14:41:31 -06:00
5c847f552a
Merge pull request #110 from hpc/fix-gnu99-101
...
Moved _XOPEN_SOURCE to config.h; resolves #93
2018-12-04 14:13:16 -06:00
7db8ffd8ab
Reverted the change to HDF5 module back.
2018-12-04 19:16:41 +00:00
8902bc8722
mdtest: fix calculation for barriers
...
Currently, calculation for barriers case looks a bit
suprising, for example, Min stats is from min of
Max value of number of mpi proc, this makes Min
and Max very similar in our testing thus we get
a small Std value too.
I am not a MPI expert, but question is why we
don't use more nature calcuation which calculate
all results from different iterations and MPI proc?
Signed-off-by: Wang Shilong <wshilong@ddn.com>
2018-11-22 19:57:32 +08:00
5a8071fbd8
GNU99 extracted _XOPEN_SOURCE to config.h.
2018-10-28 18:34:22 +00:00
3a4f2ea33e
Merge pull request #101 from glennklockwood/issue93-squashed
...
compile correctly with -std=c99
2018-10-28 18:16:52 +00:00
52182afed6
Merge pull request #108 from hpc/fix-106
...
Fix 106
2018-10-19 11:06:05 +01:00
ea3aeea964
Bugfix initialize the variable to zero for proper API usage.
2018-10-17 16:16:46 +01:00
6b0b1a9830
Fix computation when using branching factor with -n and -i.
2018-10-17 16:04:49 +01:00
093113313d
mdtest: Use GetTimeStamp() instead of MPI_Wtime()
...
IOR uses the GetTimeStamp() wrapper to allow gettimeofday() to be used for timings instead of MPI_Wtime() to calculate throughput. This commits adds the same logic to mdtest timings.
2018-10-17 16:38:35 +02:00
f4afa63ebf
Bugfix #106 mostly.
2018-10-17 13:50:04 +01:00
f7dc084f32
Merge pull request #103 from DDNStorage/ime_mkdir_rmdir
...
IME: add support of mkdir/rmdir into the IME backend
2018-10-16 14:34:26 +01:00
6c5952906b
Dry-run option to test the configuration / scripts.
2018-10-11 19:58:30 +01:00
3b5f4fc002
Bugfix JSON output for multiple processes
2018-10-11 18:59:12 +01:00
fa38cb7992
Bugfix JSON header (allows parsing at line 0).
2018-10-11 18:49:25 +01:00
6f7576aa8a
Changed the parser.
...
The parser now supports concurrent parsing of all plugin options.
Moved HDF5 collective_md option into the backend as an example.
Example: ./src/ior -a dummy --dummy.delay-xfer=50000
2018-10-11 18:31:21 +01:00
Mohamad Chaarawi