Commit Graph

214 Commits (dcacb341e4214441e6de4a18891a323e84caedce)

Author SHA1 Message Date
Julian M. Kunkel fa5b24f2aa Bugfix user docu for ranks. Bugfix missmatching barriers. 2021-02-08 13:28:13 +00:00
Julian M. Kunkel 7b69fc1500 Free testcom after finalize. 2021-01-29 14:00:17 +00:00
Julian M. Kunkel 351fd4dfb9 Move initialization of TestComm before aiori->initialize() 2021-01-29 09:08:54 +00:00
Julian M. Kunkel 4edb27b41a Remove MPI timer in favor of gettimeofday() to prevent MPI issues. Remove time adjustment as measurements are relative anyway. 2021-01-20 19:38:54 +00:00
Julian Kunkel d5f5cf974d
Merge pull request #318 from hpc/fix-comm
Refactoring: remove global comm_world communicator from utilities.c
2021-01-20 18:25:53 +00:00
Julian Kunkel 9e0a8c44d5
IOR: move verbose output of detailed errors to verbosity level 1. (#315)
IOR by default outputs the numbers of errors.
Reason: The amount of error messages can be overwhelming, particularly in a parallel program (Gigabytes...).
One -v increases the verbosity level to provide the extra details without adding too many other messages.
2021-01-20 15:06:45 +00:00
Julian M. Kunkel f345a78046 Removed dependency. 2021-01-20 15:00:33 +00:00
Julian M. Kunkel 970c5ef139 Adjust complex tests for changed -z behavior. 2021-01-20 14:57:21 +00:00
Julian M. Kunkel 40c6d97e72 Replaced MPI_COMM_WORLD where needed with testComm. 2021-01-20 14:35:06 +00:00
Julian M. Kunkel e4120d600d Reduce the dependency to global MPI Communicator 2021-01-20 14:06:05 +00:00
Julian Kunkel 58fbefbd33
Bugfix IOR offsetting (fix before was incomplete). (#317) 2021-01-19 23:19:01 +00:00
Julian Kunkel 19ad73568a
Bugfix ior stonewalling case. (#316)
* Bugfix ior stonewalling case.
2021-01-19 22:28:34 +00:00
Julian Kunkel 8de13884a7
HDFS module ported to current AIORI API and improved (#302)
* HDFS module ported to current AIORI API and extended
* Added instructions how to build and run with a HDFS.
* Replaced read with pread to support offsets.
* Implemented HDFS MDTest API
* Improved sync semantics
2020-12-23 11:51:31 +00:00
Julian M. Kunkel 69e006dd89 IOR clarify return API. 2020-12-03 21:07:45 +00:00
Julian M. Kunkel 073da08292 IOR: print overwrite check only if a write operation is requested. 2020-12-03 20:54:51 +00:00
Julian Kunkel a8ab766483
Merge pull request #270 from hpc/fix-offset-integration
Fix offset integration
2020-12-03 12:08:53 +00:00
Julian M. Kunkel f71a144b19 IOR raise warning for existing file once if not filePerProc. 2020-12-02 10:14:28 +00:00
Julian M. Kunkel da03ae5c9f IOR add warning if file exists #273. To stop on warnings, use existing flag --warningAsErrors 2020-12-01 18:24:09 +00:00
Julian M. Kunkel 9dcf9f79e4 Moved barriers to right location. 2020-12-01 14:47:57 +00:00
Julian M. Kunkel 546eb05d15 Rudimentary support for stonewall added, addresses #261. 2020-12-01 14:45:07 +00:00
Julian M. Kunkel 5825dbae35 Provide new option randomPrefill for random (-z access only) that prefill the file with the specified blocksize, e.g., 2m. See issue #270.
This option works so far only without stonewalling!
2020-12-01 13:58:40 +00:00
Julian M. Kunkel a59e98d7a6 Add error to indicate changed behavior to users 2020-11-06 10:39:16 +00:00
Julian M. Kunkel af0753d8ad Random: last modifications. 2020-11-06 10:27:11 +00:00
Julian M. Kunkel a0e5e297d6 Create random for one block (1 segment) only to be repeated. 2020-11-06 09:52:35 +00:00
Julian M. Kunkel 3ee3e9ad5c Integrated sequential code into the benchmark main loop.
A bit ugly but shows that the code logic hasn't changed.
2020-11-05 14:00:09 +00:00
Julian Kunkel fb66e77072
Enable random seed to be stored. (#268) 2020-11-04 13:47:35 +00:00
Mohamad Chaarawi cb397242f9
DAOS backend cleanup (#266)
- remove legacy DAOS driver & update Readme.
- update configure options to remove cart requirement
- Optimize DFS file get_size

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2020-11-03 10:01:09 +00:00
donko dab62b57c8
fix JSON format, the end of Array object (#264) 2020-10-28 09:38:17 +00:00
Julian Kunkel bd76b45ef9
Integrated comparison checks into CompareData() which prevents additi… (#254)
* Integrated comparison checks into CompareData() which prevents additional buffer comparisons at the expense of some computation #249.

* IOR: Added code documentation to warn people for changing the creation pattern but not the verification pattern.
2020-09-28 20:09:48 +01:00
Julian Kunkel af2429b47e
Fix s3-4c implementation (#247)
* Ported S3-4c version to current AIORI interface.
* S3-4c: add crypto dependency
* S3: Store username/host in options.
2020-08-13 16:25:36 +01:00
Julian Kunkel 4e2d179052
Detection of system features for number of sockets #245 (#246)
* Configure checks for architecture-specific functions to detect nr of sockets. #245
2020-08-03 12:30:21 +01:00
Julian M. Kunkel 8ca388fc78 Merge branch 'feature-cleanup-aiori' 2020-07-21 09:26:03 +01:00
Karsten Weiss f280123d0b Spelling fixes (found by codespell) 2020-07-03 09:16:30 +02:00
Julian M. Kunkel 87c9906d70 Error checking for NULL create/opens 2020-06-30 19:41:49 +01:00
Julian M. Kunkel 82417128cd Extracted file size routine that is replicated in each module into IOR removing the MPI dependency from AIORI. 2020-06-30 12:33:56 +01:00
Julian M. Kunkel 3395fc621c Moved GetFileSize into CheckFileSize, starting to simplify get_file_size (too much replication in modules). 2020-06-30 12:03:05 +01:00
Julian M. Kunkel 38b1752bda JSON issue with results. Fix for #233. 2020-06-25 14:37:52 +01:00
Julian M. Kunkel 0bffd14de7 Added --warningAsErrors option to IOR and MDTest and refactored WARNINGs in IOR. #174 2020-06-24 11:13:12 +01:00
Julian Kunkel d5de9bcc77
Merge branch 'master' into master 2020-06-24 09:50:03 +01:00
Julian Kunkel 6a88b37171
Merge branch 'master' into fix-194 2020-06-24 09:43:37 +01:00
Mohamad Chaarawi b4a54c34e8 Merge branch 'master' into fix-modules
Conflicts:
	src/aiori-DAOS.c
	src/aiori-DFS.c
2020-06-11 19:55:23 +00:00
Julian M. Kunkel eb2cd27fac Removed offset from IOR structure. 2020-06-10 17:47:07 +01:00
Mohamad Chaarawi 3623f3b8bd DFS: allow multiple IO phases when ior_run() is called
- init/fini can be made multiple times
- reset tunables on fini so they can change on a re-init

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2020-05-31 16:20:58 +00:00
Julian M. Kunkel 8fa8ef0c02 Call backend statfs() instead of only POSIX-specific variant. 2020-05-31 13:30:31 +01:00
Julian M. Kunkel 5663593919 Moved generic IOR information to "hint" structure. Backends should only use the "hints" if set. 2020-05-31 12:50:03 +01:00
Julian M. Kunkel 8098c3740a Refactored fd into aiori_fd_t for type safety. 2020-05-31 12:12:49 +01:00
Julian M. Kunkel 294b8891e6 Created dummy option type to provide type safety. Fixed various call issues for options. 2020-05-31 11:50:15 +01:00
Julian M. Kunkel e91b79cea0 Moved Lustre/GPFS/BeeGFS options into POSIX backend. Needs testing. Additional minor fixes. 2020-05-30 20:09:37 +01:00
Julian M. Kunkel 930ccdc68d Updated HDF5 to new module specification. 2020-05-30 19:01:20 +01:00
Julian M. Kunkel 1890aaaeb0 Started to move IOR specific knowledge out of AIORI.
For now, and compatibility, IOR options can still be set/internally accessed using the backends init_xfer_options.
This should be removed in the long run to strip away this dependency.
2020-05-30 18:19:48 +01:00