Julian Kunkel
11a81e42ca
Merge pull request #360 from efajardo/main
...
Adding number of tasks to the CSV output
2021-04-21 08:47:38 +02:00
efajardo
39fcb5970d
Adding number of tasks to the CSV output
2021-04-20 15:38:04 -07:00
Julian Kunkel
0a4d58576d
Merge pull request #359 from hpc/fix-347
...
IOR: add several sanity checks for stonewalling #345
2021-04-14 08:43:00 +02:00
Julian M. Kunkel
0410a38e98
IOR: add several sanity checks for stonewalling #345
2021-04-13 11:01:30 +02:00
Julian Kunkel
a90d414a30
Merge pull request #358 from hpc/fix-357
...
MDTest, support flag to show per rank performance: #357
2021-04-07 18:28:20 +02:00
Julian M. Kunkel
8fc7ab5c9c
MDTest, support flag to show per rank performance: #357
...
Also fix a potential issue for the calculation if a test is not run (number of items = 0).
2021-04-01 17:23:10 +02:00
Julian Kunkel
0ed2dc3187
Merge pull request #355 from hpc/fix-354
...
Bugfix IOR writecheck #354
2021-03-29 21:08:31 +02:00
Julian M. Kunkel
a490ac8ba6
Bugfix IOR writecheck #354
2021-03-29 20:11:10 +02:00
Julian Kunkel
e8409edc97
Merge pull request #351 from Kawashima-Azumi/prologues
...
Support prologue and epilogue for benchmark phases
2021-03-21 18:54:29 +01:00
Julian Kunkel
1b77361e09
Change short option to long option
2021-03-21 18:40:26 +01:00
GUAN Xin
2250e6c5f9
Prologue and epilogue support for benchmark phases
2021-03-20 23:50:04 +08:00
GUAN Xin
84a169c7c2
Clean up timer manipulation
2021-03-20 23:34:43 +08:00
Julian Kunkel
a436395570
Support random data generation for memory pattern in utilities. ( #348 )
...
* Support random data generation in utilities. Update first 8 byte element in each 4k block on updates to defy dedup.
* Incorporate different packet types into mdtest/md-workbench.
* Integrated utilities memory pattern tools into IOR. Now all tools use the same patterns.
* Added IOR long option for compatibility between IOR and other tools.
* Added new tests for random buffers.
2021-03-18 21:42:50 +01:00
Julian Kunkel
3be3cfb274
Fix MPI-IO strided access. ( #347 )
...
* Remove MPI-IO malloc.
* Bugfix for MPI-IO segmented/view.
2021-03-17 00:24:23 +01:00
Mohamad Chaarawi
df5fa556c8
reset testcomm to world_comm since it can be used in finalize callbacks. ( #344 )
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2021-03-13 18:29:40 +01:00
Peter Steinbach
61f36f0c48
converting to sphinx code snippets ( #343 )
...
* converting to sphinx code snippets
* more rigorous code highlighting
2021-03-09 17:07:19 +00:00
Mohamad Chaarawi
f239b74d83
free MPI comm and group for each iteration they are created in. ( #342 )
...
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2021-03-08 18:46:58 +00:00
Rob Latham
4bad282932
Make noncontiguous I/O work again ( #341 )
2021-03-06 09:53:39 +00:00
Julian Kunkel
ad3bf97304
Fixed some issues in build system detected using Clang. ( #339 )
2021-02-28 11:45:32 +00:00
Jean-Yves VET
0829ac5a18
Fix detection of option with empty value ( #338 )
...
Context: Options with values are not properly parsed.
If the value is not set, a bug makes appear that 0 was
provided.
This patch fixes options parsing (issue #337 ). An error
is reported if 'backend.option=' argument is provided.
2021-02-19 12:30:21 +00:00
Julian Kunkel
74df77430f
Feature ior gpu #284 ( #323 )
...
* Basic support for memory allocation on GPU using CUDA unified memory. Partially addressing #284 . IOR support completed.
* Support for GPU alloc in MDTest and MD-Workbench
* Option: support repeated parsing of same option (allows option sharing across modules).
* Checks for gpuDirect
* Integrate gpuDirect options and basic hooks, more testing to be done.
* POSIX: basic gpuDirect implementation working with fake-gpudirect library.
* CUDA allow setting of DeviceID for IOR (not yet MDTest).
* CUDA/GPUDirect Support --with-X=<path>
* Bugfix in option parser for flags that are part of an argument for an option, e.g., -O=1, if 1 is a flag it is wrongly assumed to be a flag.
2021-02-18 10:40:42 +00:00
Julian Kunkel
e78613d62d
Improve and fix the reporting in MDTest. ( #336 )
...
* Improve and fix the reporting in MDTest. Reporting per rank now outputs the performance of the individual rank (before the barrier), the iteration throughput includes the barrier time (if not -B=0 is set). Before, it was the time after the barrier.
* Clarify the computation of the results.
* MDTest improve CSV output to include aggregated result.
2021-02-18 09:53:07 +00:00
Glenn K. Lockwood
4038ebdb21
fix residual references to old main branch name
2021-02-17 12:36:20 -08:00
Julian M. Kunkel
a198b0404c
Bugfix MDTest: only rank 0 shall print errors.
2021-02-11 15:50:48 +00:00
Julian Kunkel
106eebecb3
MDTest allow storing information per rank for later analysis ( #335 )
...
* MDTest allow storing information per rank for later analysis when using the --saveRankPerformanceDetails=<FILE> option
* MDTest: refactored calculation of results, added time_before_executing a barrier.
2021-02-11 09:57:22 +00:00
Julian Kunkel
b5963380ae
Feature: IOR rank details in CSV file ( #334 )
...
* IOR: Store individual rank results into a CSV file #333
Example usage: ior -O saveRankPerformanceDetailsCSV=test.csv
2021-02-09 17:54:14 +00:00
Julian Kunkel
dcacb341e4
Merge pull request #330 from hpc/fix-329
...
Bugfix user docu for ranks #329 . Bugfix missmatching barriers.
2021-02-08 13:54:06 +00:00
Julian M. Kunkel
fa5b24f2aa
Bugfix user docu for ranks. Bugfix missmatching barriers.
2021-02-08 13:28:13 +00:00
Julian Kunkel
f15956247f
Merge pull request #327 from hpc/fix-testing
...
Testing improvements.
2021-02-05 11:36:11 +00:00
Julian M. Kunkel
bdde24bd91
Save testlogs into directory per type.
2021-02-05 11:24:53 +00:00
Julian M. Kunkel
b4db470459
Testing improvements.
...
Uploaded new mdtest patterns. Make them invariant to line nrs.
Added md-workbench tests.
Added --oversubscribe option if MPI understands it (needed for some systems)
2021-02-05 11:17:15 +00:00
Julian Kunkel
e5e1053405
Merge pull request #326 from hpc/fix-mdworkbench-check
...
MDWorkbench: Fix new verification option
2021-02-05 11:14:26 +00:00
Julian M. Kunkel
265cdb2a42
MDWorkbench: Fix new verification option
2021-02-05 10:49:35 +00:00
Julian Kunkel
3f465c61ff
Merge pull request #325 from hpc/fix-init
...
Move initialization of TestComm before aiori->initialize()
2021-01-29 14:19:57 +00:00
Julian M. Kunkel
7b69fc1500
Free testcom after finalize.
2021-01-29 14:00:17 +00:00
Julian M. Kunkel
351fd4dfb9
Move initialization of TestComm before aiori->initialize()
2021-01-29 09:08:54 +00:00
Julian Kunkel
4c6f33f0f9
Merge pull request #322 from hpc/feature-mdtest-randnum
...
MDTest data verification improvements.
2021-01-22 15:38:44 +00:00
Julian M. Kunkel
0a066d8285
MD-Worbench: add -G option to set parameter.
2021-01-22 14:38:36 +00:00
Julian M. Kunkel
7061b60ed8
Extracted memory pattern comparison, added mem check option to md-workbench.
2021-01-22 14:05:58 +00:00
Julian M. Kunkel
effcb4131c
MDTest: add randomness to buffers to defend dedup efforts. Allow to set random offset externally.
2021-01-22 12:24:33 +00:00
Julian M. Kunkel
3af915aae1
MDTest data verification. Fixed bug: added missing reduce. Add rank into I/O buffer to make individual files unique.
2021-01-22 12:10:09 +00:00
Julian Kunkel
310fd37427
Feature mdtest dir rename #306 ( #311 )
...
* MDTest: Support for directory renaming.
* Refactored some MDTest variables from hardcoded numbers to ENUM symbols.
2021-01-21 14:10:23 +00:00
Julian Kunkel
4ba0eda38f
Merge pull request #321 from hpc/fix-posix-warn
...
AIORI POSIX use internal debug macros, include errno in msg.
2021-01-21 11:38:55 +00:00
Julian Kunkel
6e741054ac
Merge pull request #320 from hpc/fix-timer
...
Remove MPI timer in favor of gettimeofday() to prevent MPI timing issues.
2021-01-21 11:38:29 +00:00
Julian Kunkel
a2afcf7740
Merge pull request #319 from hpc/fix-309
...
MDTest replace fatals with warnings making the API more robust while …
2021-01-21 11:23:43 +00:00
Julian M. Kunkel
852ee3e40e
AIORI POSIX use internal debug macros, include errno in msg.
2021-01-21 11:06:13 +00:00
Julian M. Kunkel
4edb27b41a
Remove MPI timer in favor of gettimeofday() to prevent MPI issues. Remove time adjustment as measurements are relative anyway.
2021-01-20 19:38:54 +00:00
Julian Kunkel
d5f5cf974d
Merge pull request #318 from hpc/fix-comm
...
Refactoring: remove global comm_world communicator from utilities.c
2021-01-20 18:25:53 +00:00
Julian M. Kunkel
eb111f4279
MDTest replace fatals with warnings making the API more robust while it resolves #309 .
...
Example execution (can be done multiple times without cleanup):
$ mpiexec -np 2 ./src/mdtest -d test@test -u -n 100 -C
2021-01-20 15:19:38 +00:00
Julian Kunkel
9e0a8c44d5
IOR: move verbose output of detailed errors to verbosity level 1. ( #315 )
...
IOR by default outputs the numbers of errors.
Reason: The amount of error messages can be overwhelming, particularly in a parallel program (Gigabytes...).
One -v increases the verbosity level to provide the extra details without adding too many other messages.
2021-01-20 15:06:45 +00:00