Commit Graph

34 Commits (master)

Author SHA1 Message Date
Julian Kunkel a436395570
Support random data generation for memory pattern in utilities. (#348)
* Support random data generation in utilities. Update first 8 byte element in each 4k block on updates to defy dedup.
* Incorporate different packet types into mdtest/md-workbench.
* Integrated utilities memory pattern tools into IOR. Now all tools use the same patterns.
* Added IOR long option for compatibility between IOR and other tools.
* Added new tests for random buffers.
2021-03-18 21:42:50 +01:00
Julian Kunkel 74df77430f
Feature ior gpu #284 (#323)
* Basic support for memory allocation on GPU using CUDA unified memory. Partially addressing #284. IOR support completed.
* Support for GPU alloc in MDTest and MD-Workbench
* Option: support repeated parsing of same option (allows option sharing across modules).
* Checks for gpuDirect
* Integrate gpuDirect options and basic hooks, more testing to be done.
* POSIX: basic gpuDirect implementation working with fake-gpudirect library.
* CUDA allow setting of DeviceID for IOR (not yet MDTest).
* CUDA/GPUDirect Support --with-X=<path>
* Bugfix in option parser for flags that are part of an argument for an option, e.g., -O=1, if 1 is a flag it is wrongly assumed to be a flag.
2021-02-18 10:40:42 +00:00
Julian M. Kunkel 7061b60ed8 Extracted memory pattern comparison, added mem check option to md-workbench. 2021-01-22 14:05:58 +00:00
Julian M. Kunkel 4edb27b41a Remove MPI timer in favor of gettimeofday() to prevent MPI issues. Remove time adjustment as measurements are relative anyway. 2021-01-20 19:38:54 +00:00
Julian M. Kunkel 970c5ef139 Adjust complex tests for changed -z behavior. 2021-01-20 14:57:21 +00:00
Julian M. Kunkel e4120d600d Reduce the dependency to global MPI Communicator 2021-01-20 14:06:05 +00:00
Julian Kunkel a8ab766483
Merge pull request #270 from hpc/fix-offset-integration
Fix offset integration
2020-12-03 12:08:53 +00:00
Julian Kunkel 5d489064d8
Merge pull request #271 from hpc/fix-statvfs
Remove duplicated functionality between mdtest and IOR.
2020-11-13 09:24:24 +00:00
Julian M. Kunkel 277f380139 Remove duplicated functionality between mdtest and IOR.
Refactored the ShowFileSystemSize function.
2020-11-09 16:23:34 +00:00
Julian M. Kunkel a0e5e297d6 Create random for one block (1 segment) only to be repeated. 2020-11-06 09:52:35 +00:00
Julian M. Kunkel ad985af763 Location problem on non Linux systems. 2020-11-06 09:30:59 +00:00
Julian M. Kunkel 145c71f7c3 Trivial cleanup: Extracted debug-related stuff into new header. 2020-06-28 17:16:56 +01:00
Julian Kunkel d5de9bcc77
Merge branch 'master' into master 2020-06-24 09:50:03 +01:00
Julian M. Kunkel 8fa8ef0c02 Call backend statfs() instead of only POSIX-specific variant. 2020-05-31 13:30:31 +01:00
Afrian Jackson 46c5d4e78d Adding in dual mount functionality for NVRAM within nodes 2019-11-05 14:37:54 +00:00
Josh Schwartz 0e952f0f8c Fix #181.
On systems where numTasks is not evenly divisible by 'tasksPerNode' we were
seeing some nodes reading multiple files while others read none after
reordering.

Commonly all nodes have the same number of tasks but there is nothing
requiring that to be the case.  Imagine having 64 tasks running against 4
nodes which can run 20 tasks each.  Here you get three groups of 20 and one
group of 4.  On this sytem nodes running in the group of 4 were previously
getting tasksPerNode of 4 which meant they reordered tasks differently than
the nodes which got tasksPerNode of 20.

The key to fixing this is ensuring that every node reorders tasks the same
way, which means ensuring they all use the same input values.  Obviously on
systems where the number of tasks per node is inconsistent the reordering will
also be inconsistent (some tasks may end up on the same node, or not as far
separated as desired, etc.) but at least this way you'll always end up with a
1:1 reordering.

- Renamed nodes/nodeCount to numNodes
- Renamed tasksPerNode to numTasksOnNode0
- Ensured that numTasksOnNode0 will always have the same value regardless of
  which node you're on
- Removed inconsistently used globals numTasksWorld and tasksPerNode and
  replaced with per-test params equivalents
- Added utility functions for setting these values:
  - numNodes -> GetNumNodes
  - numTasks -> GetNumTasks
  - numTasksOnNode0 -> GetNumNodesOnTask0
- Improved MPI_VERSION < 3 logic for GetNumNodes so it works when numTasks is
  not evenly divisible by numTasksOnNode0
- Left 'nodes' and 'tasksPerNode' in output alone to not break compatibility
- Allowed command-line params to override numTasks, numNodes, and
  numTasksOnNode0 but default to using the MPI-calculated values
2019-08-30 16:45:03 -06:00
John Bent 3890b71b54 Fixed issues and followed suggestions from Glenn's review of the PR 2019-08-01 09:42:03 +09:00
John Bent a3c37808da Made FAIL take variable args so we can pass printf like args to it 2019-07-28 11:17:11 -06:00
John Bent 524d053be1 Making shift work in mdtest as well as it works in IOR and on a per-node basis.
Also added printing the nodemap so we can check the allocation.
2019-07-26 08:55:24 -06:00
Julian M. Kunkel 20e960d020 Support the setting of module-specific options per test. 2019-03-27 20:04:48 +00:00
Julian M. Kunkel 8525f9734c Extraction of output completed. 2018-07-08 13:50:20 +01:00
Julian M. Kunkel 02883d4a63 Moved most output to ior-output.c 2018-07-08 13:38:05 +01:00
Julian M. Kunkel 0f7a1f14b9 Support IO redirect from main() 2018-07-08 13:12:18 +01:00
Julian M. Kunkel ed5787811c Extracted max pathlen variable. 2018-07-07 22:39:14 +01:00
Julian M. Kunkel 48c17b4d81 Extracted (nearly identical) function DelaySeconds. 2018-07-07 22:26:57 +01:00
Julian M. Kunkel 6f8d2e8845 Some simplification / unification between IOR and MDTest.
MDTest needs more refactoring to be really maintainable.
2018-07-07 22:19:42 +01:00
Julian M. Kunkel 812b798f05 IOR allows to store actual performaned stonewalling count into a status file. 2018-07-07 14:01:11 +01:00
Julian M. Kunkel a6bfa0f94b MDTest library option.
Moved core functionality / variables used by both implementations to utilities
2018-07-07 10:42:48 +01:00
Jeff Inman 37738dab26 Numerous changes to file-modes, small build-tweaks, and a tweak to aiori-S3.c
(Only rank-0 should create the bucket, if it doesn't already exist.)

Prepping this for a push (as an experimental S3 build) to github.
2015-05-19 09:36:28 -06:00
Jeffrey Thornton Inman ca801fe0ec First-cut at PLFS support, integrating Brett K's code.
Builds, but not running yet.  Requires 'configure ... --with-plfs', so it seems okay to leave this in as an optional build, pending further work.
2014-09-18 11:20:37 -06:00
Christopher J. Morrone 8e06f07b28 Add header to set code preferences for vi and emacs. 2012-01-08 19:43:41 -08:00
Christopher J. Morrone 349f747ea5 Remove undocumented options. 2012-01-08 19:43:41 -08:00
Christopher J. Morrone 4084e14d02 Add memoryPerNode option 2012-01-08 19:36:43 -08:00
Christopher J. Morrone a60ad0b088 Impose proper file scoping and header usage.
Clean up the header files to only contain those things that
need to be shared between .c files.

Functions that are not shared are now declared static to
make their file scope explicit.  Functions that ARE shared
are declared in appropriate headers.

I am not going to claim that I caugh everything, but at
least it is a good start.
2011-11-11 17:30:20 -08:00