Commit Graph

737 Commits (e579531970551fe6358598b4d5d52db4653d8b9e)

Author SHA1 Message Date
Mohamad Chaarawi d926bf1c90 update configure.ac to look under lib64 for DAOS libs
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-11-06 23:10:45 +00:00
Adrian Jackson 339bb200c8 Fixing configure so pmdk backend not built without --with-pmdk being specified 2019-11-05 17:44:26 +00:00
Adrian Jackson c24b2f156a
Merge pull request #2 from hpc/master
Merging in latest master changes
2019-11-05 15:00:47 +00:00
Afrian Jackson b4c85798bd Tidying up PMDK backend 2019-11-05 14:39:31 +00:00
Afrian Jackson 46c5d4e78d Adding in dual mount functionality for NVRAM within nodes 2019-11-05 14:37:54 +00:00
Glenn K. Lockwood bdfc43125a
Merge pull request #198 from daos-stack/daos-devel
fix config error with older autoconf/automake versions.
2019-11-04 15:56:44 -08:00
Mohamad Chaarawi 847b8ad408 fix config error with older autocont/automake versions.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-11-04 22:57:47 +00:00
Afrian Jackson 817603f5d1 Updating copyright statement 2019-10-31 15:31:28 +00:00
Afrian Jackson fa25c31994 Adding PMDK functionality 2019-10-31 15:29:09 +00:00
Glenn K. Lockwood 3c7ac2df23
Merge pull request #196 from daos-stack/daos-devel
update DFS driver with new DAOS API changes.
2019-10-28 16:56:29 -07:00
Glenn K. Lockwood 4409082064
Merge pull request #188 from pkoutoupis/feature-global_default_dir_layout
Enable global default dir layout for subdirs in Lustre
2019-10-28 10:43:56 -07:00
Mohamad Chaarawi 92180e022d update DFS read/write APIs with new DAOS API changes
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-10-28 16:42:44 +00:00
Adrian Jackson 3d39626e86
Merge pull request #1 from hpc/master
Merge in latest master updates
2019-10-28 14:07:10 +00:00
jschwartz-cray ccba3b11e5
Merge branch 'master' into fix-194 2019-10-23 16:46:00 -06:00
Josh Schwartz 61333af822 Fix #194.
On previous versions of IOR (and most Linux CLI applications in general)
multiple flags can be combined such as -vvv to get verbose=3 rather than
having to specify each separately as -v -v -v.  This patch fixes this behavior.

It also fixes issues around handling of the verbose flag where the
output would never be printed as we were comparing verbose < VERBOSE_0
where verbose defaults to 0 and can only increase.
2019-10-23 16:17:37 -06:00
Julian Kunkel b334c3f6fd
Merge pull request #192 from daos-stack/daos-devel
fix -R option to do proper data verification
2019-10-16 16:20:43 +01:00
Mohamad Chaarawi dc82a1bf2f ior -R should memset the buffer being read, otherwise a read that does nothing
will report success in the data verification phase.

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-10-16 14:54:00 +00:00
Julian M. Kunkel a69a5916cf Hotfix for ior -F -u 2019-10-16 09:42:10 +01:00
Glenn K. Lockwood 7f4f4af215
Merge pull request #151 from haplav/patch-1
Update tutorial.rst
2019-10-02 13:10:16 -07:00
Glenn K. Lockwood f40b0744ff
Merge pull request #187 from daos-stack/daos-devel
IOR: Add latency and IOPS numbers to each iteration.
2019-10-02 13:09:26 -07:00
Petros Koutoupis b3ecd756ce Implemented AC_CHECK_HEADERS fix in style of #190 2019-09-30 04:25:59 -07:00
Mohamad Chaarawi 2aea04c8cb fix some compiler warnings.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-09-21 22:43:20 +00:00
Mohamad Chaarawi fca0a62ccb update to new DAOS API for creating dfs containers.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-09-21 22:05:54 +00:00
Julian Kunkel 604598ab2f
Merge pull request #182 from jschwartz-cray/fix-181
Fix #181.
2019-09-19 16:53:41 +01:00
Mohamad Chaarawi 0d0df855e6 update user guide with IOPS and latency numbers for each iteration.
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-09-18 19:50:43 +00:00
Mohamad Chaarawi 12284ae04a Add latency and iops numbers to each iteration.
- Latency reported is computed by taking the average latency of
  all ops from a single task, then taking the minimum of that
  between all tasks.

- IOPS is computed by taking the total number of ops across all
  tasks divided by the total access time to execute those ops.

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-09-10 18:49:32 +00:00
Julian Kunkel bcd449c360
Merge pull request #186 from daos-stack/daos-devel
add break that was accidentally removed earlier
2019-09-10 15:14:11 +01:00
Mohamad Chaarawi c58ba8ffb5 add break that was accidentally removed
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-09-10 14:11:55 +00:00
Julian Kunkel a301baecce
Merge pull request #185 from daos-stack/daos-devel
Add DAOS based drivers (DAOS Array & DFS)
2019-09-10 15:01:26 +01:00
Mohamad Chaarawi fe9d76ddf3 fix segfault when no API is specified to mdtest
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-09-09 22:03:55 +00:00
Mohamad Chaarawi 73dbda09c6 remove printf added
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-09-09 20:19:42 +00:00
Mohamad Chaarawi 5622aabf46 missed Makefile.am update
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-09-09 20:14:12 +00:00
Mohamad Chaarawi d332d586bf remove uneeded code and doc
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-09-09 20:07:55 +00:00
Mohamad Chaarawi e9dd5fe15e Merge branch 'master' into daos-devel
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>

Conflicts:
	configure.ac
	src/Makefile.am
	src/aiori-MPIIO.c
	src/aiori.c
	src/aiori.h
	src/ior.c
	src/mdtest.c
	src/option.c
2019-09-09 19:55:56 +00:00
Mohamad Chaarawi 57a16ddda8 - update debugging error checking in DFS.
- remove usleep() before daos_fini()
- fix README

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-09-09 19:14:35 +00:00
Julian Kunkel 0921556456
Merge pull request #184 from hpc/feature-aiori
Feature aiori
2019-09-09 15:20:43 +01:00
Petros Koutoupis 36d13d3b97 Enable global default dir layout for subdirs in Lustre 2019-09-02 22:26:12 -07:00
Julian M. Kunkel c83edfe39b Extracted check function into aiori. #24. #177 2019-09-01 15:59:52 +01:00
Julian Kunkel ced9b0183f
Merge pull request #172 from hpc/fix-168
Fix #168
2019-09-01 15:51:44 +01:00
Julian Kunkel 151102ff8f
Merge pull request #176 from otatebe/master
abstract I/O interface for Gfarm file system
2019-09-01 15:51:16 +01:00
Julian M. Kunkel e3db1759b2 Moded sync() to aiori backend. 2019-09-01 15:47:42 +01:00
Julian M. Kunkel 6de1ea3176 Merge branch 'feature-sync' into feature-aiori 2019-09-01 15:39:49 +01:00
Julian M. Kunkel 60a641f911 sync help corrected. 2019-09-01 15:29:12 +01:00
Julian M. Kunkel 10d3db1dc8 MDTest: fixing the memset() to account for the number of iterations. 2019-08-31 17:28:08 +01:00
Julian Kunkel 47695aea49
Merge pull request #180 from jschwartz-cray/fix-179
Fix #179.
2019-08-31 10:32:41 +01:00
Julian Kunkel 11fa33fc9f
Merge pull request #178 from jschwartz-cray/more-debug
Added some extra debug.
2019-08-31 10:30:33 +01:00
Josh Schwartz 0e952f0f8c Fix #181.
On systems where numTasks is not evenly divisible by 'tasksPerNode' we were
seeing some nodes reading multiple files while others read none after
reordering.

Commonly all nodes have the same number of tasks but there is nothing
requiring that to be the case.  Imagine having 64 tasks running against 4
nodes which can run 20 tasks each.  Here you get three groups of 20 and one
group of 4.  On this sytem nodes running in the group of 4 were previously
getting tasksPerNode of 4 which meant they reordered tasks differently than
the nodes which got tasksPerNode of 20.

The key to fixing this is ensuring that every node reorders tasks the same
way, which means ensuring they all use the same input values.  Obviously on
systems where the number of tasks per node is inconsistent the reordering will
also be inconsistent (some tasks may end up on the same node, or not as far
separated as desired, etc.) but at least this way you'll always end up with a
1:1 reordering.

- Renamed nodes/nodeCount to numNodes
- Renamed tasksPerNode to numTasksOnNode0
- Ensured that numTasksOnNode0 will always have the same value regardless of
  which node you're on
- Removed inconsistently used globals numTasksWorld and tasksPerNode and
  replaced with per-test params equivalents
- Added utility functions for setting these values:
  - numNodes -> GetNumNodes
  - numTasks -> GetNumTasks
  - numTasksOnNode0 -> GetNumNodesOnTask0
- Improved MPI_VERSION < 3 logic for GetNumNodes so it works when numTasks is
  not evenly divisible by numTasksOnNode0
- Left 'nodes' and 'tasksPerNode' in output alone to not break compatibility
- Allowed command-line params to override numTasks, numNodes, and
  numTasksOnNode0 but default to using the MPI-calculated values
2019-08-30 16:45:03 -06:00
Josh Schwartz 4c3d96bfed Fix #179.
-u (uniqueDir) will once again use the full file path specified by the
client instead of truncating it.  This was caused by a broken sprintf
which was trying to read/write overlapping buffers.

From the glibc sprintf() documentation:
"The behavior of this function is undefined if copying takes place
between objects that overlap"
2019-08-30 15:31:23 -06:00
Josh Schwartz 0bd979637e Added some extra debug including ERRF, WARNF, and MPI_CHECKF format string macros. 2019-08-30 15:11:25 -06:00
Mohamad Chaarawi 0b809b36e2 fix README_DAOS for DFS plugin
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
2019-08-30 20:35:15 +00:00