/usr/share/doc/libfftw3-doc/html/Combining-MPI-and-Threads.html is in libfftw3-doc 3.3.4-2.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 | <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<!-- This manual is for FFTW
(version 3.3.4, 20 September 2013).
Copyright (C) 2003 Matteo Frigo.
Copyright (C) 2003 Massachusetts Institute of Technology.
Permission is granted to make and distribute verbatim copies of this
manual provided the copyright notice and this permission notice are
preserved on all copies.
Permission is granted to copy and distribute modified versions of this
manual under the conditions for verbatim copying, provided that the
entire resulting derived work is distributed under the terms of a
permission notice identical to this one.
Permission is granted to copy and distribute translations of this manual
into another language, under the above conditions for modified versions,
except that this permission notice may be stated in a translation
approved by the Free Software Foundation. -->
<!-- Created by GNU Texinfo 5.2, http://www.gnu.org/software/texinfo/ -->
<head>
<title>FFTW 3.3.4: Combining MPI and Threads</title>
<meta name="description" content="FFTW 3.3.4: Combining MPI and Threads">
<meta name="keywords" content="FFTW 3.3.4: Combining MPI and Threads">
<meta name="resource-type" content="document">
<meta name="distribution" content="global">
<meta name="Generator" content="makeinfo">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<link href="index.html#Top" rel="start" title="Top">
<link href="Concept-Index.html#Concept-Index" rel="index" title="Concept Index">
<link href="index.html#SEC_Contents" rel="contents" title="Table of Contents">
<link href="Distributed_002dmemory-FFTW-with-MPI.html#Distributed_002dmemory-FFTW-with-MPI" rel="up" title="Distributed-memory FFTW with MPI">
<link href="FFTW-MPI-Reference.html#FFTW-MPI-Reference" rel="next" title="FFTW MPI Reference">
<link href="FFTW-MPI-Performance-Tips.html#FFTW-MPI-Performance-Tips" rel="prev" title="FFTW MPI Performance Tips">
<style type="text/css">
<!--
a.summary-letter {text-decoration: none}
blockquote.smallquotation {font-size: smaller}
div.display {margin-left: 3.2em}
div.example {margin-left: 3.2em}
div.indentedblock {margin-left: 3.2em}
div.lisp {margin-left: 3.2em}
div.smalldisplay {margin-left: 3.2em}
div.smallexample {margin-left: 3.2em}
div.smallindentedblock {margin-left: 3.2em; font-size: smaller}
div.smalllisp {margin-left: 3.2em}
kbd {font-style:oblique}
pre.display {font-family: inherit}
pre.format {font-family: inherit}
pre.menu-comment {font-family: serif}
pre.menu-preformatted {font-family: serif}
pre.smalldisplay {font-family: inherit; font-size: smaller}
pre.smallexample {font-size: smaller}
pre.smallformat {font-family: inherit; font-size: smaller}
pre.smalllisp {font-size: smaller}
span.nocodebreak {white-space:nowrap}
span.nolinebreak {white-space:nowrap}
span.roman {font-family:serif; font-weight:normal}
span.sansserif {font-family:sans-serif; font-weight:normal}
ul.no-bullet {list-style: none}
-->
</style>
</head>
<body lang="en" bgcolor="#FFFFFF" text="#000000" link="#0000FF" vlink="#800080" alink="#FF0000">
<a name="Combining-MPI-and-Threads"></a>
<div class="header">
<p>
Next: <a href="FFTW-MPI-Reference.html#FFTW-MPI-Reference" accesskey="n" rel="next">FFTW MPI Reference</a>, Previous: <a href="FFTW-MPI-Performance-Tips.html#FFTW-MPI-Performance-Tips" accesskey="p" rel="prev">FFTW MPI Performance Tips</a>, Up: <a href="Distributed_002dmemory-FFTW-with-MPI.html#Distributed_002dmemory-FFTW-with-MPI" accesskey="u" rel="up">Distributed-memory FFTW with MPI</a> [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Concept-Index.html#Concept-Index" title="Index" rel="index">Index</a>]</p>
</div>
<hr>
<a name="Combining-MPI-and-Threads-1"></a>
<h3 class="section">6.11 Combining MPI and Threads</h3>
<a name="index-threads-2"></a>
<p>In certain cases, it may be advantageous to combine MPI
(distributed-memory) and threads (shared-memory) parallelization.
FFTW supports this, with certain caveats. For example, if you have a
cluster of 4-processor shared-memory nodes, you may want to use
threads within the nodes and MPI between the nodes, instead of MPI for
all parallelization.
</p>
<p>In particular, it is possible to seamlessly combine the MPI FFTW
routines with the multi-threaded FFTW routines (see <a href="Multi_002dthreaded-FFTW.html#Multi_002dthreaded-FFTW">Multi-threaded FFTW</a>). However, some care must be taken in the initialization code,
which should look something like this:
</p>
<div class="example">
<pre class="example">int threads_ok;
int main(int argc, char **argv)
{
int provided;
MPI_Init_thread(&argc, &argv, MPI_THREAD_FUNNELED, &provided);
threads_ok = provided >= MPI_THREAD_FUNNELED;
if (threads_ok) threads_ok = fftw_init_threads();
fftw_mpi_init();
...
if (threads_ok) fftw_plan_with_nthreads(...);
...
MPI_Finalize();
}
</pre></div>
<a name="index-fftw_005fmpi_005finit-3"></a>
<a name="index-fftw_005finit_005fthreads-2"></a>
<a name="index-fftw_005fplan_005fwith_005fnthreads-1"></a>
<p>First, note that instead of calling <code>MPI_Init</code>, you should call
<code>MPI_Init_threads</code>, which is the initialization routine defined
by the MPI-2 standard to indicate to MPI that your program will be
multithreaded. We pass <code>MPI_THREAD_FUNNELED</code>, which indicates
that we will only call MPI routines from the main thread. (FFTW will
launch additional threads internally, but the extra threads will not
call MPI code.) (You may also pass <code>MPI_THREAD_SERIALIZED</code> or
<code>MPI_THREAD_MULTIPLE</code>, which requests additional multithreading
support from the MPI implementation, but this is not required by
FFTW.) The <code>provided</code> parameter returns what level of threads
support is actually supported by your MPI implementation; this
<em>must</em> be at least <code>MPI_THREAD_FUNNELED</code> if you want to call
the FFTW threads routines, so we define a global variable
<code>threads_ok</code> to record this. You should only call
<code>fftw_init_threads</code> or <code>fftw_plan_with_nthreads</code> if
<code>threads_ok</code> is true. For more information on thread safety in
MPI, see the
<a href="http://www.mpi-forum.org/docs/mpi-20-html/node162.htm">MPI and
Threads</a> section of the MPI-2 standard.
<a name="index-thread-safety-2"></a>
</p>
<p>Second, we must call <code>fftw_init_threads</code> <em>before</em>
<code>fftw_mpi_init</code>. This is critical for technical reasons having
to do with how FFTW initializes its list of algorithms.
</p>
<p>Then, if you call <code>fftw_plan_with_nthreads(N)</code>, <em>every</em> MPI
process will launch (up to) <code>N</code> threads to parallelize its transforms.
</p>
<p>For example, in the hypothetical cluster of 4-processor nodes, you
might wish to launch only a single MPI process per node, and then call
<code>fftw_plan_with_nthreads(4)</code> on each process to use all
processors in the nodes.
</p>
<p>This may or may not be faster than simply using as many MPI processes
as you have processors, however. On the one hand, using threads
within a node eliminates the need for explicit message passing within
the node. On the other hand, FFTW’s transpose routines are not
multi-threaded, and this means that the communications that do take
place will not benefit from parallelization within the node.
Moreover, many MPI implementations already have optimizations to
exploit shared memory when it is available, so adding the
multithreaded FFTW on top of this may be superfluous.
<a name="index-transpose-4"></a>
</p>
<hr>
<div class="header">
<p>
Next: <a href="FFTW-MPI-Reference.html#FFTW-MPI-Reference" accesskey="n" rel="next">FFTW MPI Reference</a>, Previous: <a href="FFTW-MPI-Performance-Tips.html#FFTW-MPI-Performance-Tips" accesskey="p" rel="prev">FFTW MPI Performance Tips</a>, Up: <a href="Distributed_002dmemory-FFTW-with-MPI.html#Distributed_002dmemory-FFTW-with-MPI" accesskey="u" rel="up">Distributed-memory FFTW with MPI</a> [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Concept-Index.html#Concept-Index" title="Index" rel="index">Index</a>]</p>
</div>
</body>
</html>
|