Merge lp:~jobh/dolfin/parallel-dirichlet into lp:~fenics-core/dolfin/trunk
- parallel-dirichlet
- Merge into trunk
Status: | Merged |
---|---|
Merge reported by: | Garth Wells |
Merged at revision: | not available |
Proposed branch: | lp:~jobh/dolfin/parallel-dirichlet |
Merge into: | lp:~fenics-core/dolfin/trunk |
Diff against target: |
934 lines (+352/-67) 10 files modified
dolfin/common/MPI.cpp (+30/-1) dolfin/common/MPI.h (+114/-16) dolfin/fem/DirichletBC.cpp (+92/-22) dolfin/fem/DirichletBC.h (+25/-8) dolfin/fem/DofMap.cpp (+22/-1) dolfin/fem/DofMap.h (+23/-1) dolfin/fem/DofMapBuilder.cpp (+27/-3) dolfin/fem/DofMapBuilder.h (+5/-0) dolfin/fem/GenericDofMap.h (+10/-1) dolfin/fem/SystemAssembler.cpp (+4/-14) |
To merge this branch: | bzr merge lp:~jobh/dolfin/parallel-dirichlet |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Garth Wells | Approve | ||
Review via email:
|
Commit message
Description of the change
This branch does the following to Dirichlet boundary conditions:
(a) Make DirichletBC pointwise search faster (by not doing unnecessary initialisation of facets, and only visiting vertices once).
(b) Make DirichletBC topological and geometric search safe in parallel for SystemAssembler (by adding a gather() method which exchanges boundary values with neighbour processes).
==
(a) is a simple change, by just delaying initialisation of facets until they are required. The timings for Poisson on a 48^3 cube are:
before [s] after [s]
Topological search, compiled boundary subdomain
20.3 19.9
Topological search, python boundary subdomain
29.2 29.5
Pointwise search, compiled boundary subdomain
21.4 19.9
Pointwise search, python boundary subdomain
46.4 20.5 (!)
The recorded times include mesh create, 2x assemble, 2x DirichletBC create, 2x DirichletBC apply. The times do not change very much except for the case of pointwise search with a Python subdomain, which is dramatically improved. Pointwise search is now (much) faster than topological search when using Python subdomains.
(b) is more involved, and includes adding neighbour / dof sharing information to DofMap, and MPI functionality for exchanging data with neighbours (a new distribute() call, and a new send_recv because the current one deadlocks when posting different number of send_recvs on different processes).
It passes the unit and system tests (except KrylovSolver, but that one fails for me in trunk too).
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Garth Wells (garth-wells) : | # |
Preview Diff
1 | === modified file 'dolfin/common/MPI.cpp' | |||
2 | --- dolfin/common/MPI.cpp 2011-11-18 12:14:50 +0000 | |||
3 | +++ dolfin/common/MPI.cpp 2012-02-29 14:31:44 +0000 | |||
4 | @@ -19,9 +19,10 @@ | |||
5 | 19 | // Modified by Anders Logg 2007-2011 | 19 | // Modified by Anders Logg 2007-2011 |
6 | 20 | // Modified by Ola Skavhaug 2008-2009 | 20 | // Modified by Ola Skavhaug 2008-2009 |
7 | 21 | // Modified by Niclas Jansson 2009 | 21 | // Modified by Niclas Jansson 2009 |
8 | 22 | // Modified by Joachim B Haga 2012 | ||
9 | 22 | // | 23 | // |
10 | 23 | // First added: 2007-11-30 | 24 | // First added: 2007-11-30 |
12 | 24 | // Last changed: 2011-08-25 | 25 | // Last changed: 2012-02-29 |
13 | 25 | 26 | ||
14 | 26 | #include <numeric> | 27 | #include <numeric> |
15 | 27 | #include <dolfin/log/dolfin_log.h> | 28 | #include <dolfin/log/dolfin_log.h> |
16 | @@ -145,6 +146,23 @@ | |||
17 | 145 | return r + (index - r * (n + 1)) / n; | 146 | return r + (index - r * (n + 1)) / n; |
18 | 146 | } | 147 | } |
19 | 147 | //----------------------------------------------------------------------------- | 148 | //----------------------------------------------------------------------------- |
20 | 149 | dolfin::MPINonblocking::MPINonblocking() | ||
21 | 150 | {} | ||
22 | 151 | //----------------------------------------------------------------------------- | ||
23 | 152 | dolfin::MPINonblocking::~MPINonblocking() | ||
24 | 153 | { | ||
25 | 154 | wait_all(); | ||
26 | 155 | } | ||
27 | 156 | //----------------------------------------------------------------------------- | ||
28 | 157 | void dolfin::MPINonblocking::wait_all() | ||
29 | 158 | { | ||
30 | 159 | if (!reqs.empty()) | ||
31 | 160 | { | ||
32 | 161 | boost::mpi::wait_all(reqs.begin(), reqs.end()); | ||
33 | 162 | reqs.clear(); | ||
34 | 163 | } | ||
35 | 164 | } | ||
36 | 165 | //----------------------------------------------------------------------------- | ||
37 | 148 | //----------------------------------------------------------------------------- | 166 | //----------------------------------------------------------------------------- |
38 | 149 | #else | 167 | #else |
39 | 150 | //----------------------------------------------------------------------------- | 168 | //----------------------------------------------------------------------------- |
40 | @@ -217,4 +235,15 @@ | |||
41 | 217 | return 0; | 235 | return 0; |
42 | 218 | } | 236 | } |
43 | 219 | //----------------------------------------------------------------------------- | 237 | //----------------------------------------------------------------------------- |
44 | 238 | dolfin::MPINonblocking::Nonblocking() | ||
45 | 239 | {} | ||
46 | 240 | //----------------------------------------------------------------------------- | ||
47 | 241 | dolfin::MPINonblocking::~Nonblocking() | ||
48 | 242 | {} | ||
49 | 243 | //----------------------------------------------------------------------------- | ||
50 | 244 | void dolfin::MPINonblocking::wait_all() | ||
51 | 245 | { | ||
52 | 246 | error_no_mpi("call MPINonblocking::wait_all"); | ||
53 | 247 | } | ||
54 | 248 | |||
55 | 220 | #endif | 249 | #endif |
56 | 221 | 250 | ||
57 | === modified file 'dolfin/common/MPI.h' | |||
58 | --- dolfin/common/MPI.h 2011-12-04 11:33:42 +0000 | |||
59 | +++ dolfin/common/MPI.h 2012-02-29 14:31:44 +0000 | |||
60 | @@ -18,9 +18,10 @@ | |||
61 | 18 | // Modified by Ola Skavhaug 2008-2009 | 18 | // Modified by Ola Skavhaug 2008-2009 |
62 | 19 | // Modified by Anders Logg 2008-2011 | 19 | // Modified by Anders Logg 2008-2011 |
63 | 20 | // Modified by Niclas Jansson 2009 | 20 | // Modified by Niclas Jansson 2009 |
64 | 21 | // Modified by Joachim B Haga 2012 | ||
65 | 21 | // | 22 | // |
66 | 22 | // First added: 2007-11-30 | 23 | // First added: 2007-11-30 |
68 | 23 | // Last changed: 2011-09-28 | 24 | // Last changed: 2012-02-29 |
69 | 24 | 25 | ||
70 | 25 | #ifndef __MPI_DOLFIN_WRAPPER_H | 26 | #ifndef __MPI_DOLFIN_WRAPPER_H |
71 | 26 | #define __MPI_DOLFIN_WRAPPER_H | 27 | #define __MPI_DOLFIN_WRAPPER_H |
72 | @@ -31,6 +32,7 @@ | |||
73 | 31 | #include <dolfin/log/dolfin_log.h> | 32 | #include <dolfin/log/dolfin_log.h> |
74 | 32 | 33 | ||
75 | 33 | #ifdef HAS_MPI | 34 | #ifdef HAS_MPI |
76 | 35 | #include <boost/serialization/utility.hpp> | ||
77 | 34 | #include <boost/mpi.hpp> | 36 | #include <boost/mpi.hpp> |
78 | 35 | #include <mpi.h> | 37 | #include <mpi.h> |
79 | 36 | #endif | 38 | #endif |
80 | @@ -58,6 +60,32 @@ | |||
81 | 58 | }; | 60 | }; |
82 | 59 | #endif | 61 | #endif |
83 | 60 | 62 | ||
84 | 63 | /// This class provides stateful (single communicator) non-blocking MPI functionality. | ||
85 | 64 | |||
86 | 65 | class MPINonblocking | ||
87 | 66 | { | ||
88 | 67 | public: | ||
89 | 68 | /// Create instance with associated MPI_WORLD communicator | ||
90 | 69 | MPINonblocking(); | ||
91 | 70 | |||
92 | 71 | /// Destroy instance (waits for outstanding requests) | ||
93 | 72 | ~MPINonblocking(); | ||
94 | 73 | |||
95 | 74 | /// Non-blocking send and receive | ||
96 | 75 | template<typename T> | ||
97 | 76 | void send_recv(const T& send_value, uint dest, | ||
98 | 77 | T& recv_value, uint source); | ||
99 | 78 | |||
100 | 79 | /// Wait for all requests to finish | ||
101 | 80 | void wait_all(); | ||
102 | 81 | |||
103 | 82 | #ifdef HAS_MPI | ||
104 | 83 | private: | ||
105 | 84 | MPICommunicator mpi_comm; | ||
106 | 85 | std::vector<boost::mpi::request> reqs; | ||
107 | 86 | #endif | ||
108 | 87 | }; | ||
109 | 88 | |||
110 | 61 | /// This class provides utility functions for easy communcation with MPI. | 89 | /// This class provides utility functions for easy communcation with MPI. |
111 | 62 | 90 | ||
112 | 63 | class MPI | 91 | class MPI |
113 | @@ -100,13 +128,22 @@ | |||
114 | 100 | distribute(in_values, destinations, out_values, sources); | 128 | distribute(in_values, destinations, out_values, sources); |
115 | 101 | } | 129 | } |
116 | 102 | 130 | ||
117 | 131 | /// Distribute local arrays on a group of processes (typically neighbours | ||
118 | 132 | /// from GenericDofMap::neighbours()). It is important that each process' | ||
119 | 133 | /// group includes exactly the processes that has it in their groups, | ||
120 | 134 | /// otherwise it will deadlock. | ||
121 | 135 | template<typename T> | ||
122 | 136 | static void distribute(const std::set<uint> group, | ||
123 | 137 | const std::map<uint,T>& in_values_per_dest, | ||
124 | 138 | std::map<uint,T>& out_values_per_src); | ||
125 | 139 | |||
126 | 103 | // Broadcast value from broadcaster process to all processes | 140 | // Broadcast value from broadcaster process to all processes |
127 | 104 | template<typename T> | 141 | template<typename T> |
128 | 105 | static void broadcast(T& value, uint broadcaster=0) | 142 | static void broadcast(T& value, uint broadcaster=0) |
129 | 106 | { | 143 | { |
130 | 107 | #ifdef HAS_MPI | 144 | #ifdef HAS_MPI |
131 | 108 | MPICommunicator mpi_comm; | 145 | MPICommunicator mpi_comm; |
133 | 109 | boost::mpi::communicator comm(*mpi_comm, boost::mpi::comm_duplicate); | 146 | boost::mpi::communicator comm(*mpi_comm, boost::mpi::comm_attach); |
134 | 110 | boost::mpi::broadcast(comm, value, broadcaster); | 147 | boost::mpi::broadcast(comm, value, broadcaster); |
135 | 111 | #endif | 148 | #endif |
136 | 112 | } | 149 | } |
137 | @@ -118,7 +155,7 @@ | |||
138 | 118 | { | 155 | { |
139 | 119 | #ifdef HAS_MPI | 156 | #ifdef HAS_MPI |
140 | 120 | MPICommunicator mpi_comm; | 157 | MPICommunicator mpi_comm; |
142 | 121 | boost::mpi::communicator comm(*mpi_comm, boost::mpi::comm_duplicate); | 158 | boost::mpi::communicator comm(*mpi_comm, boost::mpi::comm_attach); |
143 | 122 | boost::mpi::scatter(comm, in_values, out_value, sending_process); | 159 | boost::mpi::scatter(comm, in_values, out_value, sending_process); |
144 | 123 | #else | 160 | #else |
145 | 124 | dolfin_assert(sending_process == 0); | 161 | dolfin_assert(sending_process == 0); |
146 | @@ -134,7 +171,7 @@ | |||
147 | 134 | { | 171 | { |
148 | 135 | #ifdef HAS_MPI | 172 | #ifdef HAS_MPI |
149 | 136 | MPICommunicator mpi_comm; | 173 | MPICommunicator mpi_comm; |
151 | 137 | boost::mpi::communicator comm(*mpi_comm, boost::mpi::comm_duplicate); | 174 | boost::mpi::communicator comm(*mpi_comm, boost::mpi::comm_attach); |
152 | 138 | boost::mpi::gather(comm, in_value, out_values, receiving_process); | 175 | boost::mpi::gather(comm, in_value, out_values, receiving_process); |
153 | 139 | #else | 176 | #else |
154 | 140 | out_values.clear(); | 177 | out_values.clear(); |
155 | @@ -148,7 +185,7 @@ | |||
156 | 148 | { | 185 | { |
157 | 149 | #ifdef HAS_MPI | 186 | #ifdef HAS_MPI |
158 | 150 | MPICommunicator mpi_comm; | 187 | MPICommunicator mpi_comm; |
160 | 151 | boost::mpi::communicator comm(*mpi_comm, boost::mpi::comm_duplicate); | 188 | boost::mpi::communicator comm(*mpi_comm, boost::mpi::comm_attach); |
161 | 152 | boost::mpi::all_gather(comm, in_value, out_values); | 189 | boost::mpi::all_gather(comm, in_value, out_values); |
162 | 153 | #else | 190 | #else |
163 | 154 | out_values.clear(); | 191 | out_values.clear(); |
164 | @@ -191,7 +228,7 @@ | |||
165 | 191 | { | 228 | { |
166 | 192 | #ifdef HAS_MPI | 229 | #ifdef HAS_MPI |
167 | 193 | MPICommunicator mpi_comm; | 230 | MPICommunicator mpi_comm; |
169 | 194 | boost::mpi::communicator comm(*mpi_comm, boost::mpi::comm_duplicate); | 231 | boost::mpi::communicator comm(*mpi_comm, boost::mpi::comm_attach); |
170 | 195 | T out; | 232 | T out; |
171 | 196 | boost::mpi::all_reduce(comm, value, out, op); | 233 | boost::mpi::all_reduce(comm, value, out, op); |
172 | 197 | return out; | 234 | return out; |
173 | @@ -207,20 +244,17 @@ | |||
174 | 207 | /// reduction op) | 244 | /// reduction op) |
175 | 208 | static uint global_offset(uint range, bool exclusive); | 245 | static uint global_offset(uint range, bool exclusive); |
176 | 209 | 246 | ||
178 | 210 | /// Send-receive and data | 247 | /// Send-receive data. Note that if the number of posted send-receives may |
179 | 248 | /// differ between processes, another interface (such as MPINonblocking::send_recv) | ||
180 | 249 | /// must be used since duplicating the communicator requires participation | ||
181 | 250 | /// from all processes. | ||
182 | 211 | template<typename T> | 251 | template<typename T> |
183 | 212 | static void send_recv(const T& send_value, uint dest, | 252 | static void send_recv(const T& send_value, uint dest, |
184 | 213 | T& recv_value, uint source) | 253 | T& recv_value, uint source) |
185 | 214 | { | 254 | { |
186 | 215 | #ifdef HAS_MPI | 255 | #ifdef HAS_MPI |
195 | 216 | MPICommunicator mpi_comm; | 256 | MPINonblocking mpi; |
196 | 217 | boost::mpi::communicator comm(*mpi_comm, boost::mpi::comm_duplicate); | 257 | mpi.send_recv(send_value, dest, recv_value, source); |
189 | 218 | |||
190 | 219 | // Non-blocking send-receive | ||
191 | 220 | boost::mpi::request reqs[2]; | ||
192 | 221 | reqs[0] = comm.isend(dest, 0, send_value); | ||
193 | 222 | reqs[1] = comm.irecv(source, 0, recv_value); | ||
194 | 223 | boost::mpi::wait_all(reqs, reqs + 2); | ||
197 | 224 | #else | 258 | #else |
198 | 225 | dolfin_error("MPI.h", | 259 | dolfin_error("MPI.h", |
199 | 226 | "call MPI::send_recv", | 260 | "call MPI::send_recv", |
200 | @@ -246,6 +280,13 @@ | |||
201 | 246 | 280 | ||
202 | 247 | private: | 281 | private: |
203 | 248 | 282 | ||
204 | 283 | #ifndef HAS_MPI | ||
205 | 284 | static void error_no_mpi(const char *where) | ||
206 | 285 | { | ||
207 | 286 | dolfin_error("MPI.h", msg, "DOLFIN has been configured without MPI support"); | ||
208 | 287 | } | ||
209 | 288 | #endif | ||
210 | 289 | |||
211 | 249 | #ifdef HAS_MPI | 290 | #ifdef HAS_MPI |
212 | 250 | // Return MPI data type | 291 | // Return MPI data type |
213 | 251 | template<typename T> static MPI_Datatype mpi_type() | 292 | template<typename T> static MPI_Datatype mpi_type() |
214 | @@ -259,6 +300,7 @@ | |||
215 | 259 | 300 | ||
216 | 260 | }; | 301 | }; |
217 | 261 | 302 | ||
218 | 303 | |||
219 | 262 | #ifdef HAS_MPI | 304 | #ifdef HAS_MPI |
220 | 263 | // Specialisations for MPI_Datatypes | 305 | // Specialisations for MPI_Datatypes |
221 | 264 | template<> inline MPI_Datatype MPI::mpi_type<double>() { return MPI_DOUBLE; } | 306 | template<> inline MPI_Datatype MPI::mpi_type<double>() { return MPI_DOUBLE; } |
222 | @@ -337,7 +379,63 @@ | |||
223 | 337 | "DOLFIN has been configured without MPI support"); | 379 | "DOLFIN has been configured without MPI support"); |
224 | 338 | } | 380 | } |
225 | 339 | #endif | 381 | #endif |
227 | 340 | //--------------------------------------------------------------------------- | 382 | |
228 | 383 | //----------------------------------------------------------------------------- | ||
229 | 384 | template<typename T> | ||
230 | 385 | void dolfin::MPI::distribute(const std::set<uint> group, | ||
231 | 386 | const std::map<uint,T>& in_values_per_dest, | ||
232 | 387 | std::map<uint,T>& out_values_per_src) | ||
233 | 388 | { | ||
234 | 389 | #ifdef HAS_MPI | ||
235 | 390 | typedef typename std::map<uint,T>::const_iterator map_const_iterator; | ||
236 | 391 | typedef typename std::map<uint,T>::iterator map_iterator; | ||
237 | 392 | dolfin::MPINonblocking mpi; | ||
238 | 393 | const T no_data; | ||
239 | 394 | |||
240 | 395 | // Send and receive values to all processes in groups (non-blocking). If a | ||
241 | 396 | // given process is not found in in_values_per_dest, send empty data. | ||
242 | 397 | |||
243 | 398 | out_values_per_src.clear(); | ||
244 | 399 | for (std::set<uint>::const_iterator dest = group.begin(); dest != group.end(); ++dest) | ||
245 | 400 | { | ||
246 | 401 | map_const_iterator values = in_values_per_dest.find(*dest); | ||
247 | 402 | if (values != in_values_per_dest.end()) | ||
248 | 403 | mpi.send_recv(values->second, *dest, out_values_per_src[*dest], *dest); | ||
249 | 404 | else | ||
250 | 405 | mpi.send_recv(no_data, *dest, out_values_per_src[*dest], *dest); | ||
251 | 406 | } | ||
252 | 407 | |||
253 | 408 | // Wait for all MPI calls before modifying out_values_per_src | ||
254 | 409 | |||
255 | 410 | mpi.wait_all(); | ||
256 | 411 | |||
257 | 412 | // Remove received no_data entries. | ||
258 | 413 | |||
259 | 414 | map_iterator it = out_values_per_src.begin(); | ||
260 | 415 | while (it != out_values_per_src.end()) | ||
261 | 416 | { | ||
262 | 417 | map_iterator tmp = it++; | ||
263 | 418 | if (tmp->second.empty()) | ||
264 | 419 | out_values_per_src.erase(tmp); // map::erase only invalidates current iterator | ||
265 | 420 | } | ||
266 | 421 | #else | ||
267 | 422 | error_no_mpi("call MPI::distribute"); | ||
268 | 423 | #endif | ||
269 | 424 | } | ||
270 | 425 | |||
271 | 426 | //----------------------------------------------------------------------------- | ||
272 | 427 | template<typename T> | ||
273 | 428 | void dolfin::MPINonblocking::send_recv(const T& send_value, uint dest, | ||
274 | 429 | T& recv_value, uint source) | ||
275 | 430 | { | ||
276 | 431 | #ifdef HAS_MPI | ||
277 | 432 | boost::mpi::communicator comm(*mpi_comm, boost::mpi::comm_attach); | ||
278 | 433 | reqs.push_back(comm.isend(dest, 0, send_value)); | ||
279 | 434 | reqs.push_back(comm.irecv(source, 0, recv_value)); | ||
280 | 435 | #else | ||
281 | 436 | error_no_mpi("call MPINonblocking::send_recv"); | ||
282 | 437 | #endif | ||
283 | 438 | } | ||
284 | 341 | 439 | ||
285 | 342 | } | 440 | } |
286 | 343 | 441 | ||
287 | 344 | 442 | ||
288 | === modified file 'dolfin/fem/DirichletBC.cpp' | |||
289 | --- dolfin/fem/DirichletBC.cpp 2011-12-15 00:03:23 +0000 | |||
290 | +++ dolfin/fem/DirichletBC.cpp 2012-02-29 14:31:44 +0000 | |||
291 | @@ -18,13 +18,15 @@ | |||
292 | 18 | // Modified by Kristian Oelgaard, 2008 | 18 | // Modified by Kristian Oelgaard, 2008 |
293 | 19 | // Modified by Martin Sandve Alnes, 2008 | 19 | // Modified by Martin Sandve Alnes, 2008 |
294 | 20 | // Modified by Johan Hake, 2009 | 20 | // Modified by Johan Hake, 2009 |
295 | 21 | // Modified by Joachim B Haga, 2009 | ||
296 | 21 | // | 22 | // |
297 | 22 | // First added: 2007-04-10 | 23 | // First added: 2007-04-10 |
299 | 23 | // Last changed: 2011-09-19 | 24 | // Last changed: 2012-02-29 |
300 | 24 | 25 | ||
301 | 25 | #include <map> | 26 | #include <map> |
302 | 26 | #include <utility> | 27 | #include <utility> |
303 | 27 | #include <boost/assign/list_of.hpp> | 28 | #include <boost/assign/list_of.hpp> |
304 | 29 | #include <boost/serialization/utility.hpp> | ||
305 | 28 | 30 | ||
306 | 29 | #include <dolfin/common/constants.h> | 31 | #include <dolfin/common/constants.h> |
307 | 30 | #include <dolfin/common/Array.h> | 32 | #include <dolfin/common/Array.h> |
308 | @@ -67,7 +69,6 @@ | |||
309 | 67 | { | 69 | { |
310 | 68 | check(); | 70 | check(); |
311 | 69 | parameters = default_parameters(); | 71 | parameters = default_parameters(); |
312 | 70 | init_from_sub_domain(_user_sub_domain); | ||
313 | 71 | } | 72 | } |
314 | 72 | //----------------------------------------------------------------------------- | 73 | //----------------------------------------------------------------------------- |
315 | 73 | DirichletBC::DirichletBC(boost::shared_ptr<const FunctionSpace> V, | 74 | DirichletBC::DirichletBC(boost::shared_ptr<const FunctionSpace> V, |
316 | @@ -80,7 +81,6 @@ | |||
317 | 80 | { | 81 | { |
318 | 81 | check(); | 82 | check(); |
319 | 82 | parameters = default_parameters(); | 83 | parameters = default_parameters(); |
320 | 83 | init_from_sub_domain(_user_sub_domain); | ||
321 | 84 | } | 84 | } |
322 | 85 | //----------------------------------------------------------------------------- | 85 | //----------------------------------------------------------------------------- |
323 | 86 | DirichletBC::DirichletBC(const FunctionSpace& V, const GenericFunction& g, | 86 | DirichletBC::DirichletBC(const FunctionSpace& V, const GenericFunction& g, |
324 | @@ -89,11 +89,12 @@ | |||
325 | 89 | : BoundaryCondition(V), | 89 | : BoundaryCondition(V), |
326 | 90 | Hierarchical<DirichletBC>(*this), | 90 | Hierarchical<DirichletBC>(*this), |
327 | 91 | g(reference_to_no_delete_pointer(g)), | 91 | g(reference_to_no_delete_pointer(g)), |
329 | 92 | _method(method) | 92 | _method(method), |
330 | 93 | _user_mesh_function(reference_to_no_delete_pointer(sub_domains)), | ||
331 | 94 | _user_sub_domain_marker(sub_domain) | ||
332 | 93 | { | 95 | { |
333 | 94 | check(); | 96 | check(); |
334 | 95 | parameters = default_parameters(); | 97 | parameters = default_parameters(); |
335 | 96 | init_from_mesh_function(sub_domains, sub_domain); | ||
336 | 97 | } | 98 | } |
337 | 98 | //----------------------------------------------------------------------------- | 99 | //----------------------------------------------------------------------------- |
338 | 99 | DirichletBC::DirichletBC(boost::shared_ptr<const FunctionSpace> V, | 100 | DirichletBC::DirichletBC(boost::shared_ptr<const FunctionSpace> V, |
339 | @@ -103,22 +104,23 @@ | |||
340 | 103 | std::string method) | 104 | std::string method) |
341 | 104 | : BoundaryCondition(V), | 105 | : BoundaryCondition(V), |
342 | 105 | Hierarchical<DirichletBC>(*this), | 106 | Hierarchical<DirichletBC>(*this), |
344 | 106 | g(g), _method(method) | 107 | g(g), _method(method), |
345 | 108 | _user_mesh_function(sub_domains), | ||
346 | 109 | _user_sub_domain_marker(sub_domain) | ||
347 | 107 | { | 110 | { |
348 | 108 | check(); | 111 | check(); |
349 | 109 | parameters = default_parameters(); | 112 | parameters = default_parameters(); |
350 | 110 | init_from_mesh_function(*sub_domains, sub_domain); | ||
351 | 111 | } | 113 | } |
352 | 112 | //----------------------------------------------------------------------------- | 114 | //----------------------------------------------------------------------------- |
353 | 113 | DirichletBC::DirichletBC(const FunctionSpace& V, const GenericFunction& g, | 115 | DirichletBC::DirichletBC(const FunctionSpace& V, const GenericFunction& g, |
354 | 114 | uint sub_domain, std::string method) | 116 | uint sub_domain, std::string method) |
355 | 115 | : BoundaryCondition(V), | 117 | : BoundaryCondition(V), |
356 | 116 | Hierarchical<DirichletBC>(*this), | 118 | Hierarchical<DirichletBC>(*this), |
358 | 117 | g(reference_to_no_delete_pointer(g)), _method(method) | 119 | g(reference_to_no_delete_pointer(g)), _method(method), |
359 | 120 | _user_sub_domain_marker(sub_domain) | ||
360 | 118 | { | 121 | { |
361 | 119 | check(); | 122 | check(); |
362 | 120 | parameters = default_parameters(); | 123 | parameters = default_parameters(); |
363 | 121 | init_from_mesh(sub_domain); | ||
364 | 122 | } | 124 | } |
365 | 123 | //----------------------------------------------------------------------------- | 125 | //----------------------------------------------------------------------------- |
366 | 124 | DirichletBC::DirichletBC(boost::shared_ptr<const FunctionSpace> V, | 126 | DirichletBC::DirichletBC(boost::shared_ptr<const FunctionSpace> V, |
367 | @@ -126,11 +128,11 @@ | |||
368 | 126 | uint sub_domain, std::string method) | 128 | uint sub_domain, std::string method) |
369 | 127 | : BoundaryCondition(V), | 129 | : BoundaryCondition(V), |
370 | 128 | Hierarchical<DirichletBC>(*this), | 130 | Hierarchical<DirichletBC>(*this), |
372 | 129 | g(g), _method(method) | 131 | g(g), _method(method), |
373 | 132 | _user_sub_domain_marker(sub_domain) | ||
374 | 130 | { | 133 | { |
375 | 131 | check(); | 134 | check(); |
376 | 132 | parameters = default_parameters(); | 135 | parameters = default_parameters(); |
377 | 133 | init_from_mesh(sub_domain); | ||
378 | 134 | } | 136 | } |
379 | 135 | //----------------------------------------------------------------------------- | 137 | //----------------------------------------------------------------------------- |
380 | 136 | DirichletBC::DirichletBC(boost::shared_ptr<const FunctionSpace> V, | 138 | DirichletBC::DirichletBC(boost::shared_ptr<const FunctionSpace> V, |
381 | @@ -201,6 +203,43 @@ | |||
382 | 201 | apply(&A, &b, &x); | 203 | apply(&A, &b, &x); |
383 | 202 | } | 204 | } |
384 | 203 | //----------------------------------------------------------------------------- | 205 | //----------------------------------------------------------------------------- |
385 | 206 | void DirichletBC::gather(Map& boundary_values) const | ||
386 | 207 | { | ||
387 | 208 | typedef std::vector<std::pair<uint, double> > bv_vec_type; | ||
388 | 209 | typedef std::map<uint, bv_vec_type> map_type; | ||
389 | 210 | |||
390 | 211 | typedef boost::unordered_map<uint, std::vector<uint> > shared_dof_type; | ||
391 | 212 | typedef shared_dof_type::const_iterator shared_dof_iterator; | ||
392 | 213 | typedef std::vector<uint>::const_iterator proc_iterator; | ||
393 | 214 | |||
394 | 215 | dolfin_assert(_function_space->dofmap()); | ||
395 | 216 | const GenericDofMap& dofmap = *_function_space->dofmap(); | ||
396 | 217 | const shared_dof_type& shared_dofs = dofmap.shared_dofs(); | ||
397 | 218 | |||
398 | 219 | // Create list of boundary values to send to each processor | ||
399 | 220 | |||
400 | 221 | map_type proc_map; | ||
401 | 222 | for (Map::const_iterator bv = boundary_values.begin(); bv != boundary_values.end(); ++bv) | ||
402 | 223 | { | ||
403 | 224 | // If the boundary value is attached to a shared dof, add it to the list of | ||
404 | 225 | // boundary values for each of the processors that share it | ||
405 | 226 | shared_dof_iterator shared_dof = shared_dofs.find(bv->first); | ||
406 | 227 | if (shared_dof != shared_dofs.end()) | ||
407 | 228 | for (proc_iterator proc = shared_dof->second.begin(); proc != shared_dof->second.end(); ++proc) | ||
408 | 229 | proc_map[*proc].push_back(*bv); | ||
409 | 230 | } | ||
410 | 231 | |||
411 | 232 | // Distribute the lists between neighbours | ||
412 | 233 | |||
413 | 234 | map_type received_bvs; | ||
414 | 235 | MPI::distribute(dofmap.neighbours(), proc_map, received_bvs); | ||
415 | 236 | |||
416 | 237 | // Add the received boundary values to the local boundary values | ||
417 | 238 | |||
418 | 239 | for (map_type::const_iterator it = received_bvs.begin(); it != received_bvs.end(); ++it) | ||
419 | 240 | boundary_values.insert(it->second.begin(), it->second.end()); | ||
420 | 241 | } | ||
421 | 242 | //----------------------------------------------------------------------------- | ||
422 | 204 | void DirichletBC::get_boundary_values(Map& boundary_values, | 243 | void DirichletBC::get_boundary_values(Map& boundary_values, |
423 | 205 | std::string method) const | 244 | std::string method) const |
424 | 206 | { | 245 | { |
425 | @@ -564,7 +603,20 @@ | |||
426 | 564 | } | 603 | } |
427 | 565 | } | 604 | } |
428 | 566 | //----------------------------------------------------------------------------- | 605 | //----------------------------------------------------------------------------- |
430 | 567 | void DirichletBC::init_from_sub_domain(boost::shared_ptr<const SubDomain> sub_domain) | 606 | void DirichletBC::init_facets() const |
431 | 607 | { | ||
432 | 608 | if (facets.size() > 0) | ||
433 | 609 | return; | ||
434 | 610 | |||
435 | 611 | if (_user_sub_domain) | ||
436 | 612 | init_from_sub_domain(_user_sub_domain); | ||
437 | 613 | else if (_user_mesh_function) | ||
438 | 614 | init_from_mesh_function(*_user_mesh_function, _user_sub_domain_marker); | ||
439 | 615 | else | ||
440 | 616 | init_from_mesh(_user_sub_domain_marker); | ||
441 | 617 | } | ||
442 | 618 | //----------------------------------------------------------------------------- | ||
443 | 619 | void DirichletBC::init_from_sub_domain(boost::shared_ptr<const SubDomain> sub_domain) const | ||
444 | 568 | { | 620 | { |
445 | 569 | dolfin_assert(facets.size() == 0); | 621 | dolfin_assert(facets.size() == 0); |
446 | 570 | 622 | ||
447 | @@ -595,7 +647,7 @@ | |||
448 | 595 | } | 647 | } |
449 | 596 | //----------------------------------------------------------------------------- | 648 | //----------------------------------------------------------------------------- |
450 | 597 | void DirichletBC::init_from_mesh_function(const MeshFunction<uint>& sub_domains, | 649 | void DirichletBC::init_from_mesh_function(const MeshFunction<uint>& sub_domains, |
452 | 598 | uint sub_domain) | 650 | uint sub_domain) const |
453 | 599 | { | 651 | { |
454 | 600 | dolfin_assert(facets.size() == 0); | 652 | dolfin_assert(facets.size() == 0); |
455 | 601 | 653 | ||
456 | @@ -624,7 +676,7 @@ | |||
457 | 624 | } | 676 | } |
458 | 625 | } | 677 | } |
459 | 626 | //----------------------------------------------------------------------------- | 678 | //----------------------------------------------------------------------------- |
461 | 627 | void DirichletBC::init_from_mesh(uint sub_domain) | 679 | void DirichletBC::init_from_mesh(uint sub_domain) const |
462 | 628 | { | 680 | { |
463 | 629 | dolfin_assert(facets.size() == 0); | 681 | dolfin_assert(facets.size() == 0); |
464 | 630 | 682 | ||
465 | @@ -678,6 +730,8 @@ | |||
466 | 678 | dolfin_assert(_function_space); | 730 | dolfin_assert(_function_space); |
467 | 679 | dolfin_assert(g); | 731 | dolfin_assert(g); |
468 | 680 | 732 | ||
469 | 733 | init_facets(); | ||
470 | 734 | |||
471 | 681 | // Special case | 735 | // Special case |
472 | 682 | if (facets.size() == 0) | 736 | if (facets.size() == 0) |
473 | 683 | { | 737 | { |
474 | @@ -724,18 +778,19 @@ | |||
475 | 724 | const double value = data.w[data.facet_dofs[i]]; | 778 | const double value = data.w[data.facet_dofs[i]]; |
476 | 725 | boundary_values[global_dof] = value; | 779 | boundary_values[global_dof] = value; |
477 | 726 | } | 780 | } |
478 | 727 | |||
479 | 728 | p++; | 781 | p++; |
480 | 729 | } | 782 | } |
481 | 730 | } | 783 | } |
482 | 731 | //----------------------------------------------------------------------------- | 784 | //----------------------------------------------------------------------------- |
483 | 732 | void DirichletBC::compute_bc_geometric(Map& boundary_values, | 785 | void DirichletBC::compute_bc_geometric(Map& boundary_values, |
485 | 733 | BoundaryCondition::LocalData& data) const | 786 | BoundaryCondition::LocalData& data) const |
486 | 734 | { | 787 | { |
487 | 735 | dolfin_assert(_function_space); | 788 | dolfin_assert(_function_space); |
488 | 736 | dolfin_assert(_function_space->element()); | 789 | dolfin_assert(_function_space->element()); |
489 | 737 | dolfin_assert(g); | 790 | dolfin_assert(g); |
490 | 738 | 791 | ||
491 | 792 | init_facets(); | ||
492 | 793 | |||
493 | 739 | // Special case | 794 | // Special case |
494 | 740 | if (facets.size() == 0) | 795 | if (facets.size() == 0) |
495 | 741 | { | 796 | { |
496 | @@ -814,7 +869,11 @@ | |||
497 | 814 | dolfin_assert(_function_space); | 869 | dolfin_assert(_function_space); |
498 | 815 | dolfin_assert(_function_space->element()); | 870 | dolfin_assert(_function_space->element()); |
499 | 816 | dolfin_assert(g); | 871 | dolfin_assert(g); |
501 | 817 | dolfin_assert(_user_sub_domain); | 872 | |
502 | 873 | if (!_user_sub_domain) | ||
503 | 874 | dolfin_error("DirichletBC.cpp", | ||
504 | 875 | "computing Dirichlet boundary values, pointwise search", | ||
505 | 876 | "A SubDomain is required for pointwise search"); | ||
506 | 818 | 877 | ||
507 | 819 | // Get mesh and dofmap | 878 | // Get mesh and dofmap |
508 | 820 | dolfin_assert(_function_space->mesh()); | 879 | dolfin_assert(_function_space->mesh()); |
509 | @@ -828,9 +887,13 @@ | |||
510 | 828 | // Create UFC cell object | 887 | // Create UFC cell object |
511 | 829 | UFCCell ufc_cell(mesh); | 888 | UFCCell ufc_cell(mesh); |
512 | 830 | 889 | ||
513 | 890 | // Speeder-upper | ||
514 | 891 | std::pair<uint,uint> local_range = dofmap.ownership_range(); | ||
515 | 892 | std::vector<bool> already_visited(local_range.second - local_range.first); | ||
516 | 893 | std::fill(already_visited.begin(), already_visited.end(), false); | ||
517 | 894 | |||
518 | 831 | // Iterate over cells | 895 | // Iterate over cells |
519 | 832 | Progress p("Computing Dirichlet boundary values, pointwise search", mesh.num_cells()); | 896 | Progress p("Computing Dirichlet boundary values, pointwise search", mesh.num_cells()); |
520 | 833 | Array<double> x(gdim); | ||
521 | 834 | for (CellIterator cell(mesh); !cell.end(); ++cell) | 897 | for (CellIterator cell(mesh); !cell.end(); ++cell) |
522 | 835 | { | 898 | { |
523 | 836 | // Update UFC cell | 899 | // Update UFC cell |
524 | @@ -848,9 +911,17 @@ | |||
525 | 848 | // Loop all dofs on cell | 911 | // Loop all dofs on cell |
526 | 849 | for (uint i = 0; i < dofmap.cell_dimension(cell->index()); ++i) | 912 | for (uint i = 0; i < dofmap.cell_dimension(cell->index()); ++i) |
527 | 850 | { | 913 | { |
531 | 851 | // Check if the coordinates are part of the sub domain (calls user-defined 'indside' function) | 914 | const uint global_dof = cell_dofs[i]; |
532 | 852 | for (uint j = 0; j < gdim; ++j) | 915 | if (global_dof >= local_range.first && global_dof < local_range.second) |
533 | 853 | x[j] = data.coordinates[i][j]; | 916 | { |
534 | 917 | const uint dof_index = global_dof - local_range.first; | ||
535 | 918 | if (already_visited[dof_index]) | ||
536 | 919 | continue; | ||
537 | 920 | already_visited[dof_index] = true; | ||
538 | 921 | } | ||
539 | 922 | |||
540 | 923 | // Check if the coordinates are part of the sub domain (calls user-defined 'inside' function) | ||
541 | 924 | Array<double> x(gdim, &data.coordinates[i][0]); | ||
542 | 854 | if (!_user_sub_domain->inside(x, false)) | 925 | if (!_user_sub_domain->inside(x, false)) |
543 | 855 | continue; | 926 | continue; |
544 | 856 | 927 | ||
545 | @@ -863,7 +934,6 @@ | |||
546 | 863 | } | 934 | } |
547 | 864 | 935 | ||
548 | 865 | // Set boundary value | 936 | // Set boundary value |
549 | 866 | const uint global_dof = cell_dofs[i]; | ||
550 | 867 | const double value = data.w[i]; | 937 | const double value = data.w[i]; |
551 | 868 | boundary_values[global_dof] = value; | 938 | boundary_values[global_dof] = value; |
552 | 869 | } | 939 | } |
553 | 870 | 940 | ||
554 | === modified file 'dolfin/fem/DirichletBC.h' | |||
555 | --- dolfin/fem/DirichletBC.h 2011-10-26 08:59:01 +0000 | |||
556 | +++ dolfin/fem/DirichletBC.h 2012-02-29 14:31:44 +0000 | |||
557 | @@ -17,9 +17,10 @@ | |||
558 | 17 | // | 17 | // |
559 | 18 | // Modified by Kristian Oelgaard, 2007 | 18 | // Modified by Kristian Oelgaard, 2007 |
560 | 19 | // Modified by Johan Hake, 2009 | 19 | // Modified by Johan Hake, 2009 |
561 | 20 | // Modified by Joachim B Haga, 2012 | ||
562 | 20 | // | 21 | // |
563 | 21 | // First added: 2007-04-10 | 22 | // First added: 2007-04-10 |
565 | 22 | // Last changed: 2011-04-13 | 23 | // Last changed: 2012-02-29 |
566 | 23 | // | 24 | // |
567 | 24 | // FIXME: This class needs some cleanup, in particular collecting | 25 | // FIXME: This class needs some cleanup, in particular collecting |
568 | 25 | // FIXME: all data from different representations into a common | 26 | // FIXME: all data from different representations into a common |
569 | @@ -94,8 +95,6 @@ | |||
570 | 94 | /// three possibilties are "topological", "geometric" and | 95 | /// three possibilties are "topological", "geometric" and |
571 | 95 | /// "pointwise". | 96 | /// "pointwise". |
572 | 96 | 97 | ||
573 | 97 | /// This class specifies the interface for setting (strong) | ||
574 | 98 | |||
575 | 99 | class DirichletBC : public BoundaryCondition, public Hierarchical<DirichletBC> | 98 | class DirichletBC : public BoundaryCondition, public Hierarchical<DirichletBC> |
576 | 100 | { | 99 | { |
577 | 101 | 100 | ||
578 | @@ -301,6 +300,16 @@ | |||
579 | 301 | void get_boundary_values(Map& boundary_values, | 300 | void get_boundary_values(Map& boundary_values, |
580 | 302 | std::string method="default") const; | 301 | std::string method="default") const; |
581 | 303 | 302 | ||
582 | 303 | |||
583 | 304 | /// Get boundary values from neighbour processes. If a method other than | ||
584 | 305 | /// "pointwise" is used, this is necessary to ensure all boundary dofs are | ||
585 | 306 | /// marked on all processes. | ||
586 | 307 | /// | ||
587 | 308 | /// *Arguments* | ||
588 | 309 | /// boundary_values (boost::unordered_map<uint, double>) | ||
589 | 310 | /// Map from dof to boundary value. | ||
590 | 311 | void gather(Map& boundary_values) const; | ||
591 | 312 | |||
592 | 304 | /// Make rows of matrix associated with boundary condition zero, | 313 | /// Make rows of matrix associated with boundary condition zero, |
593 | 305 | /// useful for non-diagonal matrices in a block matrix. | 314 | /// useful for non-diagonal matrices in a block matrix. |
594 | 306 | /// | 315 | /// |
595 | @@ -401,15 +410,18 @@ | |||
596 | 401 | // Check input data to constructor | 410 | // Check input data to constructor |
597 | 402 | void check() const; | 411 | void check() const; |
598 | 403 | 412 | ||
599 | 413 | // Initialize facets (from sub domain, mesh, etc) | ||
600 | 414 | void init_facets() const; | ||
601 | 415 | |||
602 | 404 | // Initialize sub domain markers from sub domain | 416 | // Initialize sub domain markers from sub domain |
604 | 405 | void init_from_sub_domain(boost::shared_ptr<const SubDomain> sub_domain); | 417 | void init_from_sub_domain(boost::shared_ptr<const SubDomain> sub_domain) const; |
605 | 406 | 418 | ||
606 | 407 | // Initialize sub domain markers from MeshFunction | 419 | // Initialize sub domain markers from MeshFunction |
607 | 408 | void init_from_mesh_function(const MeshFunction<uint>& sub_domains, | 420 | void init_from_mesh_function(const MeshFunction<uint>& sub_domains, |
609 | 409 | uint sub_domain); | 421 | uint sub_domain) const; |
610 | 410 | 422 | ||
611 | 411 | // Initialize sub domain markers from mesh | 423 | // Initialize sub domain markers from mesh |
613 | 412 | void init_from_mesh(uint sub_domain); | 424 | void init_from_mesh(uint sub_domain) const; |
614 | 413 | 425 | ||
615 | 414 | // Compute dofs and values for application of boundary conditions using | 426 | // Compute dofs and values for application of boundary conditions using |
616 | 415 | // given method | 427 | // given method |
617 | @@ -444,8 +456,13 @@ | |||
618 | 444 | boost::shared_ptr<const SubDomain> _user_sub_domain; | 456 | boost::shared_ptr<const SubDomain> _user_sub_domain; |
619 | 445 | 457 | ||
620 | 446 | // Boundary facets, stored as pairs (cell, local facet number) | 458 | // Boundary facets, stored as pairs (cell, local facet number) |
623 | 447 | std::vector<std::pair<uint, uint> > facets; | 459 | mutable std::vector<std::pair<uint, uint> > facets; |
624 | 448 | 460 | ||
625 | 461 | // User defined mesh function | ||
626 | 462 | boost::shared_ptr<const MeshFunction<uint> > _user_mesh_function; | ||
627 | 463 | |||
628 | 464 | // User defined sub domain marker for mesh or mesh function | ||
629 | 465 | uint _user_sub_domain_marker; | ||
630 | 449 | }; | 466 | }; |
631 | 450 | 467 | ||
632 | 451 | } | 468 | } |
633 | 452 | 469 | ||
634 | === modified file 'dolfin/fem/DofMap.cpp' | |||
635 | --- dolfin/fem/DofMap.cpp 2011-11-18 12:14:50 +0000 | |||
636 | +++ dolfin/fem/DofMap.cpp 2012-02-29 14:31:44 +0000 | |||
637 | @@ -19,9 +19,10 @@ | |||
638 | 19 | // Modified by Kent-Andre Mardal, 2009 | 19 | // Modified by Kent-Andre Mardal, 2009 |
639 | 20 | // Modified by Ola Skavhaug, 2009 | 20 | // Modified by Ola Skavhaug, 2009 |
640 | 21 | // Modified by Niclas Jansson, 2009 | 21 | // Modified by Niclas Jansson, 2009 |
641 | 22 | // Modified by Joachim B Haga, 2012 | ||
642 | 22 | // | 23 | // |
643 | 23 | // First added: 2007-03-01 | 24 | // First added: 2007-03-01 |
645 | 24 | // Last changed: 2011-10-31 | 25 | // Last changed: 2012-02-29 |
646 | 25 | 26 | ||
647 | 26 | #include <dolfin/common/MPI.h> | 27 | #include <dolfin/common/MPI.h> |
648 | 27 | #include <dolfin/common/NoDeleter.h> | 28 | #include <dolfin/common/NoDeleter.h> |
649 | @@ -173,6 +174,8 @@ | |||
650 | 173 | // Modify dofmap for non-UFC numbering | 174 | // Modify dofmap for non-UFC numbering |
651 | 174 | ufc_map_to_dofmap.clear(); | 175 | ufc_map_to_dofmap.clear(); |
652 | 175 | _off_process_owner.clear(); | 176 | _off_process_owner.clear(); |
653 | 177 | _shared_dofs.clear(); | ||
654 | 178 | _neighbours.clear(); | ||
655 | 176 | if (parent_dofmap.ufc_map_to_dofmap.size() > 0) | 179 | if (parent_dofmap.ufc_map_to_dofmap.size() > 0) |
656 | 177 | { | 180 | { |
657 | 178 | boost::unordered_map<uint, uint>::const_iterator ufc_to_current_dof; | 181 | boost::unordered_map<uint, uint>::const_iterator ufc_to_current_dof; |
658 | @@ -196,6 +199,14 @@ | |||
659 | 196 | boost::unordered_map<uint, uint>::const_iterator parent_off_proc = parent_dofmap._off_process_owner.find(*dof); | 199 | boost::unordered_map<uint, uint>::const_iterator parent_off_proc = parent_dofmap._off_process_owner.find(*dof); |
660 | 197 | if (parent_off_proc != parent_dofmap._off_process_owner.end()) | 200 | if (parent_off_proc != parent_dofmap._off_process_owner.end()) |
661 | 198 | _off_process_owner.insert(*parent_off_proc); | 201 | _off_process_owner.insert(*parent_off_proc); |
662 | 202 | |||
663 | 203 | // Add to shared-dof process map, and update the set of neighbours | ||
664 | 204 | boost::unordered_map<uint, std::vector<uint> >::const_iterator parent_shared = parent_dofmap._shared_dofs.find(*dof); | ||
665 | 205 | if (parent_shared != parent_dofmap._shared_dofs.end()) | ||
666 | 206 | { | ||
667 | 207 | _shared_dofs.insert(*parent_shared); | ||
668 | 208 | _neighbours.insert(parent_shared->second.begin(), parent_shared->second.end()); | ||
669 | 209 | } | ||
670 | 199 | } | 210 | } |
671 | 200 | } | 211 | } |
672 | 201 | } | 212 | } |
673 | @@ -309,6 +320,16 @@ | |||
674 | 309 | return _off_process_owner; | 320 | return _off_process_owner; |
675 | 310 | } | 321 | } |
676 | 311 | //----------------------------------------------------------------------------- | 322 | //----------------------------------------------------------------------------- |
677 | 323 | const boost::unordered_map<uint, std::vector<uint> >& DofMap::shared_dofs() const | ||
678 | 324 | { | ||
679 | 325 | return _shared_dofs; | ||
680 | 326 | } | ||
681 | 327 | //----------------------------------------------------------------------------- | ||
682 | 328 | const std::set<uint>& DofMap::neighbours() const | ||
683 | 329 | { | ||
684 | 330 | return _neighbours; | ||
685 | 331 | } | ||
686 | 332 | //----------------------------------------------------------------------------- | ||
687 | 312 | void DofMap::tabulate_facet_dofs(uint* dofs, uint local_facet) const | 333 | void DofMap::tabulate_facet_dofs(uint* dofs, uint local_facet) const |
688 | 313 | { | 334 | { |
689 | 314 | dolfin_assert(_ufc_dofmap); | 335 | dolfin_assert(_ufc_dofmap); |
690 | 315 | 336 | ||
691 | === modified file 'dolfin/fem/DofMap.h' | |||
692 | --- dolfin/fem/DofMap.h 2011-11-18 12:14:50 +0000 | |||
693 | +++ dolfin/fem/DofMap.h 2012-02-29 14:31:44 +0000 | |||
694 | @@ -18,9 +18,10 @@ | |||
695 | 18 | // Modified by Martin Alnes, 2008 | 18 | // Modified by Martin Alnes, 2008 |
696 | 19 | // Modified by Kent-Andre Mardal, 2009 | 19 | // Modified by Kent-Andre Mardal, 2009 |
697 | 20 | // Modified by Ola Skavhaug, 2009 | 20 | // Modified by Ola Skavhaug, 2009 |
698 | 21 | // Modified by Joachim B Haga, 2012 | ||
699 | 21 | // | 22 | // |
700 | 22 | // First added: 2007-03-01 | 23 | // First added: 2007-03-01 |
702 | 23 | // Last changed: 2011-10-31 | 24 | // Last changed: 2012-02-29 |
703 | 24 | 25 | ||
704 | 25 | #ifndef __DOLFIN_DOF_MAP_H | 26 | #ifndef __DOLFIN_DOF_MAP_H |
705 | 26 | #define __DOLFIN_DOF_MAP_H | 27 | #define __DOLFIN_DOF_MAP_H |
706 | @@ -178,6 +179,21 @@ | |||
707 | 178 | /// The map from non-local dofs. | 179 | /// The map from non-local dofs. |
708 | 179 | const boost::unordered_map<unsigned int, unsigned int>& off_process_owner() const; | 180 | const boost::unordered_map<unsigned int, unsigned int>& off_process_owner() const; |
709 | 180 | 181 | ||
710 | 182 | /// Return map from all shared dofs to the processes (not including the current | ||
711 | 183 | /// process) that share it. | ||
712 | 184 | /// | ||
713 | 185 | /// *Returns* | ||
714 | 186 | /// boost::unordered_map<uint, std::vector<uint> > | ||
715 | 187 | /// The map from dofs to list of processes | ||
716 | 188 | const boost::unordered_map<uint, std::vector<uint> >& shared_dofs() const; | ||
717 | 189 | |||
718 | 190 | /// Return set of all neighbouring processes. | ||
719 | 191 | /// | ||
720 | 192 | /// *Returns* | ||
721 | 193 | /// boost::unordered_set<uint> | ||
722 | 194 | /// The set of processes | ||
723 | 195 | const std::set<uint>& neighbours() const; | ||
724 | 196 | |||
725 | 181 | /// Local-to-global mapping of dofs on a cell | 197 | /// Local-to-global mapping of dofs on a cell |
726 | 182 | /// | 198 | /// |
727 | 183 | /// *Arguments* | 199 | /// *Arguments* |
728 | @@ -346,6 +362,12 @@ | |||
729 | 346 | // this process | 362 | // this process |
730 | 347 | boost::unordered_map<uint, uint> _off_process_owner; | 363 | boost::unordered_map<uint, uint> _off_process_owner; |
731 | 348 | 364 | ||
732 | 365 | // List of processes that share a given dof | ||
733 | 366 | boost::unordered_map<uint, std::vector<uint> > _shared_dofs; | ||
734 | 367 | |||
735 | 368 | // Neighbours (processes that we share dofs with) | ||
736 | 369 | std::set<uint> _neighbours; | ||
737 | 370 | |||
738 | 349 | // True iff sub-dofmap (a view, i.e. not collapsed) | 371 | // True iff sub-dofmap (a view, i.e. not collapsed) |
739 | 350 | bool _is_view; | 372 | bool _is_view; |
740 | 351 | 373 | ||
741 | 352 | 374 | ||
742 | === modified file 'dolfin/fem/DofMapBuilder.cpp' | |||
743 | --- dolfin/fem/DofMapBuilder.cpp 2011-11-18 12:14:50 +0000 | |||
744 | +++ dolfin/fem/DofMapBuilder.cpp 2012-02-29 14:31:44 +0000 | |||
745 | @@ -17,9 +17,10 @@ | |||
746 | 17 | // | 17 | // |
747 | 18 | // Modified by Niclas Jansson 2009. | 18 | // Modified by Niclas Jansson 2009. |
748 | 19 | // Modified by Garth N. Wells 2010. | 19 | // Modified by Garth N. Wells 2010. |
749 | 20 | // Modified by Joachim B Haga, 2012. | ||
750 | 20 | // | 21 | // |
751 | 21 | // First added: 2008-08-12 | 22 | // First added: 2008-08-12 |
753 | 22 | // Last changed: 2011-11-14 | 23 | // Last changed: 2012-02-29 |
754 | 23 | 24 | ||
755 | 24 | #include <ufc.h> | 25 | #include <ufc.h> |
756 | 25 | #include <boost/random.hpp> | 26 | #include <boost/random.hpp> |
757 | @@ -87,18 +88,20 @@ | |||
758 | 87 | { | 88 | { |
759 | 88 | // Create data structures | 89 | // Create data structures |
760 | 89 | DofMapBuilder::set owned_dofs, shared_owned_dofs, shared_unowned_dofs; | 90 | DofMapBuilder::set owned_dofs, shared_owned_dofs, shared_unowned_dofs; |
761 | 91 | DofMapBuilder::vec_map shared_dof_processes; | ||
762 | 90 | 92 | ||
763 | 91 | // Computed owned and shared dofs (and owned and un-owned) | 93 | // Computed owned and shared dofs (and owned and un-owned) |
764 | 92 | compute_ownership(owned_dofs, shared_owned_dofs, shared_unowned_dofs, | 94 | compute_ownership(owned_dofs, shared_owned_dofs, shared_unowned_dofs, |
766 | 93 | dofmap, global_dofs, mesh); | 95 | shared_dof_processes, dofmap, global_dofs, mesh); |
767 | 94 | 96 | ||
768 | 95 | // Renumber dofs owned dofs and received new numbering for unowned shared dofs | 97 | // Renumber dofs owned dofs and received new numbering for unowned shared dofs |
769 | 96 | parallel_renumber(owned_dofs, shared_owned_dofs, shared_unowned_dofs, | 98 | parallel_renumber(owned_dofs, shared_owned_dofs, shared_unowned_dofs, |
771 | 97 | dofmap, mesh); | 99 | shared_dof_processes, dofmap, mesh); |
772 | 98 | } | 100 | } |
773 | 99 | //----------------------------------------------------------------------------- | 101 | //----------------------------------------------------------------------------- |
774 | 100 | void DofMapBuilder::compute_ownership(set& owned_dofs, set& shared_owned_dofs, | 102 | void DofMapBuilder::compute_ownership(set& owned_dofs, set& shared_owned_dofs, |
775 | 101 | set& shared_unowned_dofs, | 103 | set& shared_unowned_dofs, |
776 | 104 | vec_map& shared_dof_processes, | ||
777 | 102 | const DofMap& dofmap, | 105 | const DofMap& dofmap, |
778 | 103 | const DofMapBuilder::set& global_dofs, | 106 | const DofMapBuilder::set& global_dofs, |
779 | 104 | const Mesh& mesh) | 107 | const Mesh& mesh) |
780 | @@ -174,6 +177,7 @@ | |||
781 | 174 | { | 177 | { |
782 | 175 | const uint received_dof = recv_buffer[i]; | 178 | const uint received_dof = recv_buffer[i]; |
783 | 176 | const uint received_vote = recv_buffer[i + 1]; | 179 | const uint received_vote = recv_buffer[i + 1]; |
784 | 180 | |||
785 | 177 | if (shared_owned_dofs.find(received_dof) != shared_owned_dofs.end()) | 181 | if (shared_owned_dofs.find(received_dof) != shared_owned_dofs.end()) |
786 | 178 | { | 182 | { |
787 | 179 | // Move dofs with higher ownership votes from shared to shared but not owned | 183 | // Move dofs with higher ownership votes from shared to shared but not owned |
788 | @@ -191,6 +195,14 @@ | |||
789 | 191 | "compute mapping of degrees of freedom", | 195 | "compute mapping of degrees of freedom", |
790 | 192 | "Cannot decide on dof ownership; votes are equal"); | 196 | "Cannot decide on dof ownership; votes are equal"); |
791 | 193 | } | 197 | } |
792 | 198 | |||
793 | 199 | // Remember the sharing of the dof | ||
794 | 200 | shared_dof_processes[received_dof].push_back(src); | ||
795 | 201 | } | ||
796 | 202 | else if (shared_unowned_dofs.find(received_dof) != shared_unowned_dofs.end()) | ||
797 | 203 | { | ||
798 | 204 | // Remember the sharing of the dof | ||
799 | 205 | shared_dof_processes[received_dof].push_back(src); | ||
800 | 194 | } | 206 | } |
801 | 195 | } | 207 | } |
802 | 196 | } | 208 | } |
803 | @@ -240,6 +252,7 @@ | |||
804 | 240 | void DofMapBuilder::parallel_renumber(const set& owned_dofs, | 252 | void DofMapBuilder::parallel_renumber(const set& owned_dofs, |
805 | 241 | const set& shared_owned_dofs, | 253 | const set& shared_owned_dofs, |
806 | 242 | const set& shared_unowned_dofs, | 254 | const set& shared_unowned_dofs, |
807 | 255 | const vec_map& shared_dof_processes, | ||
808 | 243 | DofMap& dofmap, const Mesh& mesh) | 256 | DofMap& dofmap, const Mesh& mesh) |
809 | 244 | { | 257 | { |
810 | 245 | log(TRACE, "Renumber dofs for parallel dof map"); | 258 | log(TRACE, "Renumber dofs for parallel dof map"); |
811 | @@ -317,6 +330,17 @@ | |||
812 | 317 | } | 330 | } |
813 | 318 | } | 331 | } |
814 | 319 | 332 | ||
815 | 333 | // Insert the shared-dof-to-process mapping into the dofmap, renumbering as necessary | ||
816 | 334 | for (vec_map::const_iterator it = shared_dof_processes.begin(); it != shared_dof_processes.end(); ++it) | ||
817 | 335 | { | ||
818 | 336 | boost::unordered_map<uint, uint>::const_iterator new_index = old_to_new_dof_index.find(it->first); | ||
819 | 337 | if (new_index == old_to_new_dof_index.end()) | ||
820 | 338 | dofmap._shared_dofs.insert(*it); | ||
821 | 339 | else | ||
822 | 340 | dofmap._shared_dofs.insert(std::make_pair(new_index->second, it->second)); | ||
823 | 341 | dofmap._neighbours.insert(it->second.begin(), it->second.end()); | ||
824 | 342 | } | ||
825 | 343 | |||
826 | 320 | // Build new dof map | 344 | // Build new dof map |
827 | 321 | for (CellIterator cell(mesh); !cell.end(); ++cell) | 345 | for (CellIterator cell(mesh); !cell.end(); ++cell) |
828 | 322 | { | 346 | { |
829 | 323 | 347 | ||
830 | === modified file 'dolfin/fem/DofMapBuilder.h' | |||
831 | --- dolfin/fem/DofMapBuilder.h 2011-11-24 17:51:44 +0000 | |||
832 | +++ dolfin/fem/DofMapBuilder.h 2012-02-29 14:31:44 +0000 | |||
833 | @@ -25,6 +25,7 @@ | |||
834 | 25 | 25 | ||
835 | 26 | #include <set> | 26 | #include <set> |
836 | 27 | #include <boost/unordered_set.hpp> | 27 | #include <boost/unordered_set.hpp> |
837 | 28 | #include <boost/unordered_map.hpp> | ||
838 | 28 | #include <dolfin/common/Set.h> | 29 | #include <dolfin/common/Set.h> |
839 | 29 | 30 | ||
840 | 30 | namespace ufc | 31 | namespace ufc |
841 | @@ -55,6 +56,8 @@ | |||
842 | 55 | 56 | ||
843 | 56 | typedef std::vector<dolfin::uint>::const_iterator vector_it; | 57 | typedef std::vector<dolfin::uint>::const_iterator vector_it; |
844 | 57 | 58 | ||
845 | 59 | typedef boost::unordered_map<uint, std::vector<uint> > vec_map; | ||
846 | 60 | |||
847 | 58 | public: | 61 | public: |
848 | 59 | 62 | ||
849 | 60 | static void build(DofMap& dofmap, const Mesh& dolfin_mesh, | 63 | static void build(DofMap& dofmap, const Mesh& dolfin_mesh, |
850 | @@ -70,6 +73,7 @@ | |||
851 | 70 | 73 | ||
852 | 71 | static void compute_ownership(set& owned_dofs, set& shared_owned_dofs, | 74 | static void compute_ownership(set& owned_dofs, set& shared_owned_dofs, |
853 | 72 | set& shared_unowned_dofs, | 75 | set& shared_unowned_dofs, |
854 | 76 | vec_map& shared_dof_processes, | ||
855 | 73 | const DofMap& dofmap, | 77 | const DofMap& dofmap, |
856 | 74 | const DofMapBuilder::set& global_dofs, | 78 | const DofMapBuilder::set& global_dofs, |
857 | 75 | const Mesh& mesh); | 79 | const Mesh& mesh); |
858 | @@ -77,6 +81,7 @@ | |||
859 | 77 | static void parallel_renumber(const set& owned_dofs, | 81 | static void parallel_renumber(const set& owned_dofs, |
860 | 78 | const set& shared_owned_dofs, | 82 | const set& shared_owned_dofs, |
861 | 79 | const set& shared_unowned_dofs, | 83 | const set& shared_unowned_dofs, |
862 | 84 | const vec_map& shared_dof_processes, | ||
863 | 80 | DofMap& dofmap, const Mesh& mesh); | 85 | DofMap& dofmap, const Mesh& mesh); |
864 | 81 | 86 | ||
865 | 82 | /// Compute set of global dofs (e.g. Reals associated with global | 87 | /// Compute set of global dofs (e.g. Reals associated with global |
866 | 83 | 88 | ||
867 | === modified file 'dolfin/fem/GenericDofMap.h' | |||
868 | --- dolfin/fem/GenericDofMap.h 2011-09-27 07:14:59 +0000 | |||
869 | +++ dolfin/fem/GenericDofMap.h 2012-02-29 14:31:44 +0000 | |||
870 | @@ -15,8 +15,10 @@ | |||
871 | 15 | // You should have received a copy of the GNU Lesser General Public License | 15 | // You should have received a copy of the GNU Lesser General Public License |
872 | 16 | // along with DOLFIN. If not, see <http://www.gnu.org/licenses/>. | 16 | // along with DOLFIN. If not, see <http://www.gnu.org/licenses/>. |
873 | 17 | // | 17 | // |
874 | 18 | // Modified by Joachim B Haga, 2012 | ||
875 | 19 | // | ||
876 | 18 | // First added: 2010-05-26 | 20 | // First added: 2010-05-26 |
878 | 19 | // Last changed: 2011-02-23 | 21 | // Last changed: 2012-02-29 |
879 | 20 | 22 | ||
880 | 21 | #ifndef __GENERIC_DOF_MAP_H | 23 | #ifndef __GENERIC_DOF_MAP_H |
881 | 22 | #define __GENERIC_DOF_MAP_H | 24 | #define __GENERIC_DOF_MAP_H |
882 | @@ -108,6 +110,13 @@ | |||
883 | 108 | /// Return the set of dof indices | 110 | /// Return the set of dof indices |
884 | 109 | virtual boost::unordered_set<uint> dofs() const = 0; | 111 | virtual boost::unordered_set<uint> dofs() const = 0; |
885 | 110 | 112 | ||
886 | 113 | /// Return map from shared dofs to the processes (not including the current | ||
887 | 114 | /// process) that share it. | ||
888 | 115 | virtual const boost::unordered_map<uint, std::vector<uint> >& shared_dofs() const = 0; | ||
889 | 116 | |||
890 | 117 | /// Return set of all processes that share dofs with the current process. | ||
891 | 118 | virtual const std::set<uint>& neighbours() const = 0; | ||
892 | 119 | |||
893 | 111 | /// Re-number based on provided re-numbering map | 120 | /// Re-number based on provided re-numbering map |
894 | 112 | virtual void renumber(const std::vector<uint>& renumbering_map) = 0; | 121 | virtual void renumber(const std::vector<uint>& renumbering_map) = 0; |
895 | 113 | 122 | ||
896 | 114 | 123 | ||
897 | === modified file 'dolfin/fem/SystemAssembler.cpp' | |||
898 | --- dolfin/fem/SystemAssembler.cpp 2012-02-22 09:40:00 +0000 | |||
899 | +++ dolfin/fem/SystemAssembler.cpp 2012-02-29 14:31:44 +0000 | |||
900 | @@ -16,9 +16,10 @@ | |||
901 | 16 | // along with DOLFIN. If not, see <http://www.gnu.org/licenses/>. | 16 | // along with DOLFIN. If not, see <http://www.gnu.org/licenses/>. |
902 | 17 | // | 17 | // |
903 | 18 | // Modified by Anders Logg 2008-2011 | 18 | // Modified by Anders Logg 2008-2011 |
904 | 19 | // Modified by Joachim B Haga 2012 | ||
905 | 19 | // | 20 | // |
906 | 20 | // First added: 2009-06-22 | 21 | // First added: 2009-06-22 |
908 | 21 | // Last changed: 2011-11-14 | 22 | // Last changed: 2012-02-29 |
909 | 22 | 23 | ||
910 | 23 | #include <dolfin/log/dolfin_log.h> | 24 | #include <dolfin/log/dolfin_log.h> |
911 | 24 | #include <dolfin/common/Timer.h> | 25 | #include <dolfin/common/Timer.h> |
912 | @@ -168,20 +169,9 @@ | |||
913 | 168 | DirichletBC::Map boundary_values; | 169 | DirichletBC::Map boundary_values; |
914 | 169 | for (uint i = 0; i < bcs.size(); ++i) | 170 | for (uint i = 0; i < bcs.size(); ++i) |
915 | 170 | { | 171 | { |
918 | 171 | // Methods other than 'pointwise' are not robust in parallel since a vertex | 172 | bcs[i]->get_boundary_values(boundary_values); |
917 | 172 | // can have a bc applied, but the partition might not have a facet on the boundary. | ||
919 | 173 | if (MPI::num_processes() > 1 && bcs[i]->method() != "pointwise") | 173 | if (MPI::num_processes() > 1 && bcs[i]->method() != "pointwise") |
931 | 174 | { | 174 | bcs[i]->gather(boundary_values); |
921 | 175 | if (MPI::process_number() == 0) | ||
922 | 176 | { | ||
923 | 177 | warning("Dirichlet boundary condition method '%s' is not robust in parallel with symmetric assembly.", bcs[i]->method().c_str()); | ||
924 | 178 | //warning("Caution: 'on_boundary' does not work with 'pointwise' boundary conditions,"); | ||
925 | 179 | } | ||
926 | 180 | bcs[i]->get_boundary_values(boundary_values); | ||
927 | 181 | //bcs[i]->get_boundary_values(boundary_values, "pointwise"); | ||
928 | 182 | } | ||
929 | 183 | else | ||
930 | 184 | bcs[i]->get_boundary_values(boundary_values); | ||
932 | 185 | } | 175 | } |
933 | 186 | 176 | ||
934 | 187 | // Modify boundary values for incremental (typically nonlinear) problems | 177 | // Modify boundary values for incremental (typically nonlinear) problems |
This crashes the auto-adaptive- poisson demo, which is probably due to a bug in the adaptive solver (see below). I suspect that the solver fails to properly initialise the mesh, but the the initialisation was previously done by chance in the bc code.
*** ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- --- /answers. launchpad. net/dolfin ------- ------- ------- ------- ------- ------- ------- ------- ------- --- ------- ------- ------- ------- ------- ------- ------- ------- ------- ---
*** DOLFIN encountered an error. If you are not able to resolve this issue
*** using the information listed below, you can ask for help at
***
*** https:/
***
*** Remember to include the error message listed below and, if possible,
*** include a *minimal* running example to reproduce the error.
***
*** -------
*** Error: Unable to initialize mapping of degrees of freedom.
*** Reason: Missing entities of dimension 1. Try calling mesh.init(1).
*** Where: This error was encountered inside DofMap.cpp.
*** -------