MPI wrapper functions

Module containing custom MPI functions

geobipy.src.base.MPI.Bcast(self, world, root=0, dtype=None, ndim=None, shape=None)

Broadcast a string or a numpy array

Broadcast a string or a numpy array from a root rank to all ranks in an MPI communicator. Must be called collectively. In order to call this function collectively, the variable ‘self’ must be instantiated on every rank. See the example section for more details.

Parameters:

self (str or numpy.ndarray) – A string or numpy array to broadcast from root.
world (mpi4py.MPI.Comm) – MPI parallel communicator.
root (int, optional) – The MPI rank to broadcast from. Default is 0.

Returns:

out – The broadcast object on every rank.

Return type:

same type as self

Raises:

TypeError – If self is a list, tell the user to use the specific Bcast_list function. While it has less code and seems like it might be faster, MPI actually pickles the list, broadcasts that binary stream, and unpickles on the other side. For a large number of lists, this can take a long time. This way, the user is made aware of the time benefits of using numpy arrays.

Examples

Given a numpy array instantiated on the master rank 0, in order to broadcast it, I must also instantiate a variable with the same name on all other ranks.

>>> import numpy as np
>>> from mpi4py import MPI
>>> from geobipy.src.base import MPI as myMPI
>>> world = MPI.COMM_WORLD
>>> if world.rank == 0:
>>>     x=StatArray(arange(10))
>>> # Instantiate on all other ranks before broadcasting
>>> else:
>>>     x=None
>>> y = myMPI.Bcast(x, world)
>>>
>>> # A string example
>>> if (world.rank == 0):
>>>     s = 'some string'  # This may have been read in through an input file for production code
>>> else:
>>>     s = ''
>>> s = myMPI.Bcast(s,world)

geobipy.src.base.MPI.Bcast_1int(self, world, root=0)

Broadcast a single integer

In order to broadcast scalar values using the faster numpy approach, the value must cast into a 1D ndarray. Must be called collectively.

Parameters:

self (int) – The integer to broadcast.
world (mpi4py.MPI.Comm) – MPI parallel communicator.
root (int, optional) – The MPI rank to broadcast from. Default is 0.

Returns:

out – The broadcast integer.

Return type:

int

Examples

Given an integer instantiated on the master rank 0, in order to broadcast it, I must also instantiate a variable with the same name on all other ranks.

>>> import numpy as np
>>> from mpi4py import MPI
>>> from geobipy.src.base import MPI as myMPI
>>> world = MPI.COMM_WORLD
>>> if world.rank == 0:
>>>     i = 5
>>> # Instantiate on all other ranks before broadcasting
>>> else:
>>>     i=None
>>> i = myMPI.Bcast(i, world)

geobipy.src.base.MPI.Bcast_list(self, world, root=0)

Broadcast a list by pickling, sending, and unpickling. This is slower than using numpy arrays and uppercase (Bcast) mpi4py routines. Must be called collectively.

Parameters:

self (list) – A list to broadcast.
world (mpi4py.MPI.Comm) – MPI parallel communicator.
root (int, optional) – The MPI rank to broadcast from. Default is 0.

Returns:

out – The broadcast list on every MPI rank.

Return type:

list

geobipy.src.base.MPI.Irecv(source, world, dtype=None, ndim=None, shape=None): Irecv a numpy array. Auto determines data type and shape. Must be accompanied by Isend on the source rank.

geobipy.src.base.MPI.IrecvFromLeft(world, wrap=True): Irecv an array from the rank left of world.rank.

geobipy.src.base.MPI.IrecvFromRight(world, wrap=True): IRecv an array from the rank right of world.rank.

geobipy.src.base.MPI.Irecv_1int(source, world)

Recv a single integer. Must be accompanied by Isend_1int on the source rank.

Parameters:

self (int) – Integer to Recv
source (int) – Receive from this rank.
world (mpi4py.MPI.Comm) – MPI parallel communicator.

Returns:

out – The received integer.

Return type:

int

geobipy.src.base.MPI.Isend(self, dest, world, dtype=None, ndim=None, shape=None): Isend a numpy array. Auto determines data type and shape. Must be accompanied by Irecv on the dest rank.

geobipy.src.base.MPI.IsendToLeft(self, world, wrap=True): ISend an array to the rank left of world.rank.

geobipy.src.base.MPI.IsendToRight(self, world, wrap=True): ISend an array to the rank left of world.rank.

geobipy.src.base.MPI.Isend_1int(self, dest, world)

Send a single integer. Must be accompanied by Irecv_1int on the dest rank.

Parameters:

self (int) – The integer to Send.
dest (int) – Rank to receive
world (mpi4py.MPI.Comm) – MPI parallel communicator.

Returns:

out – The sent integer.

Return type:

int

geobipy.src.base.MPI.Scatterv(self, starts, chunks, world, axis=0, root=0)

ScatterV an array to all ranks in an MPI communicator.

Each rank gets a chunk defined by a starting index and chunk size. Must be called collectively. The ‘starts’ and ‘chunks’ must be available on every MPI rank. Must be called collectively. See the example for more details.

Parameters:

self (numpy.ndarray) – A numpy array to broadcast from root.
starts (array of ints) – 1D array of ints with size equal to the number of MPI ranks. Each element gives the starting index for a chunk to be sent to that core. e.g. starts[0] is the starting index for rank = 0.
chunks (array of ints) – 1D array of ints with size equal to the number of MPI ranks. Each element gives the size of a chunk to be sent to that core. e.g. chunks[0] is the chunk size for rank = 0.
world (mpi4py.MPI.Comm) – MPI parallel communicator.
axis (int, optional) – Axis along which to Scatterv to the ranks if self is a 2D numpy array. Default is 0
root (int, optional) – The MPI rank to broadcast from. Default is 0.

Returns:

out – A chunk of self on each MPI rank with size chunk[world.rank].

Return type:

numpy.ndarray

Examples

>>> import numpy as np
>>> from mpi4py import MPI
>>> from geobipy.src.base import MPI as myMPI
>>> world = MPI.COMM_WORLD
>>> # Globally define a size N
>>> N = 1000
>>> # On each rank, compute the starting indices and chunk size for the given world.
>>> starts,chunks=loadBalance_shrinkingArrays(N, world.size)
>>> # Create an array on the master rank
>>> if (world.rank == 0):
>>>     x = arange(N)
>>> else:
>>>     x = None
>>> # Scatter the array x among ranks.
>>> myChunk = myMPI.Scatterv(x, starts, chunks, world, root=0)

geobipy.src.base.MPI.Scatterv_list(self, starts, chunks, world, root=0)

Scatterv a list by pickling, sending, receiving, and unpickling. This is slower than using numpy arrays and uppercase (Scatterv) mpi4py routines. Must be called collectively.

Parameters:

self (list) – A list to scatterv.
starts (array of ints) – 1D array of ints with size equal to the number of MPI ranks. Each element gives the starting index for a chunk to be sent to that core. e.g. starts[0] is the starting index for rank = 0.
chunks (array of ints) – 1D array of ints with size equal to the number of MPI ranks. Each element gives the size of a chunk to be sent to that core. e.g. chunks[0] is the chunk size for rank = 0.
world (mpi4py.MPI.Comm) – MPI parallel communicator.
root (int, optional) – The MPI rank to broadcast from. Default is 0.

Returns:

out – A chunk of self on each MPI rank with size chunk[world.rank].

Return type:

list

geobipy.src.base.MPI.Scatterv_numpy(self, starts, chunks, dtype, world, axis=0, root=0)

ScatterV a numpy array to all ranks in an MPI communicator.

Each rank gets a chunk defined by a starting index and chunk size. Must be called collectively. The ‘starts’ and ‘chunks’ must be available on every MPI rank. See the example for more details. Must be called collectively.

Parameters:

self (numpy.ndarray) – A numpy array to broadcast from root.
starts (array of ints) – 1D array of ints with size equal to the number of MPI ranks. Each element gives the starting index for a chunk to be sent to that core. e.g. starts[0] is the starting index for rank = 0.
chunks (array of ints) – 1D array of ints with size equal to the number of MPI ranks. Each element gives the size of a chunk to be sent to that core. e.g. chunks[0] is the chunk size for rank = 0.
dtype (type) – The type of the numpy array being scattered. Must exist on all ranks.
world (mpi4py.MPI.Comm) – MPI parallel communicator.
axis (int, optional) – Axis along which to Scatterv to the ranks if self is a 2D numpy array. Default is 0
root (int, optional) – The MPI rank to broadcast from. Default is 0.

Returns:

out – A chunk of self on each MPI rank with size chunk[world.rank].

Return type:

numpy.ndarray

geobipy.src.base.MPI.banner(world, aStr=None, end='\n', rank=0)

Prints a String with Separators above and below

Parameters:

world (mpi4py.MPI.Comm) – MPI parallel communicator.
aStr (str) – A string to print.
end (str) – string appended after the last value, default is a newline.
rank (int) – The rank to print from, default is the master rank, 0.

geobipy.src.base.MPI.bcastType(self, world, root=0)

Gets the type of an object and broadcasts it to every rank in an MPI communicator.

Adaptively broadcasts the type of an object. Must be called collectively.

Parameters:

self (object) – For numpy arrays and numpy scalars, a numpy data type will be broadcast. For arbitrary objects, the attached __class__.__name__ will be broadcast. For lists, the data type will be list
world (mpi4py.MPI.Comm) – MPI parallel communicator.
root (int, optional) – The MPI rank to broadcast from. Default is 0.

Returns:

out – The data type broadcast to every rank including the rank broadcast from.

Return type:

object

geobipy.src.base.MPI.helloWorld(world)

Print hello from every rank in an MPI communicator

Parameters:: world (mpi4py.MPI.Comm) – MPI parallel communicator.

geobipy.src.base.MPI.loadBalance1D_shrinkingArrays(N, nChunks)

Splits the length of an array into a number of chunks. Load balances the chunks in a shrinking arrays fashion.

Given a length N, split N up into nChunks and return the starting index and size of each chunk. After being split equally among the chunks, the remainder is split so that the first remainder chunks get +1 in size. e.g. N=10, nChunks=3 would return starts=[0,4,7] chunks=[4,3,3]

Parameters:

N (int) – A size to split into chunks.
nChunks (int) – The number of chunks to split N into.

Returns:

starts (ndarray of ints) – The starting indices of each chunk.
chunks (ndarray of ints) – The size of each chunk.

geobipy.src.base.MPI.loadBalance3D_shrinkingArrays(shape, nChunks)

Splits three dimensions among nChunks.

The number of chunks honours the relative difference in the values of shape. e.g. if shape is [600, 600, 300], then the number of chunks will be larger for the first two dimensions, and less for the third. Once the chunks are obtained, the start indices and chunk sizes for each dimension are returned.

Parameters:

N (array_like) – A 3D shape to split.
nChunks (int) – The number of chunks to split shape into.

Returns:

starts (ndarray of ints) – The starting indices of each chunk.
chunks (ndarray of ints) – The size of each chunk.

geobipy.src.base.MPI.ordered_print(world, this, title=None)

Prints numbers from each rank in order of rank

This routine will print an item from each rank in order of rank. This routine is SLOW due to lots of communication, but is useful for illustration purposes, or debugging. Do not use this in production code! The title is used in a banner

Parameters:

world (mpi4py.MPI.Comm) – MPI parallel communicator.
this (array_like) – Variable to print, must exist on every rank in the communicator.
title (str, optional) – Creates a banner to separate output with a clear indication of what is being written.

geobipy.src.base.MPI.print(aStr='', end='\n', **kwargs)

Prints the str to sys.stdout and flushes the buffer so that printing is immediate

Parameters:

aStr (str) – A string to print.
end (str) – string appended after the last value, default is a newline.

geobipy.src.base.MPI.rankPrint(world, aStr='', end='\n', rank=0)

Prints only from the specified MPI rank

Parameters:

world (mpi4py.MPI.Comm) – MPI parallel communicator.
aStr (str) – A string to print.
end (str) – string appended after the last value, default is a newline.
rank (int) – The rank to print from, default is the master rank, 0.