Jan, Frederik and I just released the first – somewhat stable – version of pupyMPI, a 100% pyre python implementation of the MPI standard. Or, as close to the standard as we saw fit.
Most python-mpi projects are bindings to some C implementation which gives a lot of strengths. It runs very very fast for one thing. So fast you can actually use it for real production if you want. In our opinion it’s not that useful, as most real applications will depend om some further systems and most real clusters will probably only allow you to run your C and FORTRAN stuff. They do however give developers a nice way to develop rapid prototypes, learn and play with MPI. pupyMPI boosts all threes while keeping the system to close to regular MPI, you can probably convert to regular MPI if you need the performance.
A quick example
Just to show how fast you can actually implementing programs in pupyMPI here is a quick example of a distributed program. It does a monte carlo pi simulation with some user defined number of simulations:
#!/usr/bin/python/
from mpi import MPI
import sys
from random import random
from math import sqrt
mpi = MPI()
world = mpi.MPI_COMM_WORLD
rank = world.rank()
try:
simulations = int(sys.argv[1])
except:
simulations = 1000000
per_rank = simulations / world.size()
hits = 0
for i in xrange(per_rank):
# Simulate a part hitting inside the unity circle.
x = random()
y = random()
if sqrt(x*x+y*y) < 1.0:
hits+=1
# Gather the sum of hits from each process at the process
# with rank 0
total_hits = world.reduce(hits, sum, root=0)
if rank == 0:
pi = float(total_hits)*4 / simulations
print "Estimating PI on %d nodes through %d simulations yield %f"
% (world.size(), simulations, pi)
mpi.finalize()
The output of several runs given below:
$ mpirun.py -c 2 monte_carlo_pi.py -- 1000 Estimating PI on 2 nodes through 1000 simulations yield 3.184000 $ mpirun.py -c 4 monte_carlo_pi.py -- 10000 Estimating PI on 4 nodes through 10000 simulations yield 3.135200 $ mpirun.py -c 8 monte_carlo_pi.py -- 100000 Estimating PI on 8 nodes through 100000 simulations yield 3.136560 $ mpirun.py -c 10 monte_carlo_pi.py -- 1000000 Estimating PI on 10 nodes through 1000000 simulations yield 3.144464 $ mpirun.py -c 10 monte_carlo_pi.py -- 10000000 Estimating PI on 10 nodes through 10000000 simulations yield 3.141069 $ mpirun.py -c 10 monte_carlo_pi.py -- 100000000 Estimating PI on 10 nodes through 100000000 simulations yield 3.141677
The above example have very little communication involved other than the final exchange of hits. We implemented a lot of different communication operations as you can see in the online documentation.
Performance
Don't expect much, as a pupyMPI program will normally run 15-20 times slower than the C equivalent. But hopefully it will prove a fine educational tool and maybe also be used for fast prototyping. If we get the time (and credit at school) we'll performance tune it to get within a factor 10 of the C version.