MPI Programming

• MPICH - How It Works
• A Hello World Program
• MPI Communication

MPICH - How It Works

MPI stands for "Message Passing Interface." It is a standardized protocol for communicating between processes on a collection of machines called a cluster.

The implementation of MPI we use is called MPICH. It comes with a set of tools and libraries providing a full implementation of the MPI specification. The tools included allow you to run MPICH programs on any number of nodes in the cluster. For the purposes of this presentation, MPICH programs running on each node will be called "children."

In MPI, there is no true notion of a master/slave relationship. Any MPICH child can communicate with any other child running in the cluster. Children are given numbered rankings when MPICH starts up. These rankings are the essentially the identity of the children. The first rank is 0. This means you can have child 3 communicate with child 19.

A Hello World Program

/* helloworld.c */

#include <stdio.h>

/* You MUST include this for the MPI_* functions */
#include "mpi.h"

int main(int argc, char **argv) {
   int rank;
   char host[150];
   int namelen;

   /* Initialize MPI. This handles mpich-specific command line arguments */
   MPI_Init(&argc, &argv);

   /* Get my rank. My rank number gets stored in the 'rank' variable */
   MPI_Comm_rank(MPI_COMM_WORLD, &rank);

   /* Look up what computer I am running on. Store it in 'host' */
   MPI_Get_processor_name(host,&namelen);

   printf("Hello world (Rank: %d / Host: %s)\n", rank, host);
   fflush(stdout);

   /* Finalize: Close connections to the other children, clean up memory
    * the MPI library has allocated, etc */
   MPI_Finalize();
   return 0;
}
			

Compiling and Running helloworld.c

Now that we have our first MPI program source code, what do we do now? We need to compile it. MPICH comes with a program called mpicc which you will want to use to compile the program.

 mpicc -o hellowowrld helloworld.c 

If there were any errors from the compiler, you'll need to fix them (obviously) before you can move on. We need to run the program to make sure it works! This requires us to use the mpirun command. This command takes a few command-line options you'll want to know about:

-np {NUMPROCS}

Specify how many processes to run

-allcpus

Use all available CPUs on every node

We have a 24-node cluster of dual-processor computers. This means there is a total of 48 cpus available. So specificying -allcpus is the same as -np 48.

Back to the task at hand: How do we run an MPICH program? I want to run my hello world program on 6 computers, so I'll type this:

 mpirun -np 6 helloworld 

This starts up 6 helloworld processes on the cluster. The machines it picks comes from a hosts list on the master node. It uses a round-robin technique to select machines to run processes on.

The output of helloworld is as follows:

Hello world (Rank: 0 / Host: master.bwee01.rit.edu)
Hello world (Rank: 2 / Host: n02.bwee01.rit.edu)
Hello world (Rank: 3 / Host: n03.bwee01.rit.edu)
Hello world (Rank: 4 / Host: n04.bwee01.rit.edu)
Hello world (Rank: 5 / Host: n05.bwee01.rit.edu)
Hello world (Rank: 1 / Host: n01.bwee01.rit.edu)

MPICH Communication

The hello world program was simple enough, right? The next thing you'll need to know how to do is interprocess communication. This is done using the MPI_Send and MPI_Recv functions. Let's slightly modify the helloworld program to have all the non-rank 0 machines send a message to the rank 0 process.

/* networld.c */
#include <stdio.h>
#include "mpi.h"

int main(int argc, char **argv) {
   int rank;
   char host[150];
   int namelen;

   /* Message tag */
   int tag = 1;

   MPI_Init(&argc, &argv);
   MPI_Comm_rank(MPI_COMM_WORLD, &rank);
   MPI_Get_processor_name(host,&namelen);

   printf("Hello world (Rank: %d / Host: %s)\n", rank, host);
   fflush(stdout);
   if (rank == 0) {
      int numprocs, x;
      char msg[50];
      MPI_Status status;

      MPI_Comm_size(MPI_COMM_WORLD, &numprocs);
      for (x = 1; x < numprocs; x++) {
         MPI_Recv(msg, 50, MPI_CHARACTER, MPI_ANY_SOURCE, tag, MPI_COMM_WORLD, &status);
         printf("Msg from %d: '%s'\n", status.MPI_SOURCE, msg);
         fflush(stdout);
      }
   } else {
      char msg[50];

      snprintf(msg, 50, "Hello from node rank %d.", rank);
      MPI_Send(msg, 50, MPI_CHARACTER, 0, tag, MPI_COMM_WORLD);
   }

   MPI_Finalize();
   return 0;
}
         

MPICH Communication (Run it)

Once again, we'll need to compile it:

 mpicc -o networld networld.c 

Assuming you don't get any errors, you can now run it:

 mpirun -np 6 networld 

A sample output is as follows:

Hello world (Rank: 0 / Host: master.bwee01.rit.edu)
Msg from 1: 'Hello from node rank 1.'
Msg from 2: 'Hello from node rank 2.'
Msg from 3: 'Hello from node rank 3.'
Msg from 4: 'Hello from node rank 4.'
Msg from 5: 'Hello from node rank 5.'
Hello world (Rank: 4 / Host: n04.bwee01.rit.edu)
Hello world (Rank: 2 / Host: n02.bwee01.rit.edu)
Hello world (Rank: 3 / Host: n03.bwee01.rit.edu)
Hello world (Rank: 1 / Host: n01.bwee01.rit.edu)
Hello world (Rank: 5 / Host: n05.bwee01.rit.edu)
			

Communication is pretty simple. Documentation on both of these functions can be found on the MPICH homepage. The only point worth mentioning of these functions is the 'tag' variable. This variable indicates the kind of message you are sending. MPI doesn't care what this number is, but if a child sends a message with tag 3 to rank 0, and rank 0 is waiting to receive data on tag 15, rank 0 will not get this message tagged 3 becuase it's wanting a message tagged 15.