Run Docker containers implementing OpenMPI on multiple nodes

# Based on the Dockefile for creating a Docker image that JupyterLab can use,
# Create a Docker container that can be used for athena++ development and execution.

# Based on the latest version of ubuntu 22.04.
FROM ubuntu:jammy-20240111

# Set bash as the default shell
ENV SHELL=/bin/bash

# Build with some basic utilities
RUN apt update && apt install -y \
        build-essential \
    python3-pip apt-utils vim \
    git git-lfs \
    curl unzip wget gnuplot \
        openmpi-bin libopenmpi-dev \
        openssh-client openssh-server

# alias python='python3'
RUN ln -s /usr/bin/python3 /usr/bin/python

# install python package to need
RUN pip install -U pip setuptools \
        && pip install numpy scipy h5py mpmath

# The following stuff is derived for horovod in docker.
# Allow OpenSSH to talk to containers without asking for confirmation
RUN mkdir -p /var/run/sshd
RUN cat /etc/ssh/ssh_config | grep -v StrictHostKeyChecking > /etc/ssh/ssh_config.new && \
    echo "    StrictHostKeyChecking no" >> /etc/ssh/ssh_config.new && \
    mv /etc/ssh/ssh_config.new /etc/ssh/ssh_config

# --allow-run-as-root
ENV OMPI_ALLOW_RUN_AS_ROOT=1
ENV OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1

# Create a working directory
WORKDIR /workdir

# command prompt
CMD ["/bin/bash"]

--------------------------------------------------------------------------
mpirun has detected an attempt to run as root.

Running as root is *strongly* discouraged as any mistake (e.g., in
defining TMPDIR) or bug can result in catastrophic damage to the OS
file system, leaving your system in an unusable state.

We strongly suggest that you run mpirun as a non-root user.

You can override this protection by adding the --allow-run-as-root option
to the cmd line or by setting two environment variables in the following way:
the variable OMPI_ALLOW_RUN_AS_ROOT=1 to indicate the desire to override this
protection, and OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1 to confirm the choice and
add one more layer of certainty that you want to do so.
We reiterate our advice against doing so - please proceed at your own risk.
--------------------------------------------------------------------------

$ sudo docker build -t openssh ./

sudo docker run -it --rm --net=host \
-v /ext/nfs/ssh:/root/.ssh \
-v /ext/nfs/athena++:/workdir \
openmpi:latest

xxx.xxx.xx.xx:/ext/nfs	/mnt/nfs2	nfs

sudo docker run -it --rm --net=host \
-v /mnt/nfs2/ssh:/root/.ssh \
-v /mnt/nfs2/athena++:/workdir \
openmpi:latest

#include <stdio.h>
#include "mpi.h"
 
int main( int argc, char *argv[] )
{
    int     rank, size, len;
    char    name[MPI_MAX_PROCESSOR_NAME];
 
    MPI_Init( &argc, &argv );
    MPI_Comm_rank( MPI_COMM_WORLD, &rank );
    MPI_Comm_size( MPI_COMM_WORLD, &size );
    MPI_Get_processor_name( name, &len );
    name[len] = '\0';
 
    printf( "Hello world: rank %d of %d running on %s\n", rank, size, name );
 
    MPI_Finalize();
    return 0;
}

# pwd
/workdir/tmp
# mpicc -o hello hello.c

# mpirun -n 4 ./hello
Authorization required, but no authorization protocol specified
Authorization required, but no authorization protocol specified
Hello world: rank 1 of 4 running on europe
Hello world: rank 2 of 4 running on europe
Hello world: rank 3 of 4 running on europe
Hello world: rank 0 of 4 running on europe

# /usr/sbin/sshd -p 12345

# cat hosts.txt
europe
ganymede

# mpirun -hostfile hosts.txt -mca plm_rsh_args "-p 12345" -n 8 ./hello
Authorization required, but no authorization protocol specified
Authorization required, but no authorization protocol specified
Authorization required, but no authorization protocol specified
Authorization required, but no authorization protocol specified
Hello world: rank 2 of 8 running on europe
Hello world: rank 3 of 8 running on europe
Hello world: rank 0 of 8 running on europe
Hello world: rank 1 of 8 running on europe
Hello world: rank 5 of 8 running on ganymede
Hello world: rank 6 of 8 running on ganymede
Hello world: rank 7 of 8 running on ganymede
Hello world: rank 4 of 8 running on ganymede