Using Fluidsim on Adastra (CINES)

We show in this directory (https://foss.heptapod.net/fluiddyn/fluidsim/-/tree/branch/default/doc/examples/clusters/adastra) how to use Fluidsim on Adastra. The main documentation for this HPC platform is here. We use modules produced by Spack.

Get a login and setup ssh

Get an account on https://www.edari.fr/.

Set the alias

alias sshadastra='ssh -X augier@adastra.cines.fr'

Setup Mercurial and clone fluidsim

Ask authorization to be able to clone the Fluidsim repository from https://foss.heptapod.net as explained here.

Install and setup Mercurial as explained here. Clone the Fluidsim repository in $HOME/dev.

Warning

The file .bashrc is not sourced at login so the user should do it to use pipx-installed applications.

mkdir ~/dev
cd ~/dev
. ~/.bashrc
hg clone https://foss.heptapod.net/fluiddyn/fluidsim
cd ~/dev/fluidsim/doc/examples/clusters/adastra

Setup a virtual environment

Execute the script setup_venv.sh.

./setup_venv.sh

#!/usr/bin/env bash
set -e

module purge
module load cpe/23.12
module load craype-x86-genoa
module load PrgEnv-gnu
module load gcc/13.2.0
module load cray-hdf5-parallel cray-fftw
module load cray-python

cd $HOME
python -m venv venv-fluidsim
.  ~/venv-fluidsim/bin/activate
pip install --upgrade pip

# install fluidsim and all dependencies from wheels!
pip install "fluidsim[fft,test]"

# fix/improve few packages (force recompilation)
pip install fluidfft --no-binary fluidfft -C setup-args=-Dnative=true --force-reinstall --no-cache-dir --no-deps -v

CC=mpicc pip install mpi4py --no-binary mpi4py --force-reinstall --no-cache-dir --no-deps -v
CC="mpicc" HDF5_MPI="ON" pip install h5py --no-binary=h5py --force-reinstall --no-cache-dir --no-deps -v

export LIBRARY_PATH=/opt/cray/pe/fftw/3.3.10.6/x86_genoa/lib
export CFLAGS="-I/opt/cray/pe/fftw/3.3.10.6/x86_genoa/include"
export PYFFTW_LIB_DIR="/opt/cray/pe/fftw/3.3.10.6/x86_genoa/lib"

pip install pyfftw --no-binary pyfftw --force-reinstall --no-cache-dir --no-deps -v

# install fluidfft pluggins
pip install fluidfft-fftw --no-binary fluidfft-fftw --force-reinstall --no-cache-dir --no-deps -v

Due to a bug in Meson (the build system used by few fluidfft pluggins, see https://github.com/mesonbuild/meson/pull/13619), we need to complete the installation:

module purge
module load cpe/23.12
module load craype-x86-genoa
module load PrgEnv-gnu
module load gcc/13.2.0
module load cray-hdf5-parallel cray-fftw
module load cray-python

export LIBRARY_PATH=/opt/cray/pe/fftw/3.3.10.6/x86_genoa/lib
export CFLAGS="-I/opt/cray/pe/fftw/3.3.10.6/x86_genoa/include"

. ~/venv-fluidsim/bin/activate

# --no-build-isolation because of the Meson bug

# because of --no-build-isolation
pip install meson-python ninja fluidfft-builder cython
cd ~/dev
hg clone https://github.com/paugier/meson.git
cd ~/dev/meson
hg up mpi-detection
pip install -e .
cd
#

pip install fluidfft-fftwmpi --no-binary fluidfft-fftwmpi --no-build-isolation --force-reinstall --no-cache-dir --no-deps -v

Install Fluidsim from source

module purge
module load cpe/23.12
module load craype-x86-genoa
module load PrgEnv-gnu
module load gcc/13.2.0
module load cray-hdf5-parallel cray-fftw
module load cray-python

. ~/venv-fluidsim/bin/activate

cd ~/dev/fluidsim
# update to the wanted commit
pip install . -v -C setup-args=-Dnative=true