Skip to content

GPU use on MPG's vera cluster

Hi all, @elena told me that you've got fargOCA running on an NVidia GPU (V100).

Our vera cluster has 3 execution nodes verag[001-003] (500 GB RAM and 4 Nvidia A100-40GB GPUs each)

I don't know much about GPU's but it might be worth trying to make it run here as well.

I tried building the code with cmake -DCMAKE_BUILD_TYPE=Release -DKokkos_ENABLE_CUDA=On -DKokkos_ARCH_VOLTA70=On -DKokkos_ENABLE_OPENMP=On -DKokkos_ENABLE_CUDA_LAMBDA=On -DFARGO_NO_PYTHON=On ../ which is the command Elena said you use on Licallo. I didn't expect it to work on my cluster without some tweaking but I tried anyway. I get:

-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- 	 Found OpenMP option -fopenmp.
-- Found MPI_C: /u/system/soft/SLE_15/packages/x86_64/intel_oneapi/2022.1/mpi/2021.5.1/lib/release/libmpi.so (found version "3.1") 
-- Found MPI: TRUE (found version "3.1") found components: C 
--  Will use Kokkos submodule.
-- Setting default Kokkos CXX standard to 17
-- Setting policy CMP0074 to use <Package>_ROOT variables
-- The project name is: Kokkos
-- Compiler Version: 11.4.120
-- kokkos_launch_compiler (/u/gpichierri/workspace/fargOCA/external/kokkos/bin/kokkos_launch_compiler) is enabled...
-- Using -std=c++1z for C++17 standard as feature
CMake Error at external/kokkos/cmake/kokkos_test_cxx_std.cmake:34 (MESSAGE):
  CMake wants to use -std=c++1z which is not supported by NVCC.  Using a more
  recent host compiler or a more recent CMake version might help.
Call Stack (most recent call first):
  external/kokkos/cmake/kokkos_test_cxx_std.cmake:76 (kokkos_set_cxx_standard_feature)
  external/kokkos/cmake/kokkos_tribits.cmake:231 (INCLUDE)
  external/kokkos/CMakeLists.txt:170 (KOKKOS_SETUP_BUILD_ENVIRONMENT)


-- Configuring incomplete, errors occurred!
See also "/u/gpichierri/workspace/fargOCA/build_master_d4377553_GPU/CMakeFiles/CMakeOutput.log".

Here's my setup:

$ module list
Currently Loaded Modulefiles:
 1) intel/21.5.0   2) impi/2021.5   3) anaconda/3/2021.11   4) hdf5-serial/1.12.0   5) boost-mpi/1.74   6) doxygen/1.8   7) cuda/11.4

Which version of nvcc are you using? I have

$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Sun_Aug_15_21:14:11_PDT_2021
Cuda compilation tools, release 11.4, V11.4.120
Build cuda_11.4.r11.4/compiler.30300941_0

Thanks!