In this contribution we look into the efficiency and scalability of our Lattice Boltzmann implementation Musubi when using OpenMP threads within an MPI parallel computation on Hawk. The Lattice Boltzmann methods enables explicit computation of incompressible flows and the mesh discretization can be automatically generated, even for complex geometries. The basic Lattice Boltzmann kernel is fairly simple and involves only few floating point operations for each lattice node. A simple loop over all lattice nodes each partition of the MPI parallel setup lends a straight forward loop parallelization with OpenMP. With increased core counts per compute node, the use of threads on the shared memory nodes is gaining importance, as it avoids overly small partitions with many outbound communications to neighboring partitions. We briefly discuss the hybrid parallelization of Musubi and investigate how the usage of OpenMP threads affects the performance when running simulations on the Hawk supercomputer at HLRS.

Background: The COVID-19 pandemic poses the risk of overburdening health care systems, and in particular intensive care units (ICUs). Non-pharmaceutical interventions (NPIs), ranging from wearing masks to (partial) lockdowns have been implemented as mitigation measures around the globe. However, especially severe NPIs are used with great caution due to their negative effects on the economy, social life and mental well-being. Thus, understanding the impact of the pandemic on ICU demand under alternative scenarios reflecting different levels of NPIs is vital for political decision-making on NPIs. The aim is to support political decision-making by forecasting COVID-19-related ICU demand under alternative scenarios of COVID-19 progression reflecting different levels of NPIs.

Methods: In this talk we will present our implementation of a spatial age-structured microsimulation model of the COVID-19 pandemic by extending the Susceptible-Exposed-Infectious-Recovered (SEIR) framework. The model accounts for regional variation in population age structure and in spatial diffusion pathways. In a first step, we calibrate the model by applying a genetic optimization algorithm against hospital data on ICU patients with COVID-19. In a second step, we forecast COVID-19-related ICU demand under alternative scenarios of COVID 19 progression reflecting different levels of NPIs. The third step is the automation of the procedure for the provision of weekly forecasts. The automated estimation of the model's parameters is done by means of Random-Forest regression.

Results: In the results section we will show the application of the model to Germany and demonstrate state-level forecasts over a 2-month period, which can be updated daily based on latest data on the progression of the pandemic.To illustrate the merits of our model, we present here "forecasts" of ICU demand for different stages of the pandemic during 2020 and 2021. Our forecasts for a quiet summer phase with low infection rates identified quite some variation in potential for relaxing NPIs across the federal states. By contrast, our forecasts during a phase of quickly rising infection numbers in autumn (second wave) suggested that all federal states should implement additional NPIs. However, the identified needs for additional NPIs varied again across federal states. In addition, our model suggests that during large infection waves ICU demand would quickly exceed supply, if there were no NPIs in place to contain the virus.

We have performed three dimensional particle-in-cell (PIC) plasma simulation to investigate the filamentary coherent structure so called blob/hole. Impurity ion transport by this structure is revealed. The impurity ion profile in the blob/hole structure becomes a dipole structure and this propagates with the blob/hole.

The performance of three dimensional PIC code on the new super computer build by NEC SX-Aurora TSUBASA at National Institute for Fusion Science will also be presented.

Molecular docking simulations are widely used in computational drug discovery to predict molecular interactions at close distances. Specifically, these simulations aim to predict the binding poses between a small molecule and a macromolecular target, both referred to as ligand and receptor, respectively. The purpose of drug discovery is to identify ligands that effectively inhibit the harmful function of a certain receptor. In that context, molecular docking simulations are critical, by using them, the time-consuming preliminary tasks consisting of identifying potential drug candidates can be significantly shortened. Subsequent wet lab experiments can be carried out using only a narrowed list of promising ligands, hence reducing the overall cost of experiments.

AutoDock is one of the most widely used software applications for molecular docking simulations. Its main engine is a Lamarckian Genetic Algorithm (LGA), which combines a genetic algorithm and a local-search method to explore several molecular poses. The prediction of the best pose is based on the score, which is a function that evaluates the free energy (kcal/mol) of a ligand-receptor system. AutoDock is characterized by nested loops with variable upper bounds and divergent control structures. Moreover, the time-intensive score evaluations are typically invoked a couple of million of times within each LGA run. Based on its computation intensity, AutoDock suffers from long execution runtimes, which are mainly attributed to its inability to leverage its embarrassing parallelism. In recent years, an OpenCL-based implementation of AutoDock has been developed to accelerate its executions on a variety of devices including multi-core CPUs, GPUs, and even FPGAs.

In this work, we present our experiences porting and optimizing the OpenCL-based AutoDock onto the SX-Aurora Vector Engine. The OpenCL code is composed of a host and device parts that are maintained in the NEC VEO version. As the API functions of OpenCL and VEOffload resemble each other, porting the host code was very smooth. While the device part was easily ported too, an extra effort was required to increase the performance on SX-Aurora. For this, we used hardware-specific techniques that involve: appropriate data types for wider vectors, leveraging the multiple cores on the SX-Aurora, pushing outer into inner loops in score calculations and local search and using multi-process VEO to overcome OpenMP limitations in NUMA mode. Our evaluations were done on VE10B and VE20B models and compared to modern multicore CPUs and GPUs.

Structural analysis using the finite element method (FEM) is widely used in the field of engineering. Recently, NEC has introduced SX-Aurora TSUBASA, which has vector accelerator boards (Vector Engine, VE). One VE has a high-speed memory with a bandwidth of about 1.2 TB/s and eight high-performance vector cores. Each core has three Fused Multiply-Add (FMA) arithmetic units, each of which can perform 32 double-precision floating point element executions simultaneously. The host CPU is called VH.

FrontISTR is one of the highly parallelized open-source FEM software programs for nonlinear structural analysis. This software firstly generates a stiffness matrix using the FEM and then solves linear equations for the sparse matrix generated by FEM. The stiffness matrix generation is not suitable for VE because it cannot process data continuously, and it involves many integer operations.

There is an API for transferring data between VH and VE called Another/Alternative/Awesome VE Offloading (AVEO), which can be used to execute compute-intensive portions of an entire program on VE. We accelerate execution speed of the overall structural analysis program by running the linear equation solvers on VE using AVEO. We chose the JAD format as the sparse matrix storage format and the conjugate gradient (CG) method as the linear solver.

In this study, we evaluate accelerated FrontISTR in terms of the following three parts. (1) the generation time of stiffness matrices on VH, (2) the transfer time of sparse matrices and vectors from VH to VE using AVEO, and (3) the calculation time of linear equations by the CG method on VE. We describe the effectiveness of accelerated structural analysis execution on NEC SX-Aurora TSUBASA.