BACKGROUND INFORMATION TO ExCALIBUR
ExCALIBUR is a Wave 2 SPF programme led by the Met Office and EPSRC along with the UK Atomic Energy Authority (UKAEA) and research councils, including the Natural Environment Research Council (NERC), the Medical Research Council (MRC) and the Science and Technologies Facilities Council (STFC). Please have a look at the ExCALIBUR website.
ExCALIBUR will meet this challenge by delivering research and innovative algorithmic development to redesign the high priority simulation codes to fully harness the power of future supercomputers across scientific and engineering applications. It will achieve this by bringing together an unprecedented range of UK domain experts, mathematicians and computational scientists who will identify common issues and opportunities in the high priority simulation codes and focus their combined scientific expertise and resources to accelerate toward interdisciplinary solutions. The programme objectives have been designed to specifically address the benefits sought:
1. Efficiency - The UK’s most important scientific simulation codes will be able to harness the power of the supercomputers of the mid-2020s resulting in an increase in scientific productivity for a given investment.
2. Capability – Capitalising on this efficiency will enable the UK to continue to push the boundaries of science across a wide range of fields delivering transformational change in capability.
3. Expertise – A new, forward-facing, interdisciplinary approach to RSE career development will position the next generation of UK software engineers at the cutting-edge of scientific supercomputing.
ExCALIBUR is built around four pillars: separation of concerns; co-design; data science; and investing in people. These pillars describe the fundamental principles that guide the development of research within the ExCALIBUR programme and are designed to ensure that the outcomes are future-proofed against the constantly evolving landscape of hardware design. It will be delivered through six main activities: the redesign of a core set of simulation codes (use cases) chosen to span a wide range of science domains; knowledge integration across the programme through widely applicable cross-cutting themes; application of learning from these activities to a second wave of use cases; exploratory research to identify and develop emerging high-performance algorithms in areas with significant potential impact; an interdisciplinary Research Software Engineer knowledge integration activity; and an annual capital investment to support the development of novel test beds to enable co-development with industry.
The ExCALIBUR programme is funded until 31 March 2025.
DDWG TURBULENCE AT THE EXASCALE
To better understand the opportunities and the challenges in computational fluid dynamics that will come with exascale computing, EPSRC and UKRI are funded a Design and Development Working Group (DDWG) targeting turbulent flow simulations at the exascale, a high priority area of research for the UK
This DDWG brings together communities from the UK Turbulence Consortium (UKTC) and the UK Consortium on Reacting Flows (UKCRF) to ensure a smooth transition to exascale computing, with the aim to develop transformative techniques for future-proofing their production simulation software ecosystems dedicated to the study of turbulent flows. Understanding, predicting and controlling turbulent flows is of central importance and a limiting factor to a vast range of industries. Many of the environmental and energy-related issues we face today cannot possibly be tackled without a better understanding of turbulence.
The project is aiming at re-engineering or extending the capabilities of four of their production and research flow solvers for exascale computing: Xcompact3d, OpenSBLI, uDALES and SENGA+. These open-source, well-established, community flow solvers are based on finite-difference methods on structured meshes and will be developed to meet the challenges associated with exascale computing while taking advantage of the significant opportunities afforded by exascale systems.
A key aim of this project is to leverage the well-established Domain Specific Language (DLS) framework OPS and the 2DECOMP&FFT library to allow Xcompact3d, OpenSBLI, uDALES and SENGA+ to run on large-scale heterogeneous computers. OPS was developed in the UK in the last ten years and it targets applications on multi-block structured meshes. It can currently generate code using CUDA, OPENACC/OPENMP5.0, OPENCL, SYCL/ONEAPI, HIP and their combinations with MPI. The OPS DSLs' capabilities will be extended in this project, specifically its code-generation tool-chain for robust, fail-safe parallel code generation. A related strand of work will use the 2DECOMP&FFT a Fortran-based library based on a 2D domain decomposition for spatially implicit numerical algorithms on monobloc structured meshes. The library includes a highly scalable and efficient interface to perform Fast Fourier Transforms (FFTs) and relies on MPI providing a user-friendly programming interface that hides communication details from application developers. 2DECOMP&FFT will be completely redesigned for a use on heterogeneous supercomputers (CPUs and GPUS from different vendors) using a hybrid strategy.
The project will also combine exascale-ready coupling interfaces, UQ capabilities, I/O & visualisation tools to our flow solvers, as well as machine learning based algorithms, to address some of the key challenges and opportunities identified by the DDWG on turbulence at the exascale. This will be done in collaboration with several of the recently funded ExCALIBUR cross-cutting projects.
The project will focus on four high-priority use cases (one for each solver), defined as high quality, high impact research made possible by a step-change in simulation performance. The use cases will focus on wind energy, green aviation, air quality and net-zero combustion. Exascale computing will be a game changer in these areas and will contribute to make the UK a greener nation (The UK commits to net zero carbon emissions by 2050). The use cases will be used to demonstrate the potential of the re-designed flow solvers based on OPS and 2DECOMP&FFT, for a wide range of hardware and parallel paradigms.
ExCALIBUR is a Wave 2 SPF programme led by the Met Office and EPSRC along with the UK Atomic Energy Authority (UKAEA) and research councils, including the Natural Environment Research Council (NERC), the Medical Research Council (MRC) and the Science and Technologies Facilities Council (STFC). Please have a look at the ExCALIBUR website.
ExCALIBUR will meet this challenge by delivering research and innovative algorithmic development to redesign the high priority simulation codes to fully harness the power of future supercomputers across scientific and engineering applications. It will achieve this by bringing together an unprecedented range of UK domain experts, mathematicians and computational scientists who will identify common issues and opportunities in the high priority simulation codes and focus their combined scientific expertise and resources to accelerate toward interdisciplinary solutions. The programme objectives have been designed to specifically address the benefits sought:
1. Efficiency - The UK’s most important scientific simulation codes will be able to harness the power of the supercomputers of the mid-2020s resulting in an increase in scientific productivity for a given investment.
2. Capability – Capitalising on this efficiency will enable the UK to continue to push the boundaries of science across a wide range of fields delivering transformational change in capability.
3. Expertise – A new, forward-facing, interdisciplinary approach to RSE career development will position the next generation of UK software engineers at the cutting-edge of scientific supercomputing.
ExCALIBUR is built around four pillars: separation of concerns; co-design; data science; and investing in people. These pillars describe the fundamental principles that guide the development of research within the ExCALIBUR programme and are designed to ensure that the outcomes are future-proofed against the constantly evolving landscape of hardware design. It will be delivered through six main activities: the redesign of a core set of simulation codes (use cases) chosen to span a wide range of science domains; knowledge integration across the programme through widely applicable cross-cutting themes; application of learning from these activities to a second wave of use cases; exploratory research to identify and develop emerging high-performance algorithms in areas with significant potential impact; an interdisciplinary Research Software Engineer knowledge integration activity; and an annual capital investment to support the development of novel test beds to enable co-development with industry.
The ExCALIBUR programme is funded until 31 March 2025.
DDWG TURBULENCE AT THE EXASCALE
To better understand the opportunities and the challenges in computational fluid dynamics that will come with exascale computing, EPSRC and UKRI are funded a Design and Development Working Group (DDWG) targeting turbulent flow simulations at the exascale, a high priority area of research for the UK
This DDWG brings together communities from the UK Turbulence Consortium (UKTC) and the UK Consortium on Reacting Flows (UKCRF) to ensure a smooth transition to exascale computing, with the aim to develop transformative techniques for future-proofing their production simulation software ecosystems dedicated to the study of turbulent flows. Understanding, predicting and controlling turbulent flows is of central importance and a limiting factor to a vast range of industries. Many of the environmental and energy-related issues we face today cannot possibly be tackled without a better understanding of turbulence.
The project is aiming at re-engineering or extending the capabilities of four of their production and research flow solvers for exascale computing: Xcompact3d, OpenSBLI, uDALES and SENGA+. These open-source, well-established, community flow solvers are based on finite-difference methods on structured meshes and will be developed to meet the challenges associated with exascale computing while taking advantage of the significant opportunities afforded by exascale systems.
A key aim of this project is to leverage the well-established Domain Specific Language (DLS) framework OPS and the 2DECOMP&FFT library to allow Xcompact3d, OpenSBLI, uDALES and SENGA+ to run on large-scale heterogeneous computers. OPS was developed in the UK in the last ten years and it targets applications on multi-block structured meshes. It can currently generate code using CUDA, OPENACC/OPENMP5.0, OPENCL, SYCL/ONEAPI, HIP and their combinations with MPI. The OPS DSLs' capabilities will be extended in this project, specifically its code-generation tool-chain for robust, fail-safe parallel code generation. A related strand of work will use the 2DECOMP&FFT a Fortran-based library based on a 2D domain decomposition for spatially implicit numerical algorithms on monobloc structured meshes. The library includes a highly scalable and efficient interface to perform Fast Fourier Transforms (FFTs) and relies on MPI providing a user-friendly programming interface that hides communication details from application developers. 2DECOMP&FFT will be completely redesigned for a use on heterogeneous supercomputers (CPUs and GPUS from different vendors) using a hybrid strategy.
The project will also combine exascale-ready coupling interfaces, UQ capabilities, I/O & visualisation tools to our flow solvers, as well as machine learning based algorithms, to address some of the key challenges and opportunities identified by the DDWG on turbulence at the exascale. This will be done in collaboration with several of the recently funded ExCALIBUR cross-cutting projects.
The project will focus on four high-priority use cases (one for each solver), defined as high quality, high impact research made possible by a step-change in simulation performance. The use cases will focus on wind energy, green aviation, air quality and net-zero combustion. Exascale computing will be a game changer in these areas and will contribute to make the UK a greener nation (The UK commits to net zero carbon emissions by 2050). The use cases will be used to demonstrate the potential of the re-designed flow solvers based on OPS and 2DECOMP&FFT, for a wide range of hardware and parallel paradigms.