Numerical models and HPC

The modelling team of APCG has developed computational facilities specialized for usage in atmospheric modelling and handling of big data related to that. This infrastructure currently includes one Dell R540 Rack Server (Dell 0VC7DK motherboard, 2 x Intel Xeon Gold 5218 CPU - 32 cores total, 201 GB RAM DDR4, 8TB HDDDell RAID hard disc, Linux Centos). Further expansion and upgrades have already been scheduled to address the increasing needs of the group.

Modelling activities are also supported by annual computational grants received by the Greek Research and Technology Network (GRNET) for computational access at the National HPC facility – ARIS.

The main models/model systems (i.e. software), operated systematically by the group, as part of the infrastructure,are listed below:

The European community Earth-System Model EC-Earth (http://www.ec-earth.org) which is a state-of-the art coupled climate model, developed and used for climate predictions and projections, by a European consortium of more than 20 operational and research institutions, including APCG group from NOA-IERSD. EC-Earth consists of an atmospheric general circulation model (GCM) based on the ECMWF Integrated Forecast System (IFS) model, with the H-TESSEL land-surface scheme and an ocean GCM from the Nucleus, for European Modeling of the Ocean (NEMO), with the Louvain-la-Neuve sea ice model (LIM). APCG is part of the development of components of TM5 (Krol et al., 2005), the global CTM, integrated into EC-Earth and it is used for simulating the atmospheric chemistry and transport of reactive gases and aerosols. EC-Earth3 also contains an advanced representation of the dust-iron cycle, which allows quantification of the contribution of different sources (dust, biomass burning, and anthropogenic aerosols) and processes (acidic and oxalate processing) to the soluble iron deposition into the ocean, as recently developed and published by our group (i.e., Myriokefalitakis et al., 2021).EC-Earth is an important modelling component of the DOMOS and ADIOS research projects of the APCG group, cited here.

COSMO-ART is a regional atmospheric model which couples online meteorology, chemistry, and aerosol dynamics (Fig. 2). It is the operational numerical weather prediction model of the German and other European weather services (Baldauf et al., 2011)a modified version CCLM (Rockel et al., 2008) of which, is used as a regional climate model. ART (Aerosols and Reactive Trace gases) is the chemistry extension of COSMO. Detailed descriptions of the model, the physic-chemical characteristics of the aerosol modes and the parameterizations of interactions of aerosols with radiation, temperature, cloud and ice condensation nuclei (CCN and IN), are given in Vogel et al. (2009), Bangert et al. (2011; 2012) and Rieger et al. (2014). The model domain chosen for the research activities of the APCG group is the extended area of Greece, centered in Athens. The setup of the model applications is given in Athanasopoulou et al (2017), largely based on the pilot study of the COSMO-ART application over the same domain (Athanasopoulou et al., 2014). Both references are given in the publication list of APCG, here. Computational time and space are systematically granted by the GRNET (HPC ARIS). COSMO-ART has been employed in the research project THESPIA 2, cited here.

The model system to simulate atmospheric conditions at the local scale is composed of the NWP model WRF and of the CTM EPISODE-CItyChem. WRF is a mesoscale numerical weather prediction model, which can serve both atmospheric research and operational forecasting needs. It features two dynamical cores, a data assimilation system and software architecture supporting parallel computation and system extensibility. For the atmospheric applications of the APCG group, the WRF model performs telescoping nesting, with the innermost domain over Athens (or any city of interest) and spatial resolution of 1 km. EPISODE-CityChem (fig. 3), is a Chemistry Transport Model to enable chemistry/transport simulations of reactive pollutants at the city scale. EPISODE is a Eulerian dispersion model developed at the Norwegian Institute for Air Research (NILU) appropriate for air quality studies at the local scale. The CityChem extension, developed at the Helmholtz-Zentrum (HZG), is designed for treating complex atmospheric chemistry in urban areas and improved representation of the near-field dispersion (Karl et al., 2019). The model is capable of simulating atmospheric chemistry in 1 km, reaching the 100 m spatial resolution, over and downstream industrial (points) and road network (lines) emissions. The WRF/CityChem model system is been used in several research projects where APCG participates, such as SMURBS, e-shape, RI-URBANS, EIFFEL, ARSINOE, all sited here. The configuration, application and results of EPISODE-CityChem model applications for Athens and Hamburg are given in the open access scientific article by Ramacher et al., 2021. Reference to be found here.

Supplementary, the group operates the TAPM atmospheric system, which simulates atmospheric conditions at the local scale, using a simpler but computationally more efficient approach. TAPM is a prognostic meteorology-dispersion model, capable of predicting the flows important to local-scale air pollution, utilizing a background of larger-scale meteorology provided by synoptic analyses. The model includes a chemistry mode with sulfur and fine particle chemistry (PM₁₀, NO_X, NO₂, O₃, SO₂, PM_2.5), which is composed of gas-phase photochemical reactions, based on the Generic Reaction Set and gas- and aqueous-phase chemical reactions for SO2 and particles. Wet and dry deposition is also simulated. This tool has enabled the APCG group to work towards identifying the interactions between climate change (climatic scenarios) and future air pollution at the municipality (or 1 by 1 km₂) level in the frame of National project ΠΕΣΠΚΑ, and to provide half-year NO₂ outputs to compare with MAX-DOAS retrievals (MAX3D project). Both projects can be found in the APCG project list, here.The configuration, application and results of the model applications for Athens are given in the open access scientific article by Grivas et al., 2020 and Gratsea et al., 2021. References to be found here.

The model system to predict plume dispersion from any (potential) industrial accident is composed by the WRF weather prediction model and FLEXPART Gaussian dispersion model. The meteorological forecast for any area of interest is supported by the GFS global weather predictions. FLEXPART is a Lagrangian transport and dispersion model, suitable for the simulation of a large range of atmospheric transport processes. Apart from transport and turbulent diffusion, it is able to simulate dry and wet deposition, decay, linear chemistry; it can be used in forward or backward mode, with defined sources or in a domain-filling setting. This system (Fig. 4) has been operating to predict plume dispersion from the real industrial accident of Aspropyrgos, as well as from potential accidents at industrial facilities outside the city of Ljubljana (Slovenia), Cork (Ireland) and Ritsona (Greece), in the frame of InfraStress and SMURBS projects, cited here.

A series of box-modeling tools are applied, built around the Thissio AMS, that can run in various configurations to simulate and study air-quality relevant species (such as O₃, CO), the secondary formation of inorganic aerosols (e.g., nitrates, ammonium), the oxidative capacity (i.e., OH and NO₃ radicals), and the acidity of particles. The aforementioned box-modeling tools use a number of state-of-the-art submodels, such as the thermodynamic module ISORROPIA, the size-resolved aerosol microphysics module M7, the kinetic preprocessor KPP software environment for solving chemical kinetics, etc.

To support city scale model applications. The modelling group of APCG has recently co-developed -in constant cooperation with the Chemistry Transport Group of Helmholtz-Zentrum (HZG)-and operates the UrbEm Hybrid Method to Derive High-Resolution Anthropogenic Emissions of air pollutants. UrbEm (Fig. 5) fully covers the emission inventory which is prerequisite for all urban air quality modeling and management efforts and serves as a starting point for such efforts in cities across Europe. The application of the UrbEm approach is realized as a software tool that is completely free of cost and open source distributed via GitHub (https://github.com/martinottopaul/UrbEm). The tool may run on different operating systems on single-core processors and need a minimum of 2 gigabyte RAM. It can be applied to any urban area in Europe and provides methodological homogeneity between different cities. Researchers of the APCG group have already applied the tool for selected pilot European cities, while they are constantly optimizing it to be further applied in more than 10 European cities in the frame of SMURBS, e-shape, RI-URBANS, EIFFEL, ARSINOE, ΠΕΣΠΚΑ, MAX3D projects, cited here. The description, application and results of UrbEm applications for Athens and Hamburg are given in the open access scientific article by Ramacher et al., 2021, reference to be found here.

Renewable energy monitoring, forecasting, management, planning and penetration into society is pursued through dedicated web tools and applications (https://solea.gr/) from the SOLEA part of the APCG group. SOLEA addresses the needs of a variety of public and private sector communities including electricity handling entities and renewable energy producers (fig 6). The operational running in real-time for the European and North Africa domain, exploits a Rack PC, Windows server with CentOS, 32 cores, 64 Gb RAM, 12 Tb RAID10. When advanced, dynamic and large scale applications are performed, using and producing big data for solar energy, the Amazon Web Services, Unix, 256 CPU cores, 24K GPU cores,3Tb RAM, 90 Tb is used. Computational applications include: Radiative Transfer Models (Emde et al., 2016) for the simulation of surface solar radiation and energy; Fast Solar Spectral Radiation architectures, using high performance and distributed computing (Kosmopoulos et al., 2018; 2021) for instant radiative transfer simulations for solar energy related operational products and services; Cloud Motion Vector Analysis for the development of pattern identification and tracking algorithms for short-term forecasting purposes (Kosmopoulos et al., 2020). SOLEA models and services have been employed in the research projects Smurbs, Eiffel, Geo-Cradle and EuroGEO e-shape.