Big data in Antarctic sciences – current status, gaps, and future perspectives

This paper was initiated by a multidisciplinary Topic Workshop in the frame of the Deutsche Forschungsgemeinschaft Priority Program 1158 “Antarctic Research with Comparative Investigations in Arctic Ice Areas”, and hence it represents only the national view without claiming to be complete but is intended to provide awareness and suggestions for the current discussion on so-called big data in many scientific fields.

The importance of the polar regions and their essential role for the Earth system are both undoubtedly recognized. However, dramatic changes in the climate and environment have been observed first in the Arctic and later in Antarctica over the past few decades. While important data have been collected and observation networks have been built in Antarctica and the Southern Ocean, this is a relatively data-scarce region due to the challenges of remote data acquisition, expensive labor, and harsh environmental conditions. There are many approaches crossing multiple scientific disciplines to better understand Antarctic processes; to evaluate ongoing climatic and environmental changes and their manifold ecological, physical, chemical, and geological consequences; and to make (improved) predictions. Together, these approaches generate very large, multivariate data sets, which can be broadly classified as “Antarctic big data”. For these large data sets, there is a pressing need for improved data acquisition, curation, integration, service, and application to support fundamental scientific research. Based on deficiencies in crossing disciplines and to attract further interest in big data in Antarctic sciences, this article will (i) describe and evaluate the current status of big data in various Antarctic-related scientific disciplines, (ii) identify current gaps, (iii) and provide solutions to fill these gaps.

1 General introduction

The enormous importance of the polar regions has been extensively stressed in the latest Intergovernmental Panel on Climate Change (IPCC) reports (Meredith et al., 2019; IPCC, 2021), where they are clearly recognized as being essential components of the Earth system. The Arctic is warming nearly 4 times faster than the rest of the planet (Rantanen et al., 2022). The melting snow and ice drastically reduces the albedo, leading to darker surfaces and hence increased amounts of solar energy absorbed in these areas. The resulting warming leads to continued loss of sea ice, glaciers, and both ice sheets, with concomitant habitat and biodiversity loss in Arctic sea ice and terrestrial ecosystems (Meredith et al., 2019; Rantanen et al., 2022). Similar trends can be observed particularly around the Antarctic Peninsula (Wille et al., 2022), and in early 2023 Antarctic sea ice extent reached a dramatic all-time low. The regions in which significantly less ice has formed than in the previous year or the long-term average for past years are mainly concentrated in the Bellingshausen Sea and Weddell Sea but also cover broad expanses of Antarctica's eastern coastline (https://www.meereisportal.de, last access: 1 August 2023). Compared to long-term data, an additional area >300 000 km² was ice-free during the 2023 season. The massive loss of ice and glaciers leads to significant global sea level rise, with strong impacts on coastal regions and cities (Smith et al., 2020; IPCC, 2021). The polar amplification of climate change is a well-known phenomenon already observable and projected by almost all climate models and will intensify over the 21st century, particularly by further substantial warming and increases in precipitation (e.g., Collins et al., 2013; Meredith et al., 2019). Key concerns are manifold and hence there are many approaches spanning multiple scientific disciplines to better understand polar processes; to evaluate ongoing climatic and environmental changes and their ecological, physical, chemical, and geological consequences; and to make (improved) predictions. These approaches, including novel tools for capturing complex ecological scenarios, result in very large, multivariate data sets, which can be broadly classified as “Antarctic big data”.

The term “big data” dates back to the 1990s (reviewed by Diebold, 2012) and usually includes data sets with sizes beyond the ability of commonly used software tools to capture, curate, manage, and process data within a tolerably short time frame (Snijders et al., 2012; Kitchin and Mcardle, 2016; Arribas et al., 2022). Big data encompasses unstructured, semi-structured, and structured data, and requires a set of techniques and technologies with new forms of integration to uncover insights from data sets that are diverse, complex, and of a massive scale. Big data analytics is the process of analyzing big data to extract concealed patterns and information that can yield improved results and new conclusions.

The scientific disciplines addressing research in Antarctica are well developed, and some interdisciplinary approaches already exist, but the latter should be fostered to deepen our knowledge and to obtain a more mechanistic understanding of Antarctic processes, their changes, and their consequences. A representative example is the complex life cycle and ecology of Antarctic krill, which is the most abundant keystone species of the Southern Ocean marine food web and plays an important role in biogeochemical cycles (Cuzin-Roudy et al., 2014; Cavan et al., 2019). Antarctic krill have three critical periods in their early life cycle that strongly affect their survival, with the last one coming during the first winter, when they rely on sea ice biota as food resource and also use sea ice for shelter (e.g., Siegel, 2016). Therefore, environmental conditions such as sea ice quality and quantity and ocean temperature strongly impact the survival of the larvae, meaning that the retreat of winter sea ice and higher temperatures can become dominant drivers of krill population decline. To fully understand and predict the fate of krill populations in the future Southern Ocean, ice-related physical, chemical, and biological expertise is therefore essential. Based on deficiencies in connecting such traditionally separated disciplines and to attract further interest in big data in Antarctic sciences, this article will (i) evaluate the current status of big data in biological, chemical, and geophysical Antarctic science, (ii) identify current gaps, and (iii) provide solutions to fill these gaps.

2 State of the art

2.1 Biological view on Antarctic big data – from molecules to ecosystems

Antarctic biodiversity seems to be immense, yet we have only scratched the surface in terms of documenting and understanding many taxonomic groups, such as those in the deep sea or prokaryotic and eukaryotic microorganisms (Gutt et al., 2010; Danis et al., 2020, and references therein). This lack of knowledge is partly related to the difficulty of performing research in harsh environments characterized by extensive sea ice, low temperatures, and long polar nights, which present considerable logistic and infrastructural challenges. Moreover, Antarctica is amongst those areas experiencing the most rapid rates of regional warming (Flexas et al., 2022) and harbors unique ecosystems that are under severe threat from climate change (Pörtner et al., 2023). With regional warming, biotas living in these frozen ecosystems will have to adapt if they are to survive, yet there is currently a very limited understanding of Antarctic biodiversity and even less understanding of the future resilience of these organisms in a changing world. To generate a priori predictions of biodiversity change in Antarctica, it is imperative to understand the true extent of biodiversity, including how organisms interact in food webs, the biological mechanisms by which they have adapted to the cold, their levels of phenotypic plasticity, and how these attributes may impact their abilities to respond to change. Critical to this understanding are a variety of “omics” approaches that exploit the high-throughput sequencing of genetic material.

The so-called omics technologies adopt a holistic view of the genetic repertoire, expression, and analysis of biomolecules that make up a cell, tissue, or organism. They are aimed primarily at the universal detection of genes (genomics), mRNA (transcriptomics), proteins (proteomics), and metabolites (metabolomics) in a specific biological sample across all size classes from bacteria to mammals (e.g., Schneider and Orchard, 2011). Our molecular toolbox has greatly expanded over the past 2 decades as omics techniques have become more advanced and cost-efficient. As a consequence, data wealth has outstripped our capacity to carefully and comprehensively process all molecular information. Analysis of the resulting very large data sets is currently hampered by various bottlenecks, which are discussed below.

All genetic information of any organism is stored in its nucleic acids, DNA and RNA, which can be extracted from individuals of a given species, from entire communities, or even from environmental samples (e.g., from the water column or from a sediment core). These nucleic acid extracts can then be subjected to a variety of high-throughput sequencing approaches, ranging from whole (meta)genome sequencing to (meta)transcriptomic and epigenetic analysis (e.g., Mason et al., 2017). These and related approaches generate huge amounts of nucleotide data, which require quality checking prior to taxonomic or functional analysis. For the latter, numerous automated bioinformatics tools and reference databases have been established, which are essential for processing and evaluating results obtained from massive raw data (Lee et al., 2012; Pappas et al., 2020). Bioinformatics is defined as the application of mathematical and computer science methods to solve problems and address questions in molecular biology that require high-performance computation and analysis (https://www.sib.swiss/what-is-bioinformatics, last access: 1 August 2023).

While the production of massive genetic raw data is relatively easy, their assemblies to genomes or transcriptomes and their interpretation still remain challenging. Genome annotation is the process of identifying and labeling regions of a genome according to their putative functions. Genome annotation, in particular for eukaryotes, remains a major challenge as it is time-consuming and many steps still have to be undertaken manually for quality control (Yandell and Ence, 2012; Salzberg, 2019). Using existing annotations from already described model organisms can be problematic given the diversity of Antarctic organisms and their long histories of evolution in isolation, which in practice tends to result in a situation where only a small proportion of genes can be successfully annotated. Genome annotation is then followed by a comprehensive genomic analysis, which includes the identification, measurement, or comparison of genomic features such as DNA sequences, structural variation, gene expression, or regulatory and functional element annotation at a genomic scale.

Molecular biological databases pervade all areas of bioinformatics. A recent study indicated the existence of over 1700 online molecular biology databases between 1991 and 2016 (Imker, 2018). These include databases on (i) DNA sequences, (ii) predicted protein structures, (iii) phylogenetic trees, (iv) metabolic and regulatory pathways, and (v) gene expression.

Molecular biological data are also important for all ecological questions, since ecology is defined as the complex relationships of organisms to one another and to their physical environment across all scales. Ecological big data systems comprise, for example, in situ and remote sensing data, community-curated data resources, biodiversity databases, citizen science, and long-term networks of ecological monitoring stations. Antarctica is not only underrepresented in these data systems (Li et al., 2020) but also strongly differs in food web structure and dynamics when compared to temperate regions. Nevertheless, rates of ecological data generation, accumulation, and interpretation are continuously growing, with rapid developments in data volumes, methods of data collection, and new analytical and computational approaches. The development and combination of data streams are inspiring because they create new opportunities to study ecological systems at high resolution and on broad temporal and spatial scales, to better understand underlying processes, and to improve ecological forecasting in Antarctica (Dietze, 2017). This evolution in ecological research helps us to fundamentally understand interactions among organisms and with their environments, although the main focus to date has been on temperate to tropical regions.

As ecology is such a diverse and complex scientific discipline, we provide an selected example on animal tracking devices on Antarctic birds, which enable the determination of diurnal and annual movements with increasing accuracy, thereby producing big data. Sensing and tracking technologies are generally becoming cheaper and smaller, producing unprecedented volumes of data for ecological and behavioral studies. In addition to increases in the precision of GPS locators, biologists can now obtain detailed data from bio-logging technologies. Standardized data-analytical pipelines are needed to translate these data into scientific knowledge. A powerful new approach is to assign and analyze behavioral states from accelerometers using machine learning (Fig. 1). Accelerometers measure the inertial acceleration during animal movements, most commonly on three axes. Specific movements are reflected in unique combinations of the three accelerometer axes over time (Fig. 1). From these movements, classes of behavior (e.g., prey captures in penguins) can be identified using support vector machines (SVMs; Carroll et al., 2014; Sutton et al., 2021). A variety of machine learning algorithms have attempted to distinguish among multiple classes of behavior. Two main approaches have been applied to accelerometry data, termed unsupervised and supervised learning methods. In the unsupervised classification approach, accelerometer data are grouped according to similarities in movement patterns using cluster analyses (Sakamoto et al., 2009) or spectral analyses (Ropert-Coudert et al., 2006). However, it has been pointed out that using too many categories of behavior may affect the ability of machine-learning algorithms to accurately classify all behaviors (Ladds et al., 2017). In addition, since different animal species exhibit distinct movement patterns, the optimal machine learning method has to be identified and applied from a set of approaches. Therefore, a “super learning method” has been proposed, which applies sets of candidate machine learning methods to a data set and chooses the optimal model or combination of models (Ladds et al., 2017).

Figure 1 Schematic analysis pipeline to assign and process behavioral states of Antarctic sea birds from accelerometers using machine learning.

2.2 Chemical view on Antarctic big data

The many different types and large inventories of biological data (Sect. 2.1) are mirrored in analytical and ecological chemistry. Chemistry deals with the properties, composition, quantities, and structure of substances; their transformations; and their thermodynamic budgets. The chemical view on polar science is mainly concerned with measuring and describing very low concentrations of inorganic and organic matter and its transformation along temporal and biological gradients. Moreover, chemistry in ocean sciences is concerned with exact analyses of nutrients, energy compounds and their turnover, elemental stoichiometry of central biological processes, and chemical signalling compounds among and between different kingdoms. Hence, there is a strong linkage between marine chemical data and marine ecology, e.g., when considering nutrients versus primary production.

While by definition big data in natural sciences refers to the collection, processing, and availability of large amounts of data, we also highlight the need for advanced statistics and machine learning tools for the analysis and interpretation of naturally complex chemical samples. Integrating and correlating the wealth of physicochemical data collected by remote observatories and autonomous instruments (e.g., Johnson et al., 2009) with biotic metadata and chemical-analytical data, e.g., data derived by ultra-high-resolution mass spectrometers (Leefmann et al., 2019), is an emerging challenge and opportunity for modern marine chemistry in the polar regions.

Notably, natural mixtures of marine organic molecules exhibit a nearly continuous range of physicochemical properties. Their molecular composition follows a dynamic equilibrium that is shaped by ecosystem characteristics with contributions from biochemical and abiotic (e.g., photo and redox chemistry) reactions. As a consequence, initial signatures of biogenic precursor molecules like lipids, glycans, proteins, and natural products are attenuated beyond recognition, resulting in the most intricate mixtures of organic molecules on Earth. This material represents a valuable inventory of information, transcribing the sum of biotic processes leading to its unique composition.

2.3 Geophysical view on Antarctic big data

Geophysical data are acquired at different levels from ground-based observations, remote controlled or autonomous platforms, and satellite missions (Fig. 2). Here we provide an exemplary perspective on the development and utilization of big data in this scientific discipline without the ambition of completeness.

Figure 2 Schematic representation of different geophysical data layers and observations from Antarctica in regard to big data.

Since the turn of the century, data from a multitude of new satellite missions have become available. These missions are game changers regarding data availability and access (open and FAIR data access is now developing towards a standard) with exponentially increasing data volumes. Although it is beyond the scope of this paper to review all of these satellite missions, we will highlight a few of them to illustrate their contributions and innovation characteristics and how this is reflected in improvements in data quality and quantity.

In this context, and with respect to environmental monitoring, the European Union/European Space Agency (EU/ESA) Earth observation program Copernicus is most prominent. It provides continuous data acquisition from a fleet of satellites (Sentinels) designed to support a variety of environmental and security purposes and services. A main aspects of the program are its vision for long-term data continuity with constant quality conditions and its expansion by additional new mission concepts. All satellites feature improved radiometric, spatial resolution, and/or spatial coverage. Since the start of data provision in 2013, large databases have been built up according to a coordinated acquisition plan, which is coordinated with missions of other space agencies. Most relevant for polar observations are the synthetic aperture radar (SAR) data from the Sentinel-1 satellites and the multi-spectral imagery of the Sentinel-2 mission. While the Sentinel-2 mission still includes two satellites in the same orbit plane, enabling a 5 d repeat coverage with the same acquisition geometry, one of Sentinel-1 satellites stopped operating in 2022. The satellite Sentinel-1 C is currently being prepared as replacement.

The ESA Earth Explorer Program additionally provides the opportunity to test experimental setups. Within this program, mission concepts like SMOS (Soil Moisture and Ocean Salinity) or CryoSat-2 (Cryosphere) have been developed that are highly relevant for monitoring the Antarctic region via the generation and analysis of data on sea ice thickness. CryoSat-2 altimetry also allows the detection of elevation changes in the continental ice sheets and glaciated areas like the Patagonian ice fields. Its successor, CRISTAL (Copernicus Polar Ice and Snow Topography Altimeter), has been committed for production and launch. Missions like SWARM (constellation of three satellites for mapping Earth's magnetic field) and GOCE (Gravity Field and Steady-State Ocean Circulation Explorer) also provide important information to couple the solid earth and cryosphere, and a follow-up on the gravity mission GOCE is planned by the ESA and NASA for the next decade (MAGIC mission, Mass-change and Geoscience International Constellation), in addition to SCOUT missions that provide shorter-term perspectives for smaller-payload satellite experiments.

Furthermore, national and international satellite missions complement our observational capabilities by deploying specific instruments into orbit. The innovative concept of the German TanDEM-X mission allows for acquiring repeated data to derive global digital elevation models. It provides bistatic interferometric SAR data that are unique regarding their quality, especially in the Antarctic region, where half of the year is spent in darkness and cloud cover is frequent. This allows the determination of glacier-specific elevation changes and sea ice mapping. The Germany–US GRACE and GRACE-FO (Gravity Recovery and Climate Experiment) missions provide unprecedented time series of mass fluxes within the Earth system. For Antarctica, they provide monthly resolution of the mass balance of the continental ice sheets and its interlinkage with global sea level change. In the optical domain, the series of Landsat satellites has been providing data continuity since 1972. Its global archive is open for data access free of charge. In addition, a multitude of small commercial satellites orbit the Earth. Here, the Planet Labs satellites and ICEye have to be mentioned. Although primarily driven by commercial interests, these missions offer scientists access to data on request, exploiting their targeted observation capabilities at high spatial and temporal resolution to supplement the monitoring activities of the large space agencies.

The spaceborne observational capabilities are supported by airborne platforms (planes and drones). Those platforms are more flexible with regard to operation in specific target areas and their payload, which can be adapted to the requirements of specific project tasks. Very often they are equipped with sensors for high-resolution data acquisition, for measurements, or for tests that cannot be performed from space. In that respect and in agreement with the previous sections, we also realize that in geophysics there is a tendency towards considerably increasing data volume sizes.

Ground-based observations are also realized with increased spatial and/or temporal sampling, although access and maintenance still limit continuous operation under polar conditions. These observation systems serve a multitude of applications ranging from inferring meteorological data to GNSS (global navigation satellite system) time series recordings to infer glacial-isostatic adjustment and further geophysical instrumentation for continuous mid- to long-term data acquisitions (e.g., seismometers), various geophysical measurements, and specialized floats and gliders to provide measurements of a variety of ocean parameters (e.g., ARGO).

3 Gaps and potential future gaps or challenges in Antarctic big data

3.1 Challenges in biology

There is currently a general scarcity of Antarctic genetic data, e.g., from bacterioplankton in the Weddell Sea and other provinces, with even basic taxonomic descriptions of such bacterial communities being missing. The same is true of microorganisms inhabiting Antarctic sediments and all types of ice habitats, while for terrestrial sites the situation is only slightly better. Antarctic deep-sea fauna are also not well addressed in terms of ecology, biodiversity, conservation, taxonomy, and evolutionary biology. The lack of Antarctic microbial data stands in contrast to the better-investigated Arctic regions such as Greenland or Svalbard. In addition, with the third Tara Ocean Expedition in the Arctic (https://fondationtaraocean.org/en/expedition/tara-oceans, last access: 27 July 2033), numerous genetic data sets are now available at least for the plankton, which allow, for example, comparative evaluation of marine microbial biodiversity from other marine provinces (Pesant et al., 2015; Sunagawa et al., 2020; Royo-Llonch et al., 2021; Vernette et al., 2022). One of the scientific highlights was the discovery that one-third of 18S-rDNA sequences could not be assigned to known eukaryotic groups, pointing to a huge unexplored biodiversity, and that most eukaryotic plankton biodiversity belonged to heterotrophic protistan groups, particularly those known to be parasites or symbiotic hosts.

Genome mining describes the exploitation of genomic information for the discovery of biosynthetic pathways of natural metabolites and their possible interactions (Albarano et al., 2020; Chevrette et al., 2021). Genetic and genomic data have been accumulating at unprecedented rates in global sequence databases such as the International Nucleotide Sequence Database Collaboration (INSDC, https://www.insdc.org/, last access: 2 June 2023), but the extraction of well-annotated functional information lags behind. This is mainly because the precise functions of many genes remain unexplored and the time-consuming experimental work to address this problem is not systematically performed. Moreover, often only a subset of the available data can be annotated or used for specific analyses, resulting in the accumulation of large amounts of unused nucleotide data whose potential should be better explored. This is particularly true of samples from less-accessible regions of Antarctica for which reference databases or even sequenced and annotated genomes remain largely incomplete.

As a prerequisite for big data analysis, curated and high-quality databases are essential, but the lack of metadata standards often presents issues (Stow et al., 2018), especially when the inability to taxonomically assign organisms hampers the usability of databases in, for example, biodiversity studies (Bayraktarov et al., 2019; Loeffler et al., 2021). Many data archives also suffer from the fact that submitters do not always adhere even to minimal sequence metadata standards. Often elementary information is missing, such as the GIS (geographic information system) coordinates of sample locations. Additionally, nomenclatural databases should be interlinked to sequence databases as an important prerequisite for the sound use of big data. In open-access data centers such as PANGAEA, there is the problem that authors submitting data sometimes use different names for the same entity, and as a result the data managers often cannot sort them out.

3.2 Challenges in chemistry

The concept of deciphering natural chemical complexity (chemodiversity) is rather new to chemistry, a field of science that traditionally attempts to predict and control reactions, kinetics, and products rather than to characterize complex molecular mixtures containing thousands of different molecules. Especially in the life sciences, this is rapidly changing, with systems chemistry (analogous to systems biology) and environmental metabolomics emerging as powerful strategies to capture and interpret biological processes and their chemical fingerprints at various levels. As ecosystems are transformed on different timescales to complex organic matter and geopolymers, in marine biogeochemistry this approach is emerging as the strategy to understand contemporary and historical processes.

Novel instrumental analytical technologies enable the partial description of this chemical diversity (Steen et al., 2020). The main challenge is the integration, processing, and validation of the chemical data produced by such instruments in order to distinguish and map the plethora of molecular formulae and (if possible) to derive structural molecular information (Leefmann et al., 2019). Mathematics and statistics offer important opportunities to understand these data and to provide context in terms of significant chemical and/or biological information. Chemometrics are valuable to mine correlated information (e.g., biotic and abiotic environmental parameters, species abundance, genetic repertoires, and gene expression) and to decipher multidimensional analytical spectra and data. For the investigation of complex natural mixtures, a systems approach using a variety of chemical and bioanalytical approaches is the key. Such systems chemistry strategies are currently emerging and involve state-of-the-art selective separation combined with high-resolution spectral and/or spectrometric technologies. They are complemented by extensive chemical big-data mining and most recently supported by machine learning.

3.3 Challenges in geophysics

The resulting amounts of data are steadily growing, which allows the analysis of time series that cover decadal time spans. It fosters the development and application of new methods in the analysis of big data. Often, a combination of different sensors at the same platform and/or at different levels from ground-based to airborne and spaceborne instruments enables improved insights but also increases the volume of data to be handled. Today, not only in Earth sciences, we are facing such huge data volumes that we urgently need to improve already existing processing and to develop new, automated processing routines by applying fast and efficient algorithms. Artificial intelligence (AI) provides novel methods for data combination to exploit the information content to an optimum and to improve the quality and significance of research outcomes (Baumhoer et al., 2019; Davari et al., 2021; Loebl et al., 2022). However, those methods require a large amount of training and validation data that might not be available in the environmental sciences, especially in Antarctica. Hence, these data may need to be generated specifically. In addition, such data volumes require respective processing facilities (e.g., high-performance computing, HPC) together with adequate and fast storage. While most research institutions have access to such HPC processing facilities, data sets are often stored or at least downloaded various times at multiple locations to facilitate processing and increase network traffic. However, the need to re-access data to restore time series severely limits the reprocessing of archives applying new software versions or novel algorithms when new product lines are inferred using complete data time series.

A variety of examples for large-scale automated processing and data dissemination for the Antarctic region are already in place, e.g., the Alfred Wegener Institute/University of Bremen sea ice portal (https://www.meereisportal.de/, last access: 1 August 2023), the glacier portal of the Friedrich-Alexander-University Erlangen-Nürnberg (Friedl et al., 2021), or the Technical University Dresden geodetic data portal (Groh and Horwath, 2021). Those portals have evolved out of single or consecutive project activities that have been heavily used at both the national and international level. However, maintaining and updating such portals is normally beyond the scope and capabilities of small university research groups, which tend to have limited core resources. In addition, such activities are in general not supported by many funding schemes. As a consequence, data product lines or prototype services are often discontinued or only reworked when new project funds allow it. There is also no mechanism to transfer such established product lines to a continuous operation, e.g., at a national institution or a Copernicus service. Commercial operators face the same problem and, in addition, have larger operation costs since HPC access has to be purchased. This is a major setback in view of the need to continue time series records for the evaluation of changes in the Antarctic region, specifically in view of climate change scenarios. Furthermore, research policy asks for more and better science communication. Access to science products with respective quality assurance accompanied by some guidance on usage for either direct access by the general public science journalists or for institutions of secondary and higher education would be a strong building block for an informed public and will also help to raise awareness and interest for the next generation of scientists.

In addition, the modeling community faces similar difficulties to the remote sensing groups. The spatial and temporal resolutions of models and their extent are steadily being improved, which is increasing demand for processing and archiving capabilities.

4 Solutions to fill the gaps in Antarctic big data

4.1 Perspectives in biology

Based on the current status of Antarctic research and the above identified knowledge gaps, we have formulated the following recommendations. Big data analyses in Antarctic research crucially rely upon high-quality databases, which should be peer reviewed and curated by experts in their respective fields. High-quality, contiguous reference genomes are also needed, and ideally these should be annotated to a high and eventually even internationally universal standard. Thus, an initiative to produce high-quality reference genomes for a representative selection of Antarctic organisms would be desirable.

We also strongly recommend the use of standardized vocabulary such as Environmental Ontology (EnvO) (https://sites.google.com/site/environmentontology/, last access: 2 June 2023) for creating (meta)data sets. This would improve data quality, as ontological software understands the different terms and recognize synonyms, as well as facilitate the machine reading of entries, which is important for data mining and machine learning. Consequently, international standards for the preparation and submission of molecular data, metadata, and GIS-referenced data into a single database are essential. For this, the FAIR data standards and FAIR principles (Findable, Accessible, Interoperable and Reusable) for digital assets have been established (https://www.go-fair.org/fair-principles/, last access: 2 June 2023). These principles emphasize machine action ability, i.e., the capacity of computational systems to find, access, inter-operate, and reuse data with no or minimal human intervention, because scientists rely more and more on computational pipelines to deal with big data due to rapid increases in volume, complexity, and creation speed (Tanhua et al., 2019). The FAIR principles should of course be applied equally to all Antarctic scientific disciplines.

A further important aspect is the need to harmonize molecular biological workflows and bioinformatics pipelines to provide optimized tools for big data analysis to the scientific community. In addition, improved communication between various existing databases would again be desirable for harmonization but also for interlinkage and comparison. A good example is GBIF (Global Biodiversity Information Facility, https://www.gbif.org, last access: 2 June 2023) as an international network and data infrastructure, which could still be improved despite already having numerous links to other databases. Such approaches would be a great step forward for big data analyses and synthesis of the Antarctic region.

Since raw molecular data are often processed differently by scientists, it is also important to understand the conceptual choices influencing the production and classification of the results. For example, the outcome of raw data analysis can be impacted by filtering steps and software version updates. Hence, we recommend that omics papers should always provide such information, ideally accompanied by the code used to implement the original analyses. This would greatly improve reproducibility and comparability to other studies.

Greater emphasis should also be placed on the generation and maintenance of long time series of genomic data sets, which are clearly essential for assessing the impacts of current and future environmental change. Without baseline information, e.g., on microbial communities, it will be challenging if not impossible to monitor and analyze temporal changes.

Improving our fundamental understanding of Antarctic biodiversity and the molecular mechanisms underlying environmental and evolutionary acclimation and adaptation is essential for predicting future changes in the structure and function of marine, freshwater, and terrestrial communities in Antarctica in relation to ongoing climate change. In particular, a systems biology approach is needed to understand the manifold complex genomic, transcriptomic, proteomic, and metabolomic interactions on the species, population, and community level, which is necessary for achieving a holistic understanding of feedbacks between biota and their (changing) environment or even how biota change their environments. This should also include new model organisms capable of representing past and current biotic and abiotic effects on various ecosystem services such as biodiversity, primary production, and biogeochemical cycling, as well as predicting such functions in the future. The latest Tara Ocean Expedition in the Arctic could serve as a template, and big data analysis would be essential to better understand and interpret the massive amount of raw data. Such studies might also provide an opportunity not only to discover organisms that are new to science but also to characterize new genes, metabolic pathways, and biotic interactions, all of which contribute to better understanding the contemporary and future biology of the Antarctic region (Fig. 3).

Figure 3 Elements of “omics” in polar big data science. Flow of genetic techniques, omics approaches, bioinformatics, and final data exploration to provide a better understanding of the current and future biological diversity and processes of the Antarctic ecosystem.

Based on currently available ecological big data, we recommend focusing on integrating ecosystem processes, structures, and functions and their responses and adaptations to global changes at the regional and global scale. The main objectives are to quantitatively assess and scientifically predict the responses of ecosystem components and key processes to climate and environmental changes. In principle, the research methods of integrative ecology include meta-analysis, data mining, and data–model synthesis. However, big data can be challenging to work with due to large data volumes, heterogeneity in data quality and uncertainty, and the need for metadata as described earlier (Yang and Huang, 2013; Ladeau et al., 2017; Chang and Grady, 2019). None of these challenges are entirely new to ecologists, but the massive increase in data in all four dimensions challenges traditional approaches to data management and analysis.

To analyze large and detailed data sets, machine learning techniques (reviewed in Valletta et al., 2017) are ideally suited to the task of extracting biological insights from complex behavioral and observational data, as well as from large data sets such as digital images and video recordings. Machine learning techniques including support vector machines (SVMs), classification and regression trees (CART), and artificial neural networks (ANNs) provide computationally powerful methods of data classification. Thus, machine learning is expected to play a central role in extracting scientific knowledge from big data in ecology.

Technological solutions to deal with big data include the development of open code- and data-sharing platforms, flexible statistical models that can handle heterogeneous data and sources of uncertainty and cloud-computing delivery of high-velocity computing. Educational solutions include providing training to both established and early career scientists and strengthening collaborations between biologists and data scientists. The broader goal is to maximize the power, scalability, and timeliness of biological insights and ecological forecasting. Hence, Antarctic big data should be considered an almost unexplored treasure containing massive amounts of hidden information waiting to be extracted and interpreted by smart technologies and by well-educated, interdisciplinary scientists.

In recent years, some national and international programs have begun to address the problems described in this paper. On the national level there is, for example, the NFDI4Biodiversity consortium (https://www.nfdi4biodiversity.org, last access: 12 July 2023), which is hosted under the umbrella of the National Research Data Infrastructure (NFDI). NFDI4Biodiversity is dedicated to the mobilization of biodiversity and environmental data for collective use according the FAIR principles. In NFDI, valuable data resources for the German science system are systematically made accessible, federated, and usable in a sustainable way. The main goal of this consortium is to provide scientific and technical competencies to users from research and practice with a service portfolio for biodiversity and environmental data. On the European level, various programs have also been established, such as Blue-Cloud (https://blue-cloud.org, last access: 12 July 2023). The mission of Blue-Cloud is to provide a virtual environment with open and seamless services for the storage, management, analysis, and reuse of research data across borders and disciplines. On the international level, the activities of the Joint Genome Institute (JGI), U.S. Department of Energy (https://jgi.doe.gov, last access: 2 June 2023), developed a vision for integrative and collaborative genome science. Their mission is to provide the global research community with access to the most advanced integrative genome science capabilities including state-of-the-art technologies and user-friendly data portals. JGI will develop and employ advanced and accessible computational approaches to enable the integrative analysis and interpretation of multiple orthogonal data types including transcriptomic, epigenomic, and metabolomics results in conjunction with genomic information.

4.2 Perspectives in chemistry

The analytical strategies outlined above present powerful opportunities to investigate the structure, origin, transformation, ecological function, and distribution of organic compounds under changing conditions in the polar oceans. The Southern Ocean Observing System (SOOS, https://www.scar.org/soos/, last access: 1 August 2023; Meredith et al., 2013) and the ARGO float program (https://www.ocean-ops.org/board?t=argo, last access: 1 August 2023; Roemmich et al.; 2009, Boening et al., 2008) are good examples of initiatives that facilitate the integration of multiple resources to enhance the analysis and interpretation of physicochemical data. The availability of high-quality, long-term observational data and the establishment of open-access data centers such as PANGAEA (https://www.pangaea.de/, last access: 1 August 2023; Dittert et al., 2002; Diepenbroek et al., 2002) or the World Ocean Database (WOA; https://www.ncei.noaa.gov/products/world-ocean-database, last access: 1 August 2023) enable chemical ocean data to be shared in larger, increasingly global contexts. Open access to these data repositories is critical to fostering scientific collaboration and progress. Moreover, the broad accessibility of data stimulates and enables research independent of traditional funding models (Levine et al., 2020). As a result of these efforts over the past 2 decades, automation in ocean sensor techniques and data acquisition in the laboratory in combination with elegant data archives has dramatically increased the size of data sets available to address research questions relating to the polar oceans and beyond.

4.3 Perspectives in geophysics

We recommend establishing a central European archive for satellite data including processing facilities with access via a data hub for national institutions. Current solutions do not hold all of the required data from Antarctica (and not even from the Sentinel satellite series), are not accessible to all polar researchers, or impose costs when considering long-term activities. Especially university groups, which cannot base their Antarctic research on core funding, face limits regarding regular processing and service. Here, new funding lines and/or operation modes to access respective data are required, together with adequate processing infrastructure. While requirements with regard to the long-term archiving of the results of research have found their way into proposals, the related facilities and associated costs are often not in place or are not sufficiently covered at most institutions to handle the enormous amount of data. Here, we see a strong need for a coordinated effort under the NFDI4Earth initiative to consider and include such large data sets at the HPC level.

Quality assurance and proper metadata documentation for subsequent usage are further challenges that arise from automated processing. The assimilation of automated data into geophysical models requires information on the product quality and reliability. Continuous data flow, as outlined above, would be desirable as an additional incentive for efforts to accordingly adapt models. To date, not all Earth observation products can be readily assimilated into respective models. Here, closer interactions among the respective research communities are required, as are intelligent assimilation strategies.

5 Conclusions

Antarctica harbors some of the least well-known biodiversity on Earth and is currently facing dramatic changes in the climate and environment. Although important (big) data have been collected and observation networks have been built in Antarctica and the Southern Ocean, this region is still relatively data scarce due to the challenges of remote data acquisition, expensive labor, and harsh environmental conditions. For example, winter sampling campaigns are missing so far. There are many approaches crossing multiple scientific disciplines to better understand Antarctic processes; to evaluate ongoing climatic and environmental changes and their manifold ecological, physical, chemical, and geological consequences; and to make (improved) predictions. Together, these approaches generate very large, multivariate data sets, which can be broadly classified as “Antarctic big data”. For these large data sets, there is a pressing need for improved data acquisition, curation, integration, service, and application to support fundamental scientific research.

Although NFDI4Biodiversity and other initiatives provide intensive training programs for students, we consider the interface between biology and bioinformatics to still be underdeveloped. Hence, this needs to be more strongly integrated into the curricula of German universities. The majority of research (at least in the university setting) is performed by doctoral students, many of whom lack even a basic grounding in bioinformatics at the start of their PhDs. More training courses, such as in NFDI4Biodiversity, and dedicated support linked to bioinformatics concepts and infrastructures would help these young scientists to acquire the necessary skills to analyze their data more quickly and effectively. Therefore, specific masters courses across disciplines are urgently needed. Better education and training of the future generation of polar scientists is essential if we as a community are to seamlessly embed big data across the traditional domains of physical oceanography, marine biology, chemistry, geophysics, and geology. A recent dedicated initiative in this direction is the new “Helmholtz Graduate School for Marine Data Science” (MarDATA; https://www.mardata.de/, last access: 1 August 2023; Verwega et al., 2021).

AI techniques promise improved performance but should not reduce the capabilities of the models to simulate the underlying processes and mechanisms. We see a strong demand for cross-disciplinary research and interaction between computer scientists, data scientists, and environmental scientists to address these requirements. Both technical skills and a fundamental understanding of the environmental problems and processes within the Earth system are required. Hence, we suggest in line with the biological and chemical sections to that there should be more interdisciplinary education efforts, particularly at the masters and PhD levels (e.g., International Doctoral Program MOCCA within the Elitenetzwerk Bayern).

With such an educational effort Antarctica will be more effectively integrated into biological, chemical, and geophysical big data systems, thereby addressing significant knowledge gaps between currently observed changes and the underlying mechanisms.