Summary

Researchers face challenges in developing accurate and efficient artificial intelligence (AI) models for geospatial analytics tasks, especially when the availability of labeled data (i.e., data that has been annotated or tagged with specific information) is limited. Your challenge is to leverage existing geospatial foundation models to develop fine-tuned models that can support disaster recovery operations or real-time environmental monitoring, with the aim of improving the effectiveness and efficiencies of these critical operations.

Background

Artificial intelligence (AI) is playing an increasingly important role in advancing our understanding of Earth science and our ability to address the pressing challenges of environmental change. AI models provide powerful tools for analyzing vast and complex datasets generated by remote sensing platforms, climate models, and environmental monitoring systems. However, it is essential to have accurate and efficient models for analyzing geospatial data, especially in situations where labeled data (i.e., data that has been annotated or tagged with specific information) is scarce. Unfortunately, developing such models is a significant challenge for many researchers.

One of the main complexities lies in the distributed nature of the Earth science data. Accessing and analyzing this data is often difficult and time-consuming, making it hard to develop effective AI models and applications to solve pressing Earth science challenges. These conditions have led to a need for more specialized skills and tools for training and deploying models, adding another layer of complexity to an already challenging task. To make matters worse, the limited availability of ground-truth data for most applications often results in AI models with poor predictive accuracy, which can severely limit their impact.

To overcome these issues, it is essential to develop better models that can accurately predict and analyze geospatial data, even when labeled data is scarce. One potential solution to this problem is the use of geospatial foundation models, which are pre-trained models that use remote sensing data (e.g., Harmonized Landsat Sentinel-2 [HLS] data) and can be fine-tuned for different tasks. These models have already shown effectiveness in applications like disaster mapping (e.g., flood detection, burn scar detection), environmental change monitoring (e.g., land use, land change), and data discovery (e.g., image reconstruction, similarity search). There is also growing interest in further advancing and adapting these models for use in geospatial analytics for critical decision making.

These more precise and efficient AI models could potentially be deployed in disaster recovery efforts and real-time environmental monitoring to enhance the efficacy of these crucial operations, with far-reaching implications for safeguarding lives and preserving our planet.

Objectives

Your challenge is to develop an accurate and efficient AI model (or models) for a range of critical geospatial applications, such as disaster recovery operations, environmental change monitoring (e.g., greenhouse gas detection and monitoring), geospatial data discovery, and more. A key aspect of this challenge is to develop AI models that require minimal labeled data and are generalizable across various applications, as well as spatial and temporal domains.

To accomplish this goal, you can leverage pre-trained geospatial foundation models and fine-tune them for different applications that make use of remote sensing data (e.g., HLS, Sentinel-2, LiDAR, etc.). Keep in mind that the most useful models are capable of predicting and analyzing geospatial data in real time, thereby empowering critical decision-making in disaster response operations and for environmental monitoring.

Examples of fine-tuning applications using remote sensing data (with potential real-world applications in areas such as disaster response planning, precision agriculture, or urban planning) include:

Land cover classification
Change detection
Object detection (such as burn scars)
Causality/Correlation: explaining changes in a given area based on related events (e.g., a riverbed is thinning over time due to nearby crop irrigation)
Similarity search (static or over time) (e.g., Given an existing tile with a tornado, find similar tiles)
Above-ground biomass estimation

For your fine-tuned model, remember to provide a detailed description of the model architecture, the fine-tuned model itself, metrics, and benchmark results (such as accuracy, number of [labeled] training examples, floating-point operations per second [FLOPS], etc.). Don’t forget to evaluate the performance of your model using appropriate metrics, comparing them against results from other state-of-the-art models.

As a participant in this challenge, you will access the geospatial foundation models, pre-built workflows for select downstream tasks, and key remote sensing datasets (e.g., HLS) and create innovative AI models that excel in analyzing geospatial data with high accuracy, even when faced with limited labeled input data. By contributing to this challenge, you will play a vital role in propelling the advancement of essential geospatial AI models and solutions.

Join us in this endeavor to harness the power of AI for a better and safer world!

Potential Considerations

As you build your models, you may (but are not required to) consider the following:

Keep in mind that it is not always important to get higher accuracy. There are other important metrics to consider, such as significantly reducing the amount of labeled input data needed for your specific use case and saving time and resources compared to traditional data science workflows.

Data Considerations:

You will have access to a limited amount of labeled input data, so consider leveraging existing labeled datasets and reducing the amount of labeled data needed for your specific use case.
Keep in mind that the models will need to be efficient and fast since the available GPUs and core hours are limited.
Consider whether you can significantly reduce the amount of labeled input data needed for your specific use case, improve productivity compared with traditional data science workflows, and save significant time and resources.

Fine-Tuning Considerations:

You will have access to geospatial foundation models and examples of pre-built workflows, which you can fine-tune for your specific use case.
Consider the constraints you will have on your fine-tuning process, such as limited training time and resources. You may want to experiment with different architectures or hyperparameters to achieve the best results given these constraints.

Tooling Considerations:

You will have access to fine-tuning tooling on the cloud and some GPUs and core hours, so take advantage of these tools to streamline your fine-tuning process.
Consider designing models that are scalable and can be deployed in different environments. Think about the trade-offs between model complexity, speed, and accuracy in the context of your specific use case.

Ethical and Societal Implications:

Consider the potential impacts of your model outputs on society and the environment. Ensure that the use of geospatial data is ethical and aligned with social and environmental values. Be aware of the limitations and biases of your data and model, and work to mitigate them. Consider the potential impacts of your model on vulnerable or marginalized populations. Ensure that your model does not contribute to discrimination or exacerbate existing inequalities.
Security and Privacy: Geospatial data can be sensitive and subject to security and privacy concerns. Ensure that your use and handling of geospatial data is compliant with relevant laws and regulations. Be aware of the risks of data breaches or unauthorized access to sensitive geospatial data. Implement appropriate security measures to protect against these risks. Consider the privacy implications of your model outputs, particularly if your model is used for surveillance purposes.

For data and resources related to this challenge, refer to the Resources tab at the top of the page.

2023 NASA Space Apps Challenge

GeoAI Reimagined: Transformative and Diverse Earth Science Applications Using Foundation Models