How Should Communities Adapt to the Changing Climate? A New Database Offers Answers

The models meteorologists use to predict next week’s weather simulate how air masses move through the atmosphere. When it comes to predicting what the climate will be like in 2034 or 2054, researchers have to take a lot more into account. 

Pierre Gentine, Maurice Ewing and J. Lamar Worzel Professor of Geophysics in the Departments of Earth and Environmental Engineering at Columbia Engineering and Earth Environmental Sciences and a member of the Data Science Institute at Columbia University, compares today’s models of Earth’s climate to a gigantic Lego structure.

“It’s very difficult to put all of those pieces together and make sure they’re doing the right thing,” he said. “We’re reaching the limit of complexity.”

For the communities that rely on predictions to guide their plans for adapting to climate change, this is a huge problem. That’s because nearly all of the uncertainty about how the climate will change over the next 40 years lies in researchers’ ability to develop accurate models rather than the amount of greenhouse gasses that we emit into the atmosphere.

“The emission pathway doesn’t matter so much before that time,” Gentine said. “It’s really about understanding the physical system.”

Researchers started developing climate models in the 1950s. In the last decade, the introduction of AI methods and an explosion in the amount of data have accelerated the rate of progress — and exposed several bottlenecks that limit what kind of information researchers can deliver to the communities that need it. 

In an effort to overcome these problems, the NSF funded a multi-institutional Science and Technology Center called LEAP (short for Learning the Earth through AI and Physics) in 2021. Led by Columbia with Gentine at the helm, LEAP is endeavoring to revolutionize climate projections for informed climate adaptation.

“We’ll have to do a lot of climate adaptation until at least 2060, as it will take time to reach net zero,” Gentine said. “This is only really possible if leaders have accurate climate projections.”

Researchers need ready access to useful data to provide leaders with those predictions. That’s where LEAP comes in. The center’s premier project, LEAP Pangeo, is a platform offering researchers and communities across the world access to more data than ever before. 

AI + Earth’s climate

How much CO2 is absorbed and released when trees in the northern hemisphere grow leaves in the spring and lose them in the fall? How much extra energy will the planet absorb where ice sheets are retreating? How do nitrogen fertilizers impact the ecosystems’ ability to absorb CO2 from the atmosphere?

These are the types of questions that climate models must take into account when predicting future climate. 

In 2018, Gentine and colleagues developed the first computational models that used AI to simulate climate processes. “We replace some of those individual Lego bricks with AI components,” he said. “For instance, clouds are very difficult to represent with traditional methods, so we replace them with a big neural network.” 

The technique is relatively straightforward. The researchers start with reliable data about some aspect of Earth’s system — either from direct observations or from high-resolution simulations — and train an AI algorithm to predict how certain starting conditions will lead to certain outcomes. Once this is done, the researchers can update their climate predictions and projects by inserting this AI into the much larger “host” models that calculate how many, many processes will interact to produce the overall climate.

For example, instead of developing equations that describe the underlying dynamics driving the formation of a certain type of cloud and its effect on the climate, the researchers could use data from satellites and weather balloons to simulate cloud formation patterns observed in the real world. One of Gentine’s students uses these techniques to model the effect of leaves forming in the spring and falling in the autumn. Another center student has used them to model changing ice sheets far more accurately than was possible using traditional methods. 

It turns out that AI methods are extremely useful. Over the last several years, researchers have been able to make predictions about the chemical, geological, and biological cycles that drive the Earth’s climate more accurately than ever before. 

But there’s a problem. Climate models are hungry for data — especially when they use AI methods. Today, the infrastructure that makes that data available for research is woefully inadequate for the gigantic task of predicting the future climate.

A data bottleneck

In the last 10 years, the number of satellites collecting data about Earth has increased exponentially. This wealth of information is incredibly valuable, but it’s proven difficult for scientists to put it to good use.

“The issue we’re facing now is a data bottleneck,” Gentine said. These satellites belong to different organizations — such as international research consortiums, national space agencies, and private companies — that don’t deliver data in a uniform way. There’s even variation within organizations. “Each satellite launched by NASA had its own portal with its own format” until recently, he said.

This massive data infrastructure gap makes it difficult or even impossible for researchers to access, download, manipulate, and use the many terabytes of data organizations are collecting. 

“Many of those satellites are not being used as much as they should. To be honest, many are barely being used at all,” Gentine said. “We have this wealth of data, but we’re using very little to evaluate our climate models. There’s a disconnect between the amount of data that’s acquired and the amount that’s used, which is very tiny.”

Making sense (and use) of the data

One of LEAP’s central missions is to address this issue through a cloud-based repository for climate data and models.

“We’re trying to organize the data and put it in the cloud so it’s always in the same place and always reachable,” Gentine said. “We also want users to do a lot of their computations in the cloud.”

The effort, called LEAP Pangeo, is a community effort connecting scientists and technologists to improve scientific computing with modern technologies. The project’s roots trace back to work initiated by Ryan Abernathey, a computational physical oceanographer at the Columbia Earth Institute.

“It became a grassroots effort with a lot of people in academia and government chipping in, basically pro bono, to create a community,” Gentine said. “We’re leveraging that at LEAP, where we have a centralized infrastructure that we’re using to support this community and provide data at scale.”

LEAP also aims to provide data to organizations across the world. The center is working with the MTA and the New York City Mayor’s office to assist with climate adaptation plans. Through its partnership with a nonprofit called Climatematch Academy, LEAP is working to provide climate data and computation tools to partners in the Global South.

“The only thing you need is a laptop and an internet connection to visualize — and potentially even analyze — the data,” Gentine said.

This kind of information could allow communities across the planet to plan for disasters like drought, flooding, wildfire, and sea level rise years or even decades before they occur, Gentine said.

Even if we move quickly to limit greenhouse gas emissions, the climate will continue warming for decades because Earth's system has a long "memory" of the emissions that have already been released.

"You need to go beyond 2050 or 2060 to see the imprint of the emissions choices we make today," he said. “Until 2060, there's no way around adaptation.”