A Rainfall Forecast Worth its Salt

Seven-to-ten day forecasts help us plan our daily lives, but when it comes to sub-seasonal to seasonal forecasts—the 3-week to 3-month weather predictions we don’t rely on the evening news for—there may be a lot more at stake. Droughts, for example, can put millions at risk for food insecurity and water management. Extreme floods and hurricane seasons can destroy neighborhoods, businesses, and infrastructure.

When WHOI scientist Ray Schmitt heard about a U.S. government-sponsored weather forecasting competition aimed at improving sub-seasonal forecasts, he wanted in. As a climate scientist, he had been on his own winning streak in predicting seasonal rainfall in regions ranging from the African Sahel to the U.S. Midwest.

“A few days before Christmas in 2016, I got an email from my WHOI colleague Steve Elgar that pointed me to a weather prediction contest that had just been announced by the U.S. Bureau of Reclamation (USBR),” said Schmitt. “They were offering big prize money if you could provide the most accurate rainfall predictions over a whole year for the U.S. West in their ‘Climate Forecast Rodeo’.”

Getting into the game, for Schmitt, was a way to leverage a unique rainfall prediction technique he’d been honing at WHOI for decades. Rather than basing seasonal forecasts on the El Nino phenomena, which is what most forecasts focus on, his method uses salinity in the ocean as a natural rain gauge.

Ray’s Rule

The idea struck him in 1993 after the Mississippi and Missouri River floods. After the seven-month dousing of the region, he noticed reports of abnormally low salinity in the Gulf of Mexico and Gulf Stream.

“Since all that flood water was freshening the ocean, conservation of water and salt requires that some part of the ocean had to get saltier before the floods. A lot of freshwater had to leave the ocean ahead of time to supply the extra rainfall on land.” he said. Using “Ray’s Rule”, that areas of water export must equal areas of water import, he realized that a large region of the ocean had to lose fresh water to evaporation in order to supply the flood on land. That evaporation would have to make the surface ocean saltier than normal. “With that basic concept in mind, I began looking at high-salinity areas of the ocean to see if variations in salinity could be used to predict rainfall.”

By May, 2016, the line of investigation took a fruitful turn. Schmitt and Laifang Li, a postdoctoral scholar in physical oceanography at WHOI, with co-authors Caroline Ummenhofer and Kris Karnauskas, published a study showing that high springtime salinity levels in the subtropical Atlantic Ocean correlated closely with increased summer-season rainfall in the African Sahel, where even small shifts in rainfall patterns can be a matter of life and death for millions of people. “It was a remarkable result,” said Schmitt.

The group also successfully applied the technique in summer rainfall predictions for the U.S. Midwest. Li and coauthors discovered that summer rains in the region correlated with springtime salinity in the western North Atlantic. This meant that if the waters from Cape Cod, MA to the Gulf of Mexico were saltier than normal in the spring, the Midwest would see summer flooding.

“Based on the salinity data in March of 2015, I was able to stand up in front of a crowd at an American Meteorological Society meeting in early June and forecast a rainy summer in the Midwest,” said Schmitt. “The prediction turned out to be quite accurate—there was severe flooding in Indiana and water levels in the Great Lakes were restored after a long drought.”

Off to The Rodeo

Schmitt’s idea, however, proved unpopular with conservative peer reviewers.  By fall of 2016, after numerous proposals and three published studies, he still had no funding for the idea and no team to pursue further research. But that didn’t change his mind about entering the forecast rodeo competition. Beyond his novel forecasting method, Schmitt had two other unique things that he felt might increase his odds of winning the competition: his 28-year-old identical twin sons.

Eric, a mechanical engineer who had dabbled in artificial intelligence (AI), and Stephen, a computer programmer, happened to have the engineering chops to train AI models on historical ocean data, which would provide the input for the sub-seasonal rainfall forecasts required in the competition.

“The contest called for the most accurate predictions over a whole year,” said Schmitt. He explained that every two weeks, they would have to predict the total rainfall amount in all the regions of the West for weeks 3 and 4 in the future, and weeks 5 and 6 in the future.

“I could see it would be a lot of work and knew I could not do it myself,” he said. “Two days after I heard about the contest, I saw Eric and Stephen at the family gathering for Christmas and learned of their interest in AI. I started working on them, telling them I had a technique that no one else would try and promising the reward money if we won. The first-place prize of $100,000 in each time category was a strong motivator!”

The twins, however, didn’t expect to win heading into the competition. Eric thought it would be fun to work on the project with his brother and father, but neither he nor his brother were climate scientists and he knew that “precipitation is pretty chaotic.” But eventually, they signed on.

“Previously, I had been playing around with AI, mainly focused towards improving our performance in “Dota 2”—an on-line video game that Stephen and I play,” he said. “We discussed the competition and figured it would be a good opportunity to test out our AI skills.”

Team Salient

And so the father-and-sons team—‘Team Salient’—was born. They went to work, chipping away at code throughout 2017 to build their “neural-network-trained” rainfall model. Eric designed the AI algorithms and Stephen coded it up to run on his home computer.

The team began submitting forecasts in late April, 2017 and about a month later, Schmitt received an email from the USBR saying the first scores were in.

“I couldn’t check the website as I was running a meeting about salinity with 100 people at WHOI,” he said. “I was busy introducing the speakers so I just forwarded the email to Stephen.”

At lunch break, Schmitt checked his inbox again and one of the emails got his attention. “Stephen had sent the score list,” he said, “and we were at the top with a very accurate forecast score and well ahead of everyone else!”

“We were blown away at how well we did in our first precipitation prediction,” said son Eric, “and that motivated us to work on improving our AI strategy even more.”

After a full year of competitive forecasting, Schmitt and his sons beat out all other competitors in the rainfall forecasting category, leading nearly the whole time and ending up well ahead of all other competitors—including professional forecasting companies and a climate model developed by the National Oceanic and Atmospheric Administration (NOAA). Team Salient took the first-place awards in both precipitation categories plus bonus awards for an 11-year “hindcast” for a whopping take-home prize of $250,000.  Schmitt said, “The win in the hindcasts was especially gratifying because it shows that we have real forecasting skill and were not just lucky during 2017-2018.”

Beyond the Glory

As thrilling as the win was for Team Salient, Schmitt is particuarly excited about the potential of the forecasting technique iteslf. He feels it can provide a path to better sub-seasonal and seasonal rainfall predications. The USBR, which manages many of the dams, hydro-electrical plants, and reservoirs scattered throughout the western U.S., can leverage the model Schmitt and his sons developed to prepare water managers for shifts in weather patterns that can lead to droughts or wet weather extremes. And Schmitt can use his new prediction system to create customized rainfall predictions that farmers, city planners, wildfire fighters, and others can use to get a read on precipitation weeks to months in advance.

“Our work is a great example of science in service to society,” said Schmitt. “Our model was accurate in forecasting the drought that led to all the western wildfires last year, and we have extended it to the whole U.S. and can easily go global. Knowing the future availability of water in general is vitally important for food security as the world population heads toward nine billion in the next few decades. This work also demonstrates the central role of the ocean in the climate system and how essential it is to continue monitoring of its properties.”