IntenseStormNet and ProbSevere

The deep-learning model IntenseStormNet uses only image patches of GOES-R ABI and GLM data to detect the most intense parts of storms. Specifically, it uses the 0.64-µm reflectance, 10.3-µm brightness temperatures, and the GLM flash-extent density. In this way, IntenseStormNet tries to holistically use spatial patterns and textures to predict the probability of intense storms from a purely satellite perspective. Figure 1 shows the output of IntenseStormNet for several supercell or supercell-like storms in Texas. Notice how the model predicts high probabilities in areas of cold cloud tops, bubbly cloud-top texture, and strong overshooting tops.

Figure 1: Output from the IntenseStormNet (probability of intense convection contours), GOES-16 sandwich product (0.64-µm reflectance and 10.3-µm brightness temperature), and severe storm reports.

Though the IntenseStormNet model isn't being evaluated explicitly in the HWT this month, it serves as an input into ProbSevere v3 (PSv3), so that the PSv3 models can utilize the rich spatial and spectral information that GOES-R provides.  The IntenseStormNet output is often in the top third of contributing predictors for ProbSevere predictions.

Figure 2: ProbSevere contours, MRMS MergedRef, and NWS severe weather warnings for two supercells in Texas during the HWT. The outer contours are colored by the probability of tornado. The time series window is for the rightmost storm.

Midland storms and ProbSevere v3

A slew of storms entered Midland's CWA. Both forecasters in Midland noted how ProbSevere v3 probabilities seem to better match their own subjective threat levels for hazards. This right-moving storm in Figure 1 has a probability of severe of 66%, compared to 34% in v2. It later dropped golfball-sized hail. 

One forecaster working this office said he likes how ProbSevere integrates and quantifies their own internal thought process to make objective and consistent guidance for warning decision-making.

Figure 1: ProbSevere contours, MRMS MergedRef, and NWS severe weather warnings for a storm entering Texas.

Forecasters working in Texas

Day 2 of the HWT has forecasters working in Midland, San Angelo, and Dallas/Fort Worth, Texas. The forecasters in DFW are also working a DSS event (PGA event in Fort Worth).

Storms are getting started along a dryline and other boundaries in south Texas (Figure 1). LightningCast picked up on these regions pretty well, with lead-times of 15, 25, and 60 minutes from the 50% threshold, for three different cells.  

Figure 1: LightningCast contours, GOES-16 day cloud convection RGB (1-minute meso scan), and GOES-16 GLM flash-extent density over south Texas.

GOES-R Proving Ground Spring Experiment 2022

Today marks the beginning of the 2022 GOES-R Proving Ground Spring Experiment at the HWT! Forecasters will get acquainted with products today in the Lubbock, TX and Columbia, SC county warning areas.

ProbSevere LightningCast is one product being evaluated this month. LightningCast provides probabilistic next-hour lightning guidance using deep learning and GOES-R ABI data. It aims to provide objective, quantitative guidance for convective initiation, maintenance, and decay. For the HWT, the data are supported in AWIPS as contours of probability (Figure 1).

Figure 1: LightningCast contours, GOES-16 GLM flash-extent density, and GOES-16 Meso2-sector day-cloud RGB, over south Texas.

ProbSevere v3 (PSv3) is also being evaluated for the second consecutive year. PSv3 utilizes gradient-boosted decision trees for its ML engine, enabling better use of all of the pertinent data available to forecasters, distilling the fire hose of data into actionable information.

Last Friday, Gaylord, MI was struck by an EF-3 tornado (see more details here). ProbSevere had it tracked well over Lake Michigan and into the lower Michigan (Figure 2). This week is looking quieter, so hopefully we won't see any tornadoes with major damage.

Figure 2: ProbSevere contours, MRMS MergedRef, and NWS severe weather warnings for storms in lower Michigan.

Figure 3: The Gaylord EF-3 tornado with ProbSevere v3 time series window.

Forecasters are also evaluating the ProbSevere AWIPS plug-in, now with a time series capability. A forecaster can double-click on a storm object, and the time series window pops up. This enables forecasters to more quickly interrogate the trends of storms. 

Salado Tornado

A supercell quickly developed on the southern flank of an arc of storms in central Texas on Tuesday, April 12. High CAPE (≥ 3000 J/kg), effective shear (≥ 50 kt) and effective SRH (≥ 170 J/kg) all contributed to an elevated probability of tornado from ProbTor v3 (PTv3) by 22:00Z. About 30 minutes later, the supercell produced an EF-3 tornado west of Salado and south of Killeen, TX. 

Figure 1: ProbSevere contours, MRMS MergedReflectivity, and NWS severe weather warnings in central Texas. The storm that produced the EF-3 tornado traveled south of Killeen, Texas. 

In the critical early stages of storm development, PTv3 probabilities exceeded PTv2 probabilities, which is noteworthy given PTv3's better-calibrated guidance. From Figure 2, we see that prior to the initial NWS tornado warning, PTv3 was consistently 10-20% greater than PTv2. Because PTv3 is better calibrated than PTv2 (i.e., probability value better match tornado occurrence), PTv3 will rarely exceed 60%, whereas PTv2 regularly hits 80-90% (but PTv2 over-predicts in that range).

Figure 2: Time series of PTv3 and PTv2 for the tornadic storm west of Salado, TX.

At 22:14Z, PTv3 = 30%, while PTv2 = 7%. In a post-mortem analysis, we found that the MESH, mid-level azimuthal shear, and effective bulk shear were the top-3 contributing predictors. The 4th leading predictor was the probability of intense convection produced by the ProbSevere IntenseStormNet. A rapid increase in this value from 29% to 99% occurred from 21:52Z to 22:14Z (see "ICP" in the meteograms). IntenseStormNet is a GOES-ABI and GOES-GLM-based convolutional neural network, which picked up an a developing cold-U signature and increasing lighting to produce a very high probability of "intense" convection (see the animation below). In this way, it provides a holistic method of leveraging important values, textures, and spatial features found in geostationary imagery. In ProbSevere v3 models, IntenseStormNet computes one value per storm per time step, which is used as a predictor. 

Figure 3: Intense convection probability contours overlaid GOES-16 "sandwich" imagery from a 1-min mesoscale scan. Note the rapidly developing supercell on the south flank of the developed convection.

ProbSevere v3 infuses spatially important satellite information into its predictions. This example shows that forecasters should pay especially close attention to developing storms when PTv3 is exceeding PTv2.

Storms pummel the Midwest

An energetic short-wave and attendant surface low rapidly intensified on April 12th, bringing quickly developing storms to a number of regions in the Midwest U.S. Large hail, severe wind gusts, and potent tornadoes were reported from Wisconsin to Texas.

Figure 1: SPC categorical outlook at 06 verification.

In the middle of the afternoon, a lone elevated storm along a stationary boundary traversed the state of Wisconsin, causing a 67-mph wind gust in La Crosse, WI, and dropping hail ranging from 1" to 1.5". ProbSevere version 3 (PSv3) had a pretty good handle on it over the course of several hours. 

Figure 2: ProbSevere storm-based contours, MRMS MergedReflectivity, and NWS severe weather warnings for a storm in Wisconsin.

While the MRMS products contributed positively to the PSv3 probabilities (e.g., MESH, Reflectivity -10C, AzShear), the IntenseStormNet probability (a predictor in PSv3 models) also contributed in the models. IntenseStormNet uses images of visible and long-wave infrared channels from GOES-R ABI, as well as images of flash-extent density from GOES-R GLM to detect intense parts of storms. From the animation below, we see that IntenseStormNet "probability of intense convection" for the storm in Wisconsin largely stayed between 50% and 90%.

The storm of the day spawned in northeast Iowa, ahead of a cold front. From ProbSevere hover-output in AWIPS, we saw that it had a strong normalized satellite growth rate at 21:31Z. PSv3 was 32% when the NWS issued its first severe thunderstorm warning, at 22:00Z. The probability of severe then soon increased to 70% by 22:18Z. The storm produced its first 1"-diameter hail report at 22:25Z. 

Figure 3: An animation of ProbSevere contours, MRMS MergedReflectivity, and NWS severe weather warnings for a tornadic storm in Iowa

A tornado warning was issued at 22:59Z, coincident with ProbTor v3 rapidly increasing to 49%. ProbTor v2 was at 13%. There was an increase in the MRMS azimuthal shears at this time, along with an increase in the significant tornado parameter (a predictor in PTv3). v2 was likely underestimating the threat due to too much contribution from stout MLCIN (-86 J/kg), dampening the probability. The machine-learning model of PTv3 (gradient-boosted decision trees) appears to better incorporate the MLCIN information than it's predecessor, in this case. 

Figure 4: ProbSevere contours, MRMS MergedReflectivity, and NWS severe weather warnings for a tornadic storm in Iowa. The ProbSevere time series window can be activated by double-clicking inside a storm object.

The IntenseStormNet's probability also contributed to the higher probability of tornado. See in the animation below how probabilities ≥ 90% are well-correlated with the most vigorous portions of the convection.

 

Later, the cold front zipped down from Nebraska into Kansas. Very strong satellite growth rates were observed, as the PSv3 values regularly exceeded 80%. The cold front was essentially warned continuously from western Iowa to southern Kansas.

Figure 5: ProbSevere contours, MRMS MergedReflectivity, and NWS severe weather warnings for a cold front from Iowa to Kansas.

The intense convection probability from IntenseStormNet quickly went from < 10% to ≥ 90% for most of the line, which later produced numerous hail and wind reports.

Allendale, SC tornado

Allendale, South Carolina sustained much damage after a major tornado tore through parts of the town on April 5th, 2022. The NOAA Storm Prediction Center issued a 10%-hatched risk of tornadoes through parts of Mississippi, Georgia, and South Carolina the morning of the storms. Strong low-level flow and destabilization led to supercells forming ahead of a squall line, such as the tornadic supercell that hit Allendale.

Figure 1: SPC tornado outlook and 06Z verification. 

A ProbSevere v3 (PSv3) model, ProbTor, tracked this storm from Georgia into South Carolina. The probability of tornado rapidly increased about 30 minutes before and again 15-20 minutes before a tornado was reported around Allendale. Increasing azimuthal shear, radar reflectivity, and lightning were noted prior to tornadogenesis. See here for meteograms of different predictors for this storm.

Figure 2: ProbSevere v3 (storm contours), MRMS MergedReflectivity, and NWS severe weather warnings. The outer PSv3 contour is colored by the probability of tornado.

One new feature at the HWT last year was the ProbSevere time series or meteogram function in AWIPS, which can be activated by double-clicking a ProbSevere time object. The window displays the latest 2 hours of probability history for the featured storm for all four ProbSevere models (hail, wind, tornado, any severe). The meteogram updates automatically as new data are processed. This feature helps forecasters more quickly interrogate storm trends and will again be available to forecasters at the 2022 HWT. 

Figure 3: The ProbSevere time series window for the tornadic supercell in Allendale, SC.

ProbTor v3 uses a different machine-learning model than ProbTor v2 (gradient-boosted decision trees vs. naive Bayesian classifier). While the maximum CSI for PTv3 is about the same as PTv2, the PTv3 probabilities are much better calibrated. What this means is that the output probability values much better match the observed frequencies of tornadoes, for any given probability value. Users should see much lower false alarm rates at higher probability bins. Given the inherent noise in doppler radar velocity data (and downstream MRMS azimuthal shear), and inherent uncertainties in detecting tornadoes, this also means that PTv3 values over 60% are exceedingly rare.

Compare the attributes diagrams for PTv3 and PTv2 below. A perfectly calibrated or "reliable" model will have predictions follow the 1:1 line. Notice how PTv2 over-predicts, while PTv3 is very close to the 1:1 line, except for some under-prediction around 50-60%. The most-skillful (i.e., highest CSI) probability range for PTv3 is 20-40%. The University of Wisconsin / CIMSS is actively working on improving ProbTor, experimenting with additional data and methods that make better use of the spatial patterns found in satellite and radar data. 

Figure 4: Attributes diagrams for PTv3 and PTv2 on a validation dataset from 2021.

The IntenseStormNet detects particularly intense storms from a satellite-only perspective, using deep learning and images of ABI and GLM data. Using GOES-16 one-minute mesoscale scans, IntenseStormNet reached over 90% on this storm about 10 minutes before the first tornado report. The higher IntenseStormNet probabilities corresponded well to a GLM lightning jump and overshooting tops in ABI imagery. While a tornado warning was already in effect, seeing this feature could add confidence to the warning forecaster. The output of IntenseStormNet is also used in the ProbSevere v3 models.

Devastation in the Mid-South

Tornadoes wreaked havoc across the U.S. Mid-South on Friday night into Saturday, as an energetic shortwave trough tapped into the abundant moisture and atmospheric instability in the southern U.S. One supercell, persisting for at least 11 hours, spawned a very long tornado (probable path length > 200 miles), generating some of the night's worst damage in Mayfield, Kentucky. Meteorologist Jack Sillin documented the forecast of the supercell's evolution and some preliminary facts (Figure 1). With over 100 tornado deaths, the day was the deadliest since 2011. 

Figure 1: Depiction of NWS forecast evolution of the Quad-State Supercell, by Jack Sillin.

ProbSevere guidance is used by NWS forecasters to aid in warning decision making. A new version of ProbSevere (version 3) is being developed and tested at the University of Wisconsin, honing and improving the probabilistic guidance. For the quad-state supercell, ProbSevere v3 (PSv3) was generally 10-20% higher than PSv2 in the hours before it became severe (Figure 2). For instance, PSv3 was 68% at 22:28 UTC, compared to 46% for PSv2. The top predictors contributing to the higher probability were:

1. 1-3 km mean wind (48 kts)
2. 3-6 km MRMS AzShear (0.011 /s)
   
3. 0-2 km MRMS AzShear (0.012 /s)
4. Effective bulk shear (65 kts)
   
5. Effective SigTor Parameter (1.73)
   
6. Normalized satellite growth rate (2.8%/min; "moderate")

Tornado probabilities were higher for v2 than v3, which we've found to often be the case (i.e., ProbTor v3 is more conservative). However, the trends in both PTv3 and PTv2 matched the trending threat of the supercell well. Users can inspect the storm's trends in probabilities and predictors here. 

Figure 2: Time series of ProbSevere v2 and ProbSevere v3 for the quad-state supercell. Note that severe reports are preliminary. 

Figure 3: ProbSevere v3 contours with MRMS MergedReflectivity at 03:30 UTC and NWS severe active weather warnings, when the tornadic quad-state supercell was in Mayfield, KY. 

Mesoanalysis Summary for E. CO

Looking at a surface map there looks to be a boundary, possibly a weak warm front, over E CO that storms are firing off of.  A similar feature can also be seen in the satellite data taken at the same time.

 

Lightning:

The FED did not give as much information about the growth stage of the updrafts as the flash minimum area did.  Also noticed the VII trend resembled a similar trend as the flash minimum area did.

The flash minimum area is also a good way to help catch the eye of what updrafts are strengthening, especially if the trend of low flash minimums persists.  Great tool to use at first glance of which storms need to be watched and which don’t.

-Dwight Schrute and Accas

Saw several examples of the flash density for lightning either muting out or not showing the trend the flash minimum area was showing.  In the past I have been using the flash minimum area to help me see trends in the lightning, but am now seeing that I should be using the flash minimum area instead if I want to see trends in lightning activity.  I use the lightning trends to help me know if the storm is rapidly intensifying or suddenly weakening and possibly about to generate a severe downdraft.  Being able to see these sorts of trends better can also help communicate a potential threat for storm intensification or severe wind development to those in the path of the storm.

-Accas

Area of coverage greater for the  minimum flash vs extent density.

Next time stamp, we can see increased minimum flash area lightning over the new updraft and a pixel from the flash extent density. So the minimum flash area would likely be the best bet for using the tool with decision support services in mind due to its higher sensitivity.

-Dwight Schrute

This was a scenario where we were baffled by how little lightning was being shown from both the minimum flash area and flash extent density products. We asked why so little lightning compared to how much ice is in the storm, combined with MESH indicating a 2” hail stone.  The lightning with this maturing storm was not being sampled well.

-Dwight Schrute

Gridded NUCAPS Issue WI/MN

 

Sampling Total PW gridded NUCAPS with contours overlaid on top, we see that the values are unrealistically high (over 3 inches). The 18 UTC soundings from the Twin Cities offices shows PWATs only around 1.30” which confirms this is incorrect. Also, the contours are in centimeters, not inches which is what the images. I plotted the NUCAPS sounding points to see if the points were “yellow” but it looks like the points were unavailable at this time step.

From this time step, the gridded NUCAPS matched up very well with the special 18 UTC soundings and the total PW values are realistic and make sense given the environment. Also the NUCAP points are all green which solidifies that the satellite was able to obtain a good sample.

– Fear the Shear