End of the experiment

Well, this marks a close to this year's Spring Experiment activities. We will begin again in the late summer this year with the Fire Weather and Heavy rain experiment starting August 24 and ending September 3. I would like to thank all of the participants this year from the various cooperative institutes and partners for coming and assisting in the product training and participating in the experiments alongside the forecasters. Without this participation, the experiment would not have been a success. I would also like to thank all the forecasters who participated this year and took valuable time away from their WFOs to come out and give us their time. The feedback gathered from the forecasters in the EFP and the EWP this year provided invaluable information to help improve the GOES-R products we evaluated this year. While not all feedback gathered this year may have been positive, it is important to remember that understanding the shortcomings in the demonstration strategy and the products themselves is essential to develop a robust product set prior to operational use once GOES-R launches. I would particularly like to thank Andy Dean, Gregg Grosshans, Israel Jirak and Chris Melick from the SPC's Science Support Branch for setting up and providing technical support for the NAWIPS systems on the EFP side. I would also like to thank Ben Baranowski and Darrel Kingfield from WDTB, as well as Kristin Kuhlman, Kevin Manross, Greg Stumpf and Travis Smith from NSSL for their assistance in setting up the AWIPS systems and providing technical support for the EWP side throughout the weeks. Without all of these people there would have been no experiment and I greatly appreciate all of their efforts. We are looking forward to continued interactions with the forecaster and product developer communities throughout the next year and into the coming years.

Thank you!

EWP weekly debrief

Today is the end of this year's Spring Experiment, and what a fitting end. Yesterday was our busiest day in the EWP over ND/MN/IA. As of the time of this entry there were 65 tornadoes reported over the area associated with the event. It was indeed a big day, and luckily there were no technical issues during the IOP. Unfortunately, due to the rapid developing nature of this event the GOES-R products were used only for a short time before it was essential to move into radar operations for warning purposes. The forecasters were inundated with issuing tornado warning pretty much in a consistent line from Canada down into IA. We were able to get some feedback and since this is the weekly debrief I was able to ask the forecasters in more detail, based on my observations this week as well as their survey responses. Here is a breakdown of what we discussed...

UWCI
I asked about the lead times from the surveys... 15-30 mins the general consensus?
"Hard to be sure because we were arriving at the HWT after convection was developing."
"Getting in there around noon would get more accurate results."
"The case event was already initiated when I got there."

"Struggling giving up screen space during severe weather" (from survey)... is there a display you think would help?
"Having an additional head would help. We were only provided with two screens so it changed our strategy."
"Would be useful for the mesoanalyst position at the WFO."
"4-panel devoted to those products works well."

I asked the forecasters if they were ok with more signals if that meant giving up some FAR...
"I would like earlier signals, like a probability signal... which part of my CWA is going to have the best chance... Some kind of signal before there's aggitated clouds would be helpful."

I asked the forecasters whether they preferred the CI or the cloud-top cooling...
"I used them both... kind of liked them both simultaneously."
"It was hard because it seemed it was 50/50 on detecting things so I lost a lot of confidence in using the product, but I did like the cloud-top cooling a little better because it seemed to do better."

I asked the forecasters if they preferred the accumulated of instantaneous fields...
"I stuck with the instantaneous for the most part, I don't think I even looked at the accumulated to be honest."
"I tended to look at the instantaneous."

Would you see a benefit in having a cloud-top cooling track?
"Yes... an overshooting top track would be very useful as well."

OTTC
Detections or magnitudes more useful?
"I looked at the detections only."

Did you see any correlations between OT detections and features developing on radar?
"Reflectivity was increasing above certain (height) levels at that time."

How often did the product correctly detect OTs that you could see on VIS/IR?
"Kinda 50/50."

"For a warning operator, I don't see this product adding much value to an on-going event..." (from survey) Why is that? Lack of detection, timeliness, applicability of product?
"Radar was more important... mix between timeliness and applicability."
"Get a good a western example case where you don't have radar."
"Maybe knock out some radars in a WES case."

PGLM
"It's an acquired taste, working with lightning data. I think it's a fundamental component in the decision making process trying to determine what's going on with the updraft... it's an interesting way of looking at it."

"With supercells I don't see any added benefit, unless it flared up right before an RFD developed... but I would have to see a lot of cases before I could determine that."

"Would be of a lot of value in low top events."

"Would be nice to see it in a non-supercell case event, like winter events."

"In terms of just lightning forecasting... I think the GLM stuff you were showing would be very useful in that area."

OVERALL / TRAINING
Overall feelings following the week...
"I'm 50/50... I'm not leaving here warm and fuzzy if that's what you're asking."
"When I came in I was very excited to have the convective initiation... I don't come out feeling as confident as I did and as excited as I did when I came in."
"The word convective initiation tool sounds awesome, maybe I went into it with high expectations as maybe seeing a tool with some probabilistic information."

How confident were you with the products following the training?
"The products were easy to understand so I was confident in using them."

What changes/additions would you like to see with the training?
Cases in the west re-iterated.
"The training itself was fine... it just seemed that it didn't perform as well as we would like to see."

I asked if the forecasters would like some sort of alert for the GOES-R products (UWCI, OTTC), so they would know when a detection was occurring, or if that would be more annoying than helpful...
"An initial alert would be nice... a continuous one would be overload."
"That would vary forecaster to forecaster."

Re-iterated the need for wind-based warning products.

"When I got really busy, I went back to the things I was used to using. I was just falling back on my normal routine."

"Would be nice on the first day to have a sort of hand-holding process to get used to the products."

"Maybe already have a base set of procedures so we don't have to spend 30 minutes setting everything up... especially on the first day."

Need cases where radar was out or nocturnal events over poor radar coverage.

Review of yesterday's event... an EWP perspective

24-hour UWCI nowcasts (top left), overshooting top (top right) and thermal couplet detections (bottom left), and SPC storm reports (bottom right) for 17 June 2010.
Today we discussed yesterday's event over ND/MN/IA and I wanted to mention a few things we discussed with the forecasters regarding the performance of the UWCI and OTTC products leading up to and during the event. We showed the forecasters the 24-hour accumulations of UWCI, cloud-top cooling, overshooting top and thermal couplet detections with regards to severe reports from the SPC (see images above), as well as their own experiences. Forecasters did notice that cloud-top cooling was occurring on southern edge, but it was already ongoing on radar. The first UWCI signals were seen at 1855 UTC with the convection that was developing over ND and MN. The first overshooting top was detected at 1915 UTC over northern MN. These occurred just prior to beginning operations in the HWT, so it is hard to determine the CI lead times for this event given that we did not see the first detections of CI. I reiterated that we are hoping to expand the experiment in the future years by having overlapping shifts that would provide additional time to evaluate GOES-R Proving Ground products in a nowcasting perspective, which is what they are meant for originally. Forecasters did not see any thermal couplets in real-time, but did notice a few overshooting tops, especially when they began. However, they mentioned that the storms seem more low topped in nature and this may have hindered the products ability to detect these features. They were honest with me in saying that as soon as they moved into warning operations they turned the GOES-R stuff off because they did not feel it provided them with additional warning information and needed the window space for their traditional radar products.

Jordan Gerth did show the forecasters an AVHRR image that demonstrated similar to what GOES-R would see regarding overshooting tops and thermal couplets and noted that the detection efficiency would be much better. The forecasters were impressed with this and agreed that the products would perform much better in the future and appreciated us showing them the products now. The forecasters also did note that in the case of a line forming upstream from main convective area, convective initiation being detected in the line behind would help prevent getting tunnel vision and increase their situational awareness.

SATCAST improvements identified

SATCAST CI nowcast overlaid on visible satellite imagery for 1945 (top left), 2002 (top right), 2015 (mid left), 2033 (mid right), and 2045 UTC (bottom center) on 17 June 2010.
Today we were able to remedy the issues we were having with the SATCAST product within our NAWIPS system. While examining today's potential severe weather outbreak over ND/MN/IA I noticed a few issues that have been relayed to the developers for future improvements. The product did not detect development over southwestern MN where some obvious convective initiation occurred between 1945 and 2045 UTC (see images above). We examined the cloud object detections provided within the output (see images below) and noticed that this area was not widely flagged as having any cloud objects within it. I discussed this with the developers and it was determined that because the clouds were developing so rapidly, the 15-minute cloud typing product used by SATCAST produced cloud types that changed so rapidly that the SATCAST algorithm's quality control threw them out. It should be noted that GOES-13 was operating in RSO during the course of the day, but due to limited computing resources it is not possible to run the product outside of the 15 minute CONUS scans at the moment. It is theorized that this issue would not exist given that the product was able to utilize RSO scans. This does help highlight a major benefit of utilizing higher temporal resolution data that will be available continuously when GOES-R is operational.



SATCAST CI nowcasts (red) and detected objects (blue) for 1945 (top left), 2002 (top right), 2015 (bottom left) and 2045 UTC (bottom right) on 17 June 2010. Note that the 2033 UTC image was not generated and the 2045 UTC image has no objects or CI nowcasts.
It also seemed as though the product was not producing output for the 2032 and 2045 UTC time periods (see images above). I also mentioned this to the developers and they discovered that the algorithm could not handle the scanning schedule changes that occurred with GOES-13 RSO. The algorithm has been changed to accommodate the scan schedule and the output has resumed.

EWP daily briefing... 6/17/2010

24-hour severe reports from 16 June 2010.
At the beginning of the briefing we discussed yesterday real-time IOP event over the Sterling, VA WFO CWA. We had the ability to use the PGLM output from the DCLMA for this event. Most of the warnings issued were for severe wind threats and were fairly marginal. One of the forecasters mentioned that he focused on the PGLM products because of the marginal nature of the event to try and pick out regions of greater importance since the radar and MRMS products were showing similar features for most of the convection around the area. However, due to the marginal nature of the event, no strong conclusions were drawn as to the usefulness of a lot of the experimental data. One of the forecasters brought up the idea of creating regional versions of some of the products to try and draw more information from them. This may eventually be useful for products like the CI and cloud-top cooling products since we have seen some regional dependence on the product performance (ie - diagnostic over SE US)

24-hour UWCI nowcasts for 16 June 2010.
We also discussed the performance of the UWCI over the same area. The forecasters reiterated the lack of detection over WV/VA border from yesterday's blog post. We believe that this was due to some microphysical issues being presented in the cloud typing product that were masking out the CI signals. We mentioned that we had wished we had moved to SD sooner, which is where we ended up for the remainder of yesterday's IOP. There were a lot of signals over that area (see image above), but we were unsure as to the timing of them since we were not watching in real time. This would be a good case to look at in the future. Also, notice the lack of signals over the Sterling and State College WFO areas where we focused at the beginning of the day. Jordan Gerth discussed with the forecasters about the idea of providing a cloud-top cooling rate 'image' with no filtering, which would allow for the forecasters to see all areas of cooling/warming regardless of cloud motion or other effects that are currently removed. This would allow the forecasters decide for themselves what are false signals and may be particularly useful for determining the life cycle of MCS's and weakening supercell storms.


24-hour overshooting top (left) and thermal couplet (right) detections for 16 June 2010.
We examined the overshooting top and thermal couplet detections (see images above) and saw that both the overshooting top and thermal couplet detections did correlate well to the location of the severe weather in this case, especially over SD. However, when these were seen during the real-time IOP, it was already obvious that severe weather was occurring based on reports being received and the standard radar data. The overshooting top detections that continue further east, as well as the blob of CI nowcasts east of the tornadic storm were questioned by the forecasters since they occurred after the IOP ended and they were not sure how legitimate they were. We did look back very quickly to see what happened and it seemed as though a MCS developed later on associated with many additional warnings and it would have been nice to see the performance of the products in that instance. Again, this would be a good case to look at in the future. We will attempt to archive the data, but it may be difficult since we do not yet have dedicated disk space to do so.

The plan for today is to operate over MN and IA. This is our last chance for real-time IOPs and it looks like it should be a good one for GOES-R with the lack of cirrus in the area... assuming we get started soon enough.

Hail Probability Product from 16 June

The image above shows the severe hail probability product from 16 June, valid from 21-00Z. Also plotted are the severe weather reports between 18-00Z; hail is denoted by 'a'. Only a couple of the reports in Montana occurred prior to 21Z, the rest being observed within the hail product's 3-hour time window. For reference, the 2115 UTC GOES-East IR image is below. The product did a nice job of highlighting the regions in which hail was observed (and other severe weather in the case of eastern Kansas). In particular, note the small region near Sydney, Nebraska. There, towering cumulus had just begun to form and a bullseye of probabilities can be seen. Hail was reported there around 2200 UTC.

Use of Simulated WRF Imagery in the EFP

As part of the morning EFP session, we took a look back at yesterday's severe weather forecast. The domain was centered over State College, PA. Many of the model guidance products pointed toward Pennsylvania as a region for possible severe weather. However, as can be seen in the visible GOES-East image above (right), low clouds were present across the eastern part of the state, and the 15Z analyzed SBCAPE showed positive values only where solar insolation had occurred. The image on the left shows the simulated 10.35 micron band (from the NSSL WRF-ARW) also at 15Z, along with the model's SBCAPE field. Note that the model correctly places the low clouds in eastern Pennsylvania, but has CAPE values a bit too high in western PA.

The images above show the same fields, except valid at 22Z. Some convection had begun to fire in the unstable region in central PA, and the model also showed convection in this area. In this case, the WRF seemed to perform quite well with the evolution of afternoon convection. However, its overprediction of the instability lead to the model storms being a bit too strong. No severe weather reports came out of Pennsylvania.

EFP simulated satellite evaluation

EFP QPF group discussing simulated WV imagery from the 00Z run of the NSSL-WRF on 17 June 2010
Today in the EFP's QPF group Dan Lindsey led a discussion on the comparison of the simulated satellite imagery from the NSSL-WRF to that which was observed over the past few hours (see image above). They used this simulated satellite imagery to determine the accuracy of the current 00Z NSSL-WRF model run for the day. It was determined that is is missing some convective development down near NE and IA, but that the main threat area up near MN is being captured fairly well. This provides them with confidence that the model should handle the rest of the day's development fairly well.

EWP real-time IOP CI feedback

Today's IOP focused over the DCLMA domain to get some real-time PGLM experience. The forecasters broke up into two groups and localized over the Sterling, VA and State College, PA WFOs. The Sterling WFO group have been issuing severe thunderstorm warnings based on PGLM and MRMS products over the past couple hours. I was sitting with the State College WFO group as they were focusing on the UWCI products, watching for convective initiation over their area. One of the forecasters had multiple GOES-R Proving Ground products (UWCI, cloud-top cooling, overshooting top, and thermal couplet) into a 4-panel display and mentioned to me that this display strategy would be extremely useful as a situational awareness tool for monitoring satellite convective information.



GOES-13 IR imagery for 1832 (top left), 1845 (top right), 1902 (bottom left) and 1915 UTC (bottom right) on 16 June 2010. Area of interest noted by green circle.
We noticed an area of obvious convective development that was not detected by the UWCI product on the WV/VA border at 1915 UTC and the forecaster asked me to explain why no CI nowcast was made. Looking at the IR, you could see significant cooling occurring from 1832 and 1915 UTC (see above), but still no signals were seen. Jordan Gerth and I looked at the cloud typing output to see if we could draw any conclusions from that to determine what was going on (see below). Most of the area was covered by cloud types identified as 'water' or 'mixed phase'. However, there were a few spots of 'cirrus' classification over WV nearby where the CI nowcast should have been made. It seemed on visible and IR satellite that this may not have been the case, but this is hard to determine for sure. Jordan suggested that because there were these spots nearby that they may have been contaminating the spatial tests required by the algorithm to flag an area as filtered cloud-top cooling, and thus not allow for a CI nowcast to be made. This may be a fluke case where nothing can be done, but it may be useful to examine this in more detail to see if something can be improved.



UW-CIMSS Cloud typing product for 1832 (top left), 1845 (top right), 1902 (bottom left), and 1915 UTC (bottom right) on 16 June 2010. Area of interest noted by red circle.

Results from Tuesday's EFP forecast

Following up Tuesday's post on the use of simulated imagery, the low cloud deck turned out to be quite important for the storm evolution. Above is the observed GOES-East visible image from 1845UTC. The stratus deck had burned off between 15-18 UTC, and a cumulus cloud/no-cloud boundary was clearly evident in the visible image across southern Indiana and central Illinois. This boundary presumably resulted from the differential heating resulting from the morning clouds. Also note the storm which had formed on the boundary in eastern Illinois, and the multiple storms in southeast Missouri. These progressed to the east-northeast and subsequently grew upscale into a bowing MCS. The severe reports are below.

The blue dots represent severe wind reports. It's interesting that the northern boundary of the wind reports across central Illinois and Indiana corresponds quite well with the cumulus cloud boundary noted above. Most of the high-res models, including the NSSL WRF (for which we looked at the simulated imagery yesterday) had a bowing MCS further north. It turned out that the low clouds limited the instability further north, and the severe weather was confined to the south where more heating had occurred. This was a nice example of how the simulated imagery might be used to diagnose the model's treatment of morning clouds.

EWP daily briefing... 6/15/2010

Psuedo-GLM total lightning flash extent density overlaid with OKLMA flash contours and associated radar reflectivity for the OKC flood event at 1100 UTC on 15 June 2010.
During today's EWP briefing we discussed the previous day's flooding event over OKC from a PGLM perspective. Unfortunately the event unfolded in the morning hours when the EWP was not operating, so we were unable to view the data in real-time. Kristin Kuhlman presented the archived data and discussed with the forecasters what the PGLM showed during the event. She also overlaid the OKLMA flash contours to show how the PGLM relates to what was actually occurring (see above), which really helped the forecasters get a grasp over what the PGLM was showing them. They noted that the lightning seemed to be mainly focused with convection developing on the backside of the system and that this would help radar operators focus on which areas had continued development and represented an increased threat for flooding and severe weather in the future.



24-hour UWCI detections (top left), overshooting top detections (top right), thermal couplet dtetections (bottom left), and SPC severe reports (bottom right) from 14 June 2010
Jordan Gerth showed the UWCI and OTTC products over the past 24 hours in comparison to the severe reports from the same time period (see above). He pointed out the the UWCI had a lot of hits over the southeast with the diurnal convection that was not necessarily severe. He was unsure about how the UWCI performed with the the widespread swatch of wind reports through NC, but it is highly possible that the mass of UWCI signals in WV may be in initiation point for this convection... but we would need to do an in depth analysis of the event to determine this. Also, the numerous severe reports over southern IN seemed to have no UWCI detections associated with them. We suspected this was due to the presence of cirrus over the area. Jordan also noted the high density of overshooting top detections over OK with very few severe reports. He made sure to mention that the overshooting top detections are not necessarily associated with severe weather at the surface, as they are more of an indication of turbulence and lightning threats. There were only a few thermal couplet detections, which are highly linked to severe weather. The three located over TX seemed to be the best examples for this day. Jordan also did a quick introduction to the UW-CIMSS Nearcasting product for the EWP forecasters and showed a couple case examples in addition to the output for today to show areas of convective destabilizations associated with differential theta-e and precipitable water fields. Unfortunately we were unable to establish data flow for this product in time for this Spring Experiment, but we do plan on having it for future experiments.

SPC Day-1 outlook for 15 June 2010
The plan for the day is to once again start early and operate over the SPC moderate risk area in southern IL/IN to examine the GOES-R Proving Ground products prior to warning operations. With a relatively cirrus free sky, it should be a good event to demonstrate the UWCI. However, some storms are currently ongoing and are possibly severe, so we are hoping for some new development further south and west of the ongoing convection. The WFO's localized for the start of the IOP will be Indianapolis, IN and Louisville, KY.

Simulated satellite imagery in the EFP

On Tuesday morning, the Ohio valley region has been chosen for the daily forecast. The left image above is the 4-km NSSL WRF-ARW simulated 10.35 micron channel valid at 15 Z, and the image on the right is the observed GOES-East image from 1515 UTC. One of the biggest advantages of the GOES-R simulated imagery is the ability to evaluate the model's representation of morning clouds. In this case, the model appears to have a pretty good handle on the cold cloud placement. A more subtle but relevant feature is the low clouds across central and northern Indiana. These clouds may inhibit heating and limit the destabilization. At 15 UTC, the model has burned off the low cloud layer across central Indiana, but the observations show that the cloud deck is still present.

The image above shows the simulated band 13 image valid at 18 UTC. Note that the model has formed a new storm in southern Illinois which is moving into the area in question in Indiana. It will be interesting to see whether such a storm forms, and how it evolves as it moves into Indiana.

Hail Probability Product

Data from yesterday's probability of severe hail product was examined. The image above shows the probability forecast (%) from 20 UTC, valid from 20-23 UTC. Also plotted are the severe reports, where hail reports are denoted by 'a'. NAWIPS requires all reports from 18-00 UTC to be plotted at the same time, so not all of the reports above occurred during the 20-23 UTC forecast period. Specifically, the 3 southernmost reports occurred after 23 UTC.

Cold cloud tops from GOES-East dominated the input parameters, but the instability from the SPC mesoanalysis and the RUC forecast contributed. Note that the majority of the hail reports fell within the 1% contour in west Texas, although not specifically within the maximum probability area. The second max across Oklahoma saw no reports; this is a weakness of the product....if cold cloud tops are observed over regions of moderate instability, as was in the case in Oklahoma, higher probabilities will result. I looked at earlier forecasts, and on this particular day the product did not provide a significant lead-time prior to the development of storms.

Real-time PGLM operations

The forecasters are now engaged in a real-time IOP over Norman, OK and Lubbock, TX. Luckily the OKLMA became operational once again right before the dinner break so we are able to get the forecasters familiar with the PGLM product. The first two forecasters to look at the PGLM data had some interesting comments regarding the product's potential uses and performance. One of the forecasters mentioned how useful the product would be in the case of being overloaded with radar information. For instance, during weak cap, high instability days when storms cover the entire radar area a radar operator can get 'tunnel vision' on a couple storms they initially think are the most important and forget about the rest. She said that a GLM product would be very useful in getting you 'back to reality' when you get overwhelmed and help you identify the cells which are most important at that time. Also, the same forecaster mentioned how they would use this in situations when they are bordering on issuing a severe warning. When the radar information is ambiguous or unchanging, the GLM would help determine whether or not to issue the warning based on any dramatic increases seen in the number of total flashes in the cell.

In one case during the IOP we saw the first occurrence of flashes from the PGLM occurring with a growing cell near the center of the OKLMA network 40 minutes prior to the first CG. Flash rates reaching 25 flashes / 8 km / min occurred 25 minutes prior to the first CG for the same cell. She mentioned that this would be extremely helpful in determining whether the cell was actually a 'thunderstorm' and something to watch out for, stating that a lot of forecasters use the first occurrence of a CG to start thinking about the possibility of that cell ever becoming severe. This would provide a much heightened level of situational awareness, especially when storms can rapidly produce tornadoes, severe hail and/or winds with little warning.

EWP daily plan

Following the training sessions for the GOES-R UWCI, OTTC and PGLM products we had planned to move into an immediate IOP focusing over the Norman, OK WFO to get some exposure for the PGLM product with the chance of severe weather entering the area. Unfortunately as we were finishing the training we had heard that some of the OKLMA stations went down due to the heavy flooding occurring in central OK. Therefore, we decided that the current plan should be to move the domain towards the VORTEX-II domain south of Lubbock, TX following the MRMS training for some severe weather applications for the remainder of the evening.

Missing data... now partially restored

Sometime last week, changes were made to the experiment's NAWIPS systems and the GOES-R Proving Ground Products SATCAST and Probability of Severe Hail were removed by an unknown party. I worked with the SPC IT staff to reincorporate the missing data into the NAWIPS systems... The Probability of Severe Hail is back up, but we are still trying to get SATCAST back up and running. The simulated satellite imagery, lightning threat, UWCI and OTTC products remained unharmed. Will keep an eye on this and work on getting the SATCAST back up and running.

The final week begins...

Today marks the start of the final week of the Spring Experiment (it's gone by so fast). This week we will once again be participating in both programs, which means back to the 14 hour days. Our visitors this week include Jim Gurka and Bonnie Reed from the GOES-R Program Office, Dan Lindsey from CIRA and Jordan Gerth from UW-CIMSS. Jordan will be our resident expert within the EWP because he is directly tied to the UWCI and OTTC products, so he will be providing the training and expertise during real-time IOPs.

This morning begins with around the room introductions and an overview of the project...