Let the 2011 Experiment Begin!

Welcome back! It's that time of year again, when we all get together for a 5-week period and talk about science!

We are officially less than 1 week away from the beginning of the experiment so I thought I would get on, knock the dust of this thing and update some of the links. Once again, participants will be encouraged to also post on this blog about their experiences at the experiment and with the products being demonstrated this year.

This year's experiment will be slightly different than previous years with the addition of a morning shift within Experimental Warning Program (EWP). Also, the Experimental Forecast Program (EFP) has added a dedicated convective initiation (CI) desk that will work closely with the EWP morning shift and GOES-R to make forecasts of convective initiation throughout the day. The EWP will also continue its traditional warning shift and the EFP will continue to operate its severe and QPF forecast desks. We are hoping to have more cross participation between the 2 programs this year starting with the EWP morning shift/EFP CI desk collaboration, as well as a new daily 1pm joint briefing.

We will be having 9 individual products being demonstrated this year... some updated versions of previous products and some completely new ones. Below is a list of what we will have this year...

- SATCAST convective initiation nowcast (UAH/SPoRT)
- UWCI / cloud-top cooling rate (UW-CIMSS)
- Overshooting-top / thermal couplet detection and magnitude (UW-CIMSS)
- Psudeo-geostationary lightning mapper (NASA SPoRT / NSSL)
- 0-9 hour Nearcast (University of Wisconsin - CIMSS)
- 0-3 hour severe and significant hail probability (CIRA)
- NSSL-WRF simulated lightning threat (NSSL/USRA)
- NSSL-WRF simulated satellite imagery (UW-CIMSS/CIRA/NSSL)
- NSSL-WRF simulated band differences (CIRA)

The details about these products will begin to emerge as the experiment unfolds and we will be providing real-time posts of product cases and interactions with forecasters during experimental forecast/warning operations.

I look forward to working with all of this year's participants and expect a very fruitful experiment. I would also like to thank everyone who has helped put this together so far, as well as those who have agreed to participate.

End of the experiment

Well, this marks a close to this year's Spring Experiment activities. We will begin again in the late summer this year with the Fire Weather and Heavy rain experiment starting August 24 and ending September 3. I would like to thank all of the participants this year from the various cooperative institutes and partners for coming and assisting in the product training and participating in the experiments alongside the forecasters. Without this participation, the experiment would not have been a success. I would also like to thank all the forecasters who participated this year and took valuable time away from their WFOs to come out and give us their time. The feedback gathered from the forecasters in the EFP and the EWP this year provided invaluable information to help improve the GOES-R products we evaluated this year. While not all feedback gathered this year may have been positive, it is important to remember that understanding the shortcomings in the demonstration strategy and the products themselves is essential to develop a robust product set prior to operational use once GOES-R launches. I would particularly like to thank Andy Dean, Gregg Grosshans, Israel Jirak and Chris Melick from the SPC's Science Support Branch for setting up and providing technical support for the NAWIPS systems on the EFP side. I would also like to thank Ben Baranowski and Darrel Kingfield from WDTB, as well as Kristin Kuhlman, Kevin Manross, Greg Stumpf and Travis Smith from NSSL for their assistance in setting up the AWIPS systems and providing technical support for the EWP side throughout the weeks. Without all of these people there would have been no experiment and I greatly appreciate all of their efforts. We are looking forward to continued interactions with the forecaster and product developer communities throughout the next year and into the coming years.

Thank you!

EWP weekly debrief

Today is the end of this year's Spring Experiment, and what a fitting end. Yesterday was our busiest day in the EWP over ND/MN/IA. As of the time of this entry there were 65 tornadoes reported over the area associated with the event. It was indeed a big day, and luckily there were no technical issues during the IOP. Unfortunately, due to the rapid developing nature of this event the GOES-R products were used only for a short time before it was essential to move into radar operations for warning purposes. The forecasters were inundated with issuing tornado warning pretty much in a consistent line from Canada down into IA. We were able to get some feedback and since this is the weekly debrief I was able to ask the forecasters in more detail, based on my observations this week as well as their survey responses. Here is a breakdown of what we discussed...

UWCI
I asked about the lead times from the surveys... 15-30 mins the general consensus?
"Hard to be sure because we were arriving at the HWT after convection was developing."
"Getting in there around noon would get more accurate results."
"The case event was already initiated when I got there."

"Struggling giving up screen space during severe weather" (from survey)... is there a display you think would help?
"Having an additional head would help. We were only provided with two screens so it changed our strategy."
"Would be useful for the mesoanalyst position at the WFO."
"4-panel devoted to those products works well."

I asked the forecasters if they were ok with more signals if that meant giving up some FAR...
"I would like earlier signals, like a probability signal... which part of my CWA is going to have the best chance... Some kind of signal before there's aggitated clouds would be helpful."

I asked the forecasters whether they preferred the CI or the cloud-top cooling...
"I used them both... kind of liked them both simultaneously."
"It was hard because it seemed it was 50/50 on detecting things so I lost a lot of confidence in using the product, but I did like the cloud-top cooling a little better because it seemed to do better."

I asked the forecasters if they preferred the accumulated of instantaneous fields...
"I stuck with the instantaneous for the most part, I don't think I even looked at the accumulated to be honest."
"I tended to look at the instantaneous."

Would you see a benefit in having a cloud-top cooling track?
"Yes... an overshooting top track would be very useful as well."

OTTC
Detections or magnitudes more useful?
"I looked at the detections only."

Did you see any correlations between OT detections and features developing on radar?
"Reflectivity was increasing above certain (height) levels at that time."

How often did the product correctly detect OTs that you could see on VIS/IR?
"Kinda 50/50."

"For a warning operator, I don't see this product adding much value to an on-going event..." (from survey) Why is that? Lack of detection, timeliness, applicability of product?
"Radar was more important... mix between timeliness and applicability."
"Get a good a western example case where you don't have radar."
"Maybe knock out some radars in a WES case."

PGLM
"It's an acquired taste, working with lightning data. I think it's a fundamental component in the decision making process trying to determine what's going on with the updraft... it's an interesting way of looking at it."

"With supercells I don't see any added benefit, unless it flared up right before an RFD developed... but I would have to see a lot of cases before I could determine that."

"Would be of a lot of value in low top events."

"Would be nice to see it in a non-supercell case event, like winter events."

"In terms of just lightning forecasting... I think the GLM stuff you were showing would be very useful in that area."

OVERALL / TRAINING
Overall feelings following the week...
"I'm 50/50... I'm not leaving here warm and fuzzy if that's what you're asking."
"When I came in I was very excited to have the convective initiation... I don't come out feeling as confident as I did and as excited as I did when I came in."
"The word convective initiation tool sounds awesome, maybe I went into it with high expectations as maybe seeing a tool with some probabilistic information."

How confident were you with the products following the training?
"The products were easy to understand so I was confident in using them."

What changes/additions would you like to see with the training?
Cases in the west re-iterated.
"The training itself was fine... it just seemed that it didn't perform as well as we would like to see."

I asked if the forecasters would like some sort of alert for the GOES-R products (UWCI, OTTC), so they would know when a detection was occurring, or if that would be more annoying than helpful...
"An initial alert would be nice... a continuous one would be overload."
"That would vary forecaster to forecaster."

Re-iterated the need for wind-based warning products.

"When I got really busy, I went back to the things I was used to using. I was just falling back on my normal routine."

"Would be nice on the first day to have a sort of hand-holding process to get used to the products."

"Maybe already have a base set of procedures so we don't have to spend 30 minutes setting everything up... especially on the first day."

Need cases where radar was out or nocturnal events over poor radar coverage.

Review of yesterday's event... an EWP perspective

24-hour UWCI nowcasts (top left), overshooting top (top right) and thermal couplet detections (bottom left), and SPC storm reports (bottom right) for 17 June 2010.
Today we discussed yesterday's event over ND/MN/IA and I wanted to mention a few things we discussed with the forecasters regarding the performance of the UWCI and OTTC products leading up to and during the event. We showed the forecasters the 24-hour accumulations of UWCI, cloud-top cooling, overshooting top and thermal couplet detections with regards to severe reports from the SPC (see images above), as well as their own experiences. Forecasters did notice that cloud-top cooling was occurring on southern edge, but it was already ongoing on radar. The first UWCI signals were seen at 1855 UTC with the convection that was developing over ND and MN. The first overshooting top was detected at 1915 UTC over northern MN. These occurred just prior to beginning operations in the HWT, so it is hard to determine the CI lead times for this event given that we did not see the first detections of CI. I reiterated that we are hoping to expand the experiment in the future years by having overlapping shifts that would provide additional time to evaluate GOES-R Proving Ground products in a nowcasting perspective, which is what they are meant for originally. Forecasters did not see any thermal couplets in real-time, but did notice a few overshooting tops, especially when they began. However, they mentioned that the storms seem more low topped in nature and this may have hindered the products ability to detect these features. They were honest with me in saying that as soon as they moved into warning operations they turned the GOES-R stuff off because they did not feel it provided them with additional warning information and needed the window space for their traditional radar products.

Jordan Gerth did show the forecasters an AVHRR image that demonstrated similar to what GOES-R would see regarding overshooting tops and thermal couplets and noted that the detection efficiency would be much better. The forecasters were impressed with this and agreed that the products would perform much better in the future and appreciated us showing them the products now. The forecasters also did note that in the case of a line forming upstream from main convective area, convective initiation being detected in the line behind would help prevent getting tunnel vision and increase their situational awareness.

SATCAST improvements identified

SATCAST CI nowcast overlaid on visible satellite imagery for 1945 (top left), 2002 (top right), 2015 (mid left), 2033 (mid right), and 2045 UTC (bottom center) on 17 June 2010.
Today we were able to remedy the issues we were having with the SATCAST product within our NAWIPS system. While examining today's potential severe weather outbreak over ND/MN/IA I noticed a few issues that have been relayed to the developers for future improvements. The product did not detect development over southwestern MN where some obvious convective initiation occurred between 1945 and 2045 UTC (see images above). We examined the cloud object detections provided within the output (see images below) and noticed that this area was not widely flagged as having any cloud objects within it. I discussed this with the developers and it was determined that because the clouds were developing so rapidly, the 15-minute cloud typing product used by SATCAST produced cloud types that changed so rapidly that the SATCAST algorithm's quality control threw them out. It should be noted that GOES-13 was operating in RSO during the course of the day, but due to limited computing resources it is not possible to run the product outside of the 15 minute CONUS scans at the moment. It is theorized that this issue would not exist given that the product was able to utilize RSO scans. This does help highlight a major benefit of utilizing higher temporal resolution data that will be available continuously when GOES-R is operational.



SATCAST CI nowcasts (red) and detected objects (blue) for 1945 (top left), 2002 (top right), 2015 (bottom left) and 2045 UTC (bottom right) on 17 June 2010. Note that the 2033 UTC image was not generated and the 2045 UTC image has no objects or CI nowcasts.
It also seemed as though the product was not producing output for the 2032 and 2045 UTC time periods (see images above). I also mentioned this to the developers and they discovered that the algorithm could not handle the scanning schedule changes that occurred with GOES-13 RSO. The algorithm has been changed to accommodate the scan schedule and the output has resumed.

EWP daily briefing... 6/17/2010

24-hour severe reports from 16 June 2010.
At the beginning of the briefing we discussed yesterday real-time IOP event over the Sterling, VA WFO CWA. We had the ability to use the PGLM output from the DCLMA for this event. Most of the warnings issued were for severe wind threats and were fairly marginal. One of the forecasters mentioned that he focused on the PGLM products because of the marginal nature of the event to try and pick out regions of greater importance since the radar and MRMS products were showing similar features for most of the convection around the area. However, due to the marginal nature of the event, no strong conclusions were drawn as to the usefulness of a lot of the experimental data. One of the forecasters brought up the idea of creating regional versions of some of the products to try and draw more information from them. This may eventually be useful for products like the CI and cloud-top cooling products since we have seen some regional dependence on the product performance (ie - diagnostic over SE US)

24-hour UWCI nowcasts for 16 June 2010.
We also discussed the performance of the UWCI over the same area. The forecasters reiterated the lack of detection over WV/VA border from yesterday's blog post. We believe that this was due to some microphysical issues being presented in the cloud typing product that were masking out the CI signals. We mentioned that we had wished we had moved to SD sooner, which is where we ended up for the remainder of yesterday's IOP. There were a lot of signals over that area (see image above), but we were unsure as to the timing of them since we were not watching in real time. This would be a good case to look at in the future. Also, notice the lack of signals over the Sterling and State College WFO areas where we focused at the beginning of the day. Jordan Gerth discussed with the forecasters about the idea of providing a cloud-top cooling rate 'image' with no filtering, which would allow for the forecasters to see all areas of cooling/warming regardless of cloud motion or other effects that are currently removed. This would allow the forecasters decide for themselves what are false signals and may be particularly useful for determining the life cycle of MCS's and weakening supercell storms.


24-hour overshooting top (left) and thermal couplet (right) detections for 16 June 2010.
We examined the overshooting top and thermal couplet detections (see images above) and saw that both the overshooting top and thermal couplet detections did correlate well to the location of the severe weather in this case, especially over SD. However, when these were seen during the real-time IOP, it was already obvious that severe weather was occurring based on reports being received and the standard radar data. The overshooting top detections that continue further east, as well as the blob of CI nowcasts east of the tornadic storm were questioned by the forecasters since they occurred after the IOP ended and they were not sure how legitimate they were. We did look back very quickly to see what happened and it seemed as though a MCS developed later on associated with many additional warnings and it would have been nice to see the performance of the products in that instance. Again, this would be a good case to look at in the future. We will attempt to archive the data, but it may be difficult since we do not yet have dedicated disk space to do so.

The plan for today is to operate over MN and IA. This is our last chance for real-time IOPs and it looks like it should be a good one for GOES-R with the lack of cirrus in the area... assuming we get started soon enough.

Hail Probability Product from 16 June

The image above shows the severe hail probability product from 16 June, valid from 21-00Z. Also plotted are the severe weather reports between 18-00Z; hail is denoted by 'a'. Only a couple of the reports in Montana occurred prior to 21Z, the rest being observed within the hail product's 3-hour time window. For reference, the 2115 UTC GOES-East IR image is below. The product did a nice job of highlighting the regions in which hail was observed (and other severe weather in the case of eastern Kansas). In particular, note the small region near Sydney, Nebraska. There, towering cumulus had just begun to form and a bullseye of probabilities can be seen. Hail was reported there around 2200 UTC.

Use of Simulated WRF Imagery in the EFP

As part of the morning EFP session, we took a look back at yesterday's severe weather forecast. The domain was centered over State College, PA. Many of the model guidance products pointed toward Pennsylvania as a region for possible severe weather. However, as can be seen in the visible GOES-East image above (right), low clouds were present across the eastern part of the state, and the 15Z analyzed SBCAPE showed positive values only where solar insolation had occurred. The image on the left shows the simulated 10.35 micron band (from the NSSL WRF-ARW) also at 15Z, along with the model's SBCAPE field. Note that the model correctly places the low clouds in eastern Pennsylvania, but has CAPE values a bit too high in western PA.

The images above show the same fields, except valid at 22Z. Some convection had begun to fire in the unstable region in central PA, and the model also showed convection in this area. In this case, the WRF seemed to perform quite well with the evolution of afternoon convection. However, its overprediction of the instability lead to the model storms being a bit too strong. No severe weather reports came out of Pennsylvania.

EFP simulated satellite evaluation

EFP QPF group discussing simulated WV imagery from the 00Z run of the NSSL-WRF on 17 June 2010
Today in the EFP's QPF group Dan Lindsey led a discussion on the comparison of the simulated satellite imagery from the NSSL-WRF to that which was observed over the past few hours (see image above). They used this simulated satellite imagery to determine the accuracy of the current 00Z NSSL-WRF model run for the day. It was determined that is is missing some convective development down near NE and IA, but that the main threat area up near MN is being captured fairly well. This provides them with confidence that the model should handle the rest of the day's development fairly well.

EWP real-time IOP CI feedback

Today's IOP focused over the DCLMA domain to get some real-time PGLM experience. The forecasters broke up into two groups and localized over the Sterling, VA and State College, PA WFOs. The Sterling WFO group have been issuing severe thunderstorm warnings based on PGLM and MRMS products over the past couple hours. I was sitting with the State College WFO group as they were focusing on the UWCI products, watching for convective initiation over their area. One of the forecasters had multiple GOES-R Proving Ground products (UWCI, cloud-top cooling, overshooting top, and thermal couplet) into a 4-panel display and mentioned to me that this display strategy would be extremely useful as a situational awareness tool for monitoring satellite convective information.



GOES-13 IR imagery for 1832 (top left), 1845 (top right), 1902 (bottom left) and 1915 UTC (bottom right) on 16 June 2010. Area of interest noted by green circle.
We noticed an area of obvious convective development that was not detected by the UWCI product on the WV/VA border at 1915 UTC and the forecaster asked me to explain why no CI nowcast was made. Looking at the IR, you could see significant cooling occurring from 1832 and 1915 UTC (see above), but still no signals were seen. Jordan Gerth and I looked at the cloud typing output to see if we could draw any conclusions from that to determine what was going on (see below). Most of the area was covered by cloud types identified as 'water' or 'mixed phase'. However, there were a few spots of 'cirrus' classification over WV nearby where the CI nowcast should have been made. It seemed on visible and IR satellite that this may not have been the case, but this is hard to determine for sure. Jordan suggested that because there were these spots nearby that they may have been contaminating the spatial tests required by the algorithm to flag an area as filtered cloud-top cooling, and thus not allow for a CI nowcast to be made. This may be a fluke case where nothing can be done, but it may be useful to examine this in more detail to see if something can be improved.



UW-CIMSS Cloud typing product for 1832 (top left), 1845 (top right), 1902 (bottom left), and 1915 UTC (bottom right) on 16 June 2010. Area of interest noted by red circle.