SSEC forums

I encountered this weird error, which shows no message by selecting "Filter Failed" at Tk Chain Runner Gui window. However, I searched log file for errors and find the following:
"TRACE - (24383.47359111752544): DBG_HIGH ProCmnMethodAudit.cpp|207|ProCmnInputItem[16ProCmnAncIntItem]::queryDMS(NCEP-ANC-Int) [0x2b13b1fb2ec0] UNKNOWN_ERROR queryMetadata: [N_Collection_Short_Name:EQ:NCEP-ANC-Int] AND [DatasetLock:EQ:0] AND [Effectivity:EQ:[2013-01-26 23:29:10.893737 - 2013-01-26 23:29:10.893737]] from file ProCmnInputItem.cpp, line 426
TRACE - (24383.47359111752544): DBG_HIGH ProCmnMethodAudit.cpp|207|ProCmnInputItem[16ProCmnAncIntItem]::acquireURs(NCEP-ANC-Int) [0x2b13b1fb2ec0] UNKNOWN_ERROR queryDMS() call from file ProCmnInputItem.cpp, line 256
TRACE - (24383.47359111752544): DBG_HIGH ProCmnMethodAudit.cpp|207|ProCmnDataItem[16ProCmnAncIntItem]::getData(NCEP-ANC-Int) [0x2b13b1fb2ec0] UNKNOWN_ERROR acquireURs() call from file ProCmnDataItem.cpp, line 956
TRACE - (24383.47359111752544): DBG_HIGH ProCmnMethodAudit.cpp|207|ProCmnInputItem[16ProCmnAncIntItem]::getData(NCEP-ANC-Int) [0x2b13b1fb2ec0] UNKNOWN_ERROR ProCmnDataItem::getData() failed from file ProCmnInputItem.cpp, line 194
TRACE - (24383.47359111752544): DBG_HIGH ProCmnMethodAudit.cpp|207|ProCmnDataItem[16ProCmnAncIntItem]::get(NCEP-ANC-Int) [0x2b13b1fb2ec0] UNKNOWN_ERROR getData() call from file ProCmnDataItem.cpp, line 567
TRACE - (24383.47359111752544): DBG_HIGH ProCmnMethodAudit.cpp|207|ProCmnAlgorithm[ProAncViirsGranulateWindSpeed]::getDataToConvert(NPP000399149517) [0x2b12b1846b40] PRO_FAIL no short name found for input: groupname NCEP_ANC_Int at index 1 from file ProCmnAlgorithm.cpp, line 2187
TRACE - (24383.47359111752544): DBG_HIGH ProCmnAlgorithm.cpp|7845|Error retrieving data for NCEP-ANC-Int".

This seems to be something related to NCEP ancillary data. The one hour data I run span from 23:30UTC from a previous day to 00:30UTC at the next day. I have downloaded all NCEP data for these two days and one additional day before the first one. I unpacked them and run runMsd.pl to convert to internal data format. The directory containing these internal format NCEP data was put in the file where INFTK_DM_ROOT point to, but I did not include temporal interpolation data location (these seem not needed and sometimes cause more problems). Now my issue is this failed granule is NOT complaining anything before SDR controller run (but failed!), and all the granules (less than 10) ahead of it shows "NCEP-ANC-Int" as RED (there are also granules show "NAAPS-ANC-Int" as red which I handled the same way as NCEP), whereas majority of others after it are all good and the runs are successful.

Do I miss anything in correctly handling the NCEP data?

Thank you!

Yue

Yue,

It does seem that you are on an edge condition with your failed granule. For instance, the query time as shown in your debug output is shown to be:

TRACE - (24383.47359111752544): DBG_HIGH ProCmnMethodAudit.cpp|207|ProCmnInputItem[16ProCmnAncIntItem]::queryDMS(NCEP-ANC-Int) [0x2b13b1fb2ec0] UNKNOWN_ERROR queryMetadata: [N_Collection_Short_Name:EQ:NCEP-ANC-Int] AND [DatasetLock:EQ:0] AND [Effectivity:EQ:[2013-01-26 23:29:10.893737 - 2013-01-26 23:29:10.893737]] from file ProCmnInputItem.cpp, line 426

While in the text below you mentioned that you had Ingested NCEP files "The one hour data I run span from 23:30UTC from a previous day to 00:30UTC at the next day. I have downloaded all NCEP data for these two days and one additional day before the first one. I unpacked them and run runMsd.pl to convert to internal data format.", which basically is slightly after the actual query start time.

The TK Chain runner software that paints the inputs green and red uses the granule start time to determine whether the input is available in the repository. It would be interesting to me what the TK Chain runner shows for the effectivity for this granule when it displays it as green. When the algorithm code does the query, it actually uses the ObservedDateTime metadata rather than the RangeDateTime metadata. The RangeDateTime is the granule start/end time, whereas the ObservedDateTime is the start and end time of the first and last scan of the data in the granule. Scan start times are normally greater than or equal to the granule start time. These times are not equal because granule times are not aligned with scan times. A possible idea on how you might have gotten an ObservedDateTime start time before the granule start time, is that maybe the granule in question had missing scans and a start time was estimated for the missing scans that ended up before the start time of the granule.

Nonetheless, if you downloaded the missing NCEP data, I'm sure you would have success in running this granule. It is still somewhat baffling as how it got painted green, why getting the effectivity of the NCEP input shown in the GUI might help with analyzing the issue as this should be the start/end time of the granule id. As it sure seems that the ObservedDateTime for the granule is prior to the granule start time to cause the GUI to display it as green.

I don't think you are doing anything, incorrect but basically hitting some sort of edge condition.

Thanks,

Bryan Henderson
Raytheon Company

I downloaded another whole day (one day before the previous day) NCEP and NAAPS data and ingest them. Those failed granules work.

Thank you very much!

It turns out there may exist some bugs for the Tk Chain Runner Gui. I ran 20 granules in ProEdrViirsCloudsFirstController, which has to run Sdr, VCM, COP first. One granule failed at Sdr due to DMS written failure (something related to parallel process I reported before). Normally I restart Tk Chain runner and only the failed SDR granule needs to be run again and then can proceed. However now every previously successful SdrViirsController granule appears red (note that in previous run they are green). Each of them has to go through Sdr again (Red flag but can run without problem).

Yue,

I wanted to let you know that we did fix the parallel process DMS initialization issue as part of the ADL 4.2 release. ADL 4.2 went out as two patch files, one to upgrade ADL 4.1/Mx7.1 to Mx7.2 and an additional patch file to upgrade to ADL 4.2. That second patch file contains the fix for the parallel processing software issue.

It does seem that there is another TK Chain Runner issue on failure that you've unconvered. I will need to run the scenario by our TK Chain Runner developer and see if he has any ideas on why the other granules also got painted red. My understanding is the TK Chain Runner looks for output items for those granules, which it should have found. Let me get back to you on this issue. Hopefully, with the additional fix, at a very minimum you should not get into this failure scenario.

Thanks,

Bryan Henderson
Raytheon Company