missing "VIIRS-RSBAUTOCAL-HISTORY-AUX" in Mx8.0?

[wzchen]

Hi,

When I am running the VIIRS SDR, it was failed because of missing one LUT file. I got the following error from the log file:

/data/data020/weizhong/ADL4.1/CSPP/ADL4.2_Mx8.0/log/ProSdrViirsController_20131216_191514_3875.log:2013/12/16 19:17:25.950.846 (3875.47080298071328): DBG_HIGH ProCmnAlgorithm.cpp|7917|Error retrieving data for VIIRS-RSBAUTOCAL-HISTORY-AUX

I downloaded new LUT for Mx8.0 from COMMON CM. However, it seems that this table was missing. Here is the granule ID I tried: NPP000637124094.
Could you please verify it?
Thanks.

[bhenders]

Hi,

The VIIRS-RSBAUTOCAL-HISTORY-AUX item, I've been told is updated once per orbit, it's an auxilliary output item of the RSB Auto Cal algorithm. Since this item is updated every orbit, it actually is not controlled in software configuration management and is the reason why you couldn't find it when you went to Common CM to try to download it. Our script that builds the data TAR files, only packages up data that it can find in source control. When ADL 4.2/Mx8.0 was delivered this item was included as part of the ProSdrViirsController input data. You can get that item at:

https://jpss.ssec.wisc.edu/adl/download ... 2_1.5.8_D/

It should be in this tar file: adlSdrViirsStaticInputDataLinks.tgz

I've been told you can also get this item from CLASS also. I'll talk to Sustainment to see if they can at least stage a seed one on the Common CM site for the VIIRS SDR Cal algorithm in the future.

Thanks,

Bryan Henderson
Raytheon Company

[JimBiard]

Hi.

I've also noticed that there isn't an LW config file for the RSB autocal application. Is that available somewhere?

Grace and peace,

Jim

[kbisanz]

That appears to have been accidentally left out of the Mx8.0 delivery. I can see that it is fixed in the 8.1 delivery. I am not sure if that is available quite yet.

I would recommend copying ProSdrViirsControllerLwFile.xml and updating the copy for use by RSB autocal. The RSB specific lines would be
<idpProcessName>ProSdrViirsRSBAutoCal.exe</idpProcessName>
<configGuideName>ProSdrViirsRSBAutoCal_GuideList.cfg</configGuideName>

You will also want to change <inputPath>, <outputPath>, and <taskDetails1> (the granule ID) to work for your data.

[wzchen]

I downloaded the "VIIRS-RSBAUTOCAL-HISTORY-AUX" files. However, they are all in h5 format. How can I convert them to binary files? Which tool should I use? Thanks.

[stewart999]

Hi,
Since this product is relatively new to the ADL release packages, not all files have been updated accordingly. If you could try the following workaround:
1. In your ADL installation, add "VIIRS-RSBAUTOCAL-HISTORY-AUX" to the <name>UNPACKABLE_AUX</name> section in $ADL_HOME/cfg/DDSADL_CFG.xml as follows:
<config>
<name>UNPACKABLE_AUX</name>
<configValue>
VIIRS-RSBAUTOCAL-HISTORY-AUX
CrIS-Correct-Matrix-AUX
...
2. Run the ADL_Unpacker.exe with the full path of your downloaded h5 file:

$ADL_HOME/tools/bin/ADL_Unpacker.exe <full path of your h5 file>

Please let us know if this does not work, or if you are still experiencing issues.

Thanks,
Sara

[stewart999]

In addition, please note that ADL_Unpacker.exe output is placed into $ADL_HOME/data/HDF5_Unpack_Area/

Thanks,
Sara

[houchin]

HI Sara, Is this update still required for Mx8.4?

Weizhong, if you can reply either way, that would be great. If this works I'll add this to the AeroADL installations as well.

Hi,
1. In your ADL installation, add "VIIRS-RSBAUTOCAL-HISTORY-AUX" to the <name>UNPACKABLE_AUX</name> section in $ADL_HOME/cfg/DDSADL_CFG.xml as follows:
<config>
<name>UNPACKABLE_AUX</name>
<configValue>
VIIRS-RSBAUTOCAL-HISTORY-AUX
CrIS-Correct-Matrix-AUX

[tsimpson]

I might as well take this opportunity to introduce myself. I'm Todd Simpson, a new member of the ADL team, and I've also been working on this issue. To reply to the question: Yes, this update is still required for 8.4.

Also, I would like to point out that I've found an endian conversion might be needed on the unpacked VIIRS-RSBAUTOCAL-HISTORY-AUX data. I've found that the ProSdrViirsRSBAutoCal algorithm seems to freeze when the data is big endian. I haven't yet determined the cause, but in case you see this happening, here's how to transform the endianness of VIIRS-RSBAUTOCAL-HISTORY-AUX:

$ADL_HOME/tools/bin/AdlFileEndianConverter <in> HistoryAux VIIRS <out>

where <in> and <out> are the input and output VIIRS-RSBAUTOCAL-HISTORY-AUX files.

[wzchen]

The unpacker worked before I made this change, however, I just can't ingest it. After I made this change, the unpacker unpacked it and generated metadata associated with too. So, I just included them into Mx8.4's input directory. It worked no problem! I also noticed that the "Effectivity" date is till "2038-01-19 03:14:04.000000". Is that normal?
Thanks,