SSEC forums

Hi colleagues,

We just tried to apply the EDR SPA's recently released by NASA GSFC Direct Readout Lab to the VIIRS SDR outputs that is available at ftp://ftp.ssec.wisc.edu/pub/eosdb/npp/viirs/

We found that the processing time for the 84 seconds swaths is far more than anticipated. On the contrary, the 84 seconds obtained from the NOAA CLASS website can be processed rather quickly and as anticipated.

We checked and compared the format of the SDR h5 files generated by CLASS and CSPP. But found no good clue.

Do you happened to have any idea on the format of the SDR h5 files or is there any standard or documentation on the format?

Thanks!

Wei

The CSPP SDRs stored on our FTP site are all compressed using HDF5 internal gzip compression to save space and bandwidth. This can increase the time needed to read data from the files. I installed and tested the current version of the ActiveFires SPA, and I found it took around 9 minutes to process a particular CSPP SDR granule over the western US.

However when I removed the HDF5 internal compression from the CSPP SDR files, the ActiveFires SPA processing completed in around 10 seconds and gave identical results. It is easy to remove the HDF5 internal compression using the HDF5 utility 'h5repack' available from the HDF group, as shown below.

1. Download the HDF5 binaries for your platform from

http://www.hdfgroup.org/HDF5/release/ob ... tml#obtain

2. Use the h5repack utility as shown below

h5repack --native --filter NONE $input_file $output_file

A simple script to implement this operation on one or more HDF5 files is available in the following script:

ftp://ftp.ssec.wisc.edu/pub/incoming/unh5repack.csh

Just download the script and give it execute permission, and then use it like this:

unh5repack.csh *.h5

I recommend that you always decompress the CSPP SDR files from our FTP site before you use them.

Please let me know if this solves your problem.