Problem with processing l1b Metop data

[zion0210]

Hi colleagues.

I have a problem with processing AMSU/MHS l1b data.
I used MIRS_v2.4 and l1b NOAA data (https://www.avl.class.noaa.gov/).

For example, I run the following command:

Code: Select all

CSPP_MIRS_2_4//scripts/run_mirs.bash -x --noSFR -d /home/terra/NPP_SOFT//MIRS_v2.4/CSPP_MIRS_2_4//data/dynamic_ancillary/ -w /home/terra/NPP_SOFT//WORK/ -s metopB -p 8 -i /home/terra/ATMS_NPP/metop_test/ -vvv

The inputs files are NSS.AMAX.M1.D21166.S0316.E0405.B4535555.MM and NSS.MHSX.M1.D21166.S0316.E0405.B4535555.MM .

And receive the following log:

Code: Select all

DEBUG %options
==============
'dynanc' => '/NFS/snpp2-4/ATMS_GFS/dynamic_ancillary',
'sat' => 'metopB',
'block' => 2,
'input_files' => '/home/terra/ATMS_NPP/metop_test/',
'xfiles' => 1,
'clean' => 1,
'verbose' => 3,
'processors' => 8,
'nothing' => 0,
'SFR' => 0,
'quarantine' => 0,
'logs' => 1,
'yikes' => 0,
'aggregate' => 1,
'resolution' => 'HI',
'work' => '/HDDS/HDD1/public/NPP_SOFT/WORK'
DEBUG %options _END_
---------------
DEBUG %filemask
===============
'metopA' => '*.{AMAX,MHSX}.M2*',
'metopB' => '*.{AMAX,MHSX}.M1*',
'metopC' => '*.{AMAX,MHSX}.M3*',
'n19' => '*.{AMAX,MHSX}.NP*',
'n21' => '{GATMO,SATMS,TATMS}_j02*.h5',
'npp' => '{GATMO,SATMS,TATMS}_npp*.h5',
'n18' => '*.{AMAX,MHSX}.NN*',
'n20' => '{GATMO,SATMS,TATMS}_j01*.h5'
DEBUG %filemask _END_
-----------------
DEBUG %dynanc_loc
=================
'GFSPROTO' => 'https://',
'GFSLOCAL' => '/NFS/snpp2-4/ATMS_GFS/dynamic_ancillary/%Y_%m_%d_%j',
'GFSSITE' => 'jpssdb.ssec.wisc.edu',
'GFSREMOTE' => 'cspp_v_2_0/ancillary/%Y_%m_%d_%j',
'GFSTEMPLATE' => 'gfs.t%Hz.pgrb2.0p50.f0%f.%Y%m%d'
DEBUG %dynanc_loc _END_
chdir to working directory /HDDS/HDD1/public/NPP_SOFT/WORK/TMP_Bzlt_K
PCF file is:  /HDDS/HDD1/public/NPP_SOFT/WORK/TMP_Bzlt_K/cspp_metopB_pcf.bash
SCS file is:  /HDDS/HDD1/public/NPP_SOFT/WORK/TMP_Bzlt_K/cspp_metopB_scs.bash
run_mirs.pl::make_L1B_bigEndian: /home/terra/ATMS_NPP/metop_test/NSS.AMAX.M1.D21166.S0316.E0405.B4535555.MM endian indeterminate
Abort at /home/terra/NPP_SOFT//MIRS_v2.4/CSPP_MIRS_2_4//scripts/run_mirs.pl line 1014.

I also receive identical error for old version MIRS.

Regards,
zion0210

[jimd]

The run_mirs.pl script tries to determine whether the input file is big- or little-endian by decoding the header record both ways and checking if the decoded year makes sense. I am wondering if your file has no header record - which would be a problem, not just for endian-ness but, as the code is presently written, also for a filename extracted from the header record.
If you could please send me the two files (AMAX and MHSX) I can take a look. Best way might be to upload to ftp://ftp.ssec.wisc.edu/pub/incoming/ and then drop me a line.
Was there an option on CLASS to get files with/without header? I would have tested CSPP_MIRS with CLASS data, but perhaps something has changed since I did so.
Cheers,
-jim.

[zion0210]

Hi Jim.

Files NSS.AMAX.M1.D21166.S0316.E0405.B4535555.MM and NSS.MHSX.M1.D21166.S0316.E0405.B4535555.MM located at ftp://ftp.ssec.wisc.edu/pub/incoming/

I used the following form getting l1b metop data
https://www.avl.class.noaa.gov/saa/prod ... submit.y=6

Regards,
zion0210

[kathys]

Hello zion0210,

We have investigated your data and we believe we have a solution.

There is header information that needs to be included in the data that you order from CLASS. To include the header information, you need to follow these instructions:

1). On the CLASS website, along the Left side options, under "User Account", choose "Preferences". This brings you to a "User Preferences" page.

2). Select "Yes" for the Delivery Preference: "Prepend TBM/ARS headers to my data sets"

3). Then scroll to the bottom of the page and select the button "Update".

Once you follow these steps, the data that you order from CLASS will include the header information that you need to execute the MiRS software.

I downloaded the same files that you are working on using this preference, and we were able to process the data using the CSPP MiRS software. I put those new CLASS input files here: https://bin.ssec.wisc.edu/pub/CSPP/test/

Good luck,

Kathy