ADL_Unpacker causes Segmentation Fault errors

[jhuangadl]

And surprisingly, it worked at certain point, but now it is not working any more. The error message and core dump appeared again. What could be the causes?
Despite of the error message, if we still use the unpacked results, will that be a problem?

[jhuangadl]

Here are the way I tested it.
I went to the folder with h5 files, and run the unpacker to unpack *.h5, then the last file will somehow get a core dumped.
If I run the unpacker with each individual h5 file, it will generate dumped core for each file it unpacks.
So, the problem happens for the last file you run the unpacker with, has nothing to do with it is NCEP or USNO.

I am testing these on a native installation rather than a Virtual Appliance. Will that make any differences? I did applied the SSEC patch though.

Code: Select all

[jhuang@rhw9103 anc_h5]$ ls
off_NAAPS-09HR-ANC_NAAPS_FNMOC_009f_20121016_201210161200Z_20121016151500Z_ee20121017000000Z_np.h5
off_NAAPS-12HR-ANC_NAAPS_FNMOC_012f_20121016_201210161200Z_20121016151501Z_ee20121017030000Z_np.h5
off_NCEP-GFS-09HR-ANC_GFS_NCEP_009f_20121016_201210161200Z_20121016153025Z_ee20121017000000Z_np.h5
off_NCEP-GFS-12HR-ANC_GFS_NCEP_012f_20121016_201210161200Z_20121016153039Z_ee20121017030000Z_np.h5
off_USNO-PolarWander-UT1-ANC_Ser7_USNO_000f_20121012_201210120000Z_20121012000002Z_ee20121019120000Z_np.h5
[jhuang@rhw9103 anc_h5]$ /data/data003/CSPP/ADL/tools/bin/ADL_Unpacker.exe *.h5

Processing off_NAAPS-09HR-ANC_NAAPS_FNMOC_009f_20121016_201210161200Z_20121016151500Z_ee20121017000000Z_np.h5... 

--------------------------------------------------------------------------------------

Processing off_NAAPS-12HR-ANC_NAAPS_FNMOC_012f_20121016_201210161200Z_20121016151501Z_ee20121017030000Z_np.h5... 

--------------------------------------------------------------------------------------

Processing off_NCEP-GFS-09HR-ANC_GFS_NCEP_009f_20121016_201210161200Z_20121016153025Z_ee20121017000000Z_np.h5... 

--------------------------------------------------------------------------------------

Processing off_NCEP-GFS-12HR-ANC_GFS_NCEP_012f_20121016_201210161200Z_20121016153039Z_ee20121017030000Z_np.h5... 

--------------------------------------------------------------------------------------

Processing off_USNO-PolarWander-UT1-ANC_Ser7_USNO_000f_20121012_201210120000Z_20121012000002Z_ee20121019120000Z_np.h5... 

Finished
HDF5 error: 
HDF5-DIAG: Error detected in HDF5 (1.8.7) thread 0:
  #000: H5E.c line 1592 in H5Eget_auto2(): wrong API function, H5Eset_auto1 has been called
    major: Error API
    minor: Can't get value


Aborted (core dumped)
[jhuang@rhw9103 anc_h5]$ /data/data003/CSPP/ADL/tools/bin/ADL_Unpacker.exe off_NCEP-GFS-09HR-ANC_GFS_NCEP_009f_20121016_201210161200Z_20121016153025Z_ee20121017000000Z_np.h5 

Processing off_NCEP-GFS-09HR-ANC_GFS_NCEP_009f_20121016_201210161200Z_20121016153025Z_ee20121017000000Z_np.h5... 

Finished
HDF5 error: 
HDF5-DIAG: Error detected in HDF5 (1.8.7) thread 0:
  #000: H5E.c line 1592 in H5Eget_auto2(): wrong API function, H5Eset_auto1 has been called
    major: Error API
    minor: Can't get value


Aborted (core dumped)
[jhuang@rhw9103 anc_h5]$ 

[mark.tolman]

For the patch listed above, so I need to recompile the hdf5 COTS as well as ADL?

Mark Tolman
MIT Lincoln Laboratory

If you give someone a program, you will frustrate them for a day;
if you teach them how to program, you will frustrate them for a lifetime.
- Anonymous

[scottm]

If you are using the UW version of the COTS and you patch ADL only ADL_Pacjker and Unpaker need to be recopiled.

from ADL_HOME

make library program

[houchin]

If we're installing HDF specifically as part of an ADL installation, is there a version that would avoid the need for this patch, and if so why doesn't the UW support script install that version? Due to other priorities, I'm stuck on ADL 4 Mx6.3 tarball for a while and I haven't had the time to apply the UW patch to that version or to just pull out the part of the patch related to HDF.

I can understand wanting a patch if I have issues (security restrictions?) that would prevent me from installing the officially supported version, but in most cases I don't think this is the case.

[mark.tolman]

The unpacker appears to work for me after applying the patch, but the packer still results in core dumps. An ATMS SDR hdf5 file appears to be created, but since ADL crashed, no further processing is done.

I noticed the patch updated the DDSASC_H5FileProperties.cpp file under ADL_Unpacker/src, but didn't update anything under ADL_Packer/src.

It appears to update error handling for the Unpacker. Is there similar code for the Packer somewhere?

Also, I don't know if it is relevant, but I'm working on updated DQTT values, and purposefully creating DQN products. I don't know if that would impact the hdf5 packer code or not, but I thought it worth mentioning.

-mark

[scottm]

Did you see the SSEC/ADL/CMN/Utilities/DDS/ADL_HDF/src/DDSUTIL_HDFInterface.cpp get updated?

[mark.tolman]

I think so. If I remember correctly, it is the H5Eset_auto2 that is causing the fault.

Here is the code from the actual file:

Code: Select all

   try
   { 
      // Clean up any previous values set
      myHDFfileID = 0;

      // Shut off error printing
  //    H5Eset_auto1(0, 0);  Added for hdf5-1.87 usage
        H5Eset_auto2(H5E_DEFAULT, 0, 0);
  

      herr_t status = 0;

      hid_t fcpl_id = H5Pcreate(H5P_FILE_CREATE);
      hid_t fapl_id = H5Pcreate(H5P_FILE_ACCESS);

[scottm]

So this worked before and now does not work? If this is true check what hdf libraries you are using

LDD ADL_Unpacker.exe
LDD ADL_Packer.exe

This will display what libraries are being used. You can send the output to adlhelp