msd ingest issue in ADL5.0

[wzchen]

I can't ingest TLE and Polarwonder files in ADL BLK2. The first problem which I noticed is that the TLE files were not unpacked to ${HDF5_Unpack_Area} like all previous ADL versions. Instead, it kept all converted files in the same input directory. Is anyway I can still make ADL5 to unpack those files to ${HDF5_Unpack_Area}?
Secondly, the ADL didn't run runMsd.pl correctly even though it reported the file has been converted successfully. However, the file was thrown into "$ING_MSD_LZ_PATH/Temp" directory.

Running ING MSD using /data/data020/weizhong/CSPP/ADL5.0_Beta/cfg/dynamic/IngMsdLwFile.xml...
ING MSD completed successfully
All file from Landing Zone converted.
Number of newly converted output files: 0

BTW, the same files have been successfully converted in ADL4.1+Mx8.8
Could you please investigate this issue?
Thanks,

[kbisanz]

We have not seen this issue.

Could you please provide DBG_LOW files for both the Unpacker and IngMsd.exe (which is executed by runMsd.pl)?

Can you also attach at least one file that is causing an issue?

[wzchen]

Please check the log files in the attachments. All TLE and Polarwonder files had the same problems.

[kbisanz]

I believe there are two things going on.

1) I believe the unpacker is working correctly. I believe that for products which do not go through Ingest (such as SDRs, EDRs, etc), those do get unpacked into the data/HDF5_Unpack_Area directory you mentioned. However, data which should go through Ingest to be converted (LUTS, TLE, PolarWander, etc) gets written to the current directory.

2) The issue with Ingest is apparently a bug in ADL 5.0. It should be fixed in ADL 5.1. The fix is very easy to do. Edit $ADL_HOME/cfg/ADL_CFG_MSD_LZ.xml and find "VALID_MSD_SUFFIXES". Add ".msd" to that list. For example:
<configValue>.txt;.bin;.xml;.grib;.grb;.dat;.teb;.grd;.ascii;.h5;.grib2;.hdfeos;.hdf;.grd;.obj;.gr;.img;.msd;</configValue>

There is no need to recompile. A return of runMsd.pl should work.

[wzchen]

Yes, all products which don't need ingest are still unpacked into "data/HDF5_Unpack_Area" directory like it supposes to be.
However, I don't want any data written into my data source directory. Is there anyway I can change the unpack process and still unpack all H5 data into "data/HDF5_Unpack_Area directory" like the previous ADL version?
Thanks,

[kbisanz]

Sorry I missed this post.

To answer the question, there is not an easy way to get MSD data (data which must go through Ingest) to be written to data/HDF5_Unpack_Area. In $ADL_HOME/CMN/Utilities/DDS/ADL_Unpacker/src/DDSASC_Granule.cpp in the method DDSASC_Granule::unwrapData(...) there is code that makes a call to "getcwd" to get the current working directory. That is where the data is written. I looked in the version history of that file and it looks like it's been that way for a long time. I don't have an old ADL install to test it though.

There are a few of options you have if this is a big issue to you.
1) Edit the code and hard code the path to be what you want.
2) Edit the code and use the "getenv" method to get the value of an environment variable that you define (for example: http://www.tutorialspoint.com/c_standar ... getenv.htm)
3) Create a wrapper script around the Unpacker tool which changes to a different directory before calling the Unpacker (which changes the current working directory for the Unpacker)