SSEC forums

I got the following error in ADL 4.2 when I tried to rebuild ADL after updating aerosol AOT LUT and Sunglint LUT:
I was advised from others somehow that "It seems padding bytes may be needed to be added which can happen if we use xmled and edit the file there."

Could anyone give me more hints on how to add padding bytes and which files I should add them to? It did not happen like this when I was running ADL4.0. Any information will be greatly appreciated! Thank you very much in advance!

Yours faithfully,
Jingfeng

Jingfeng,
To run the xml editor in ADL, in a terminal window:
1. Set your DISPLAY environment variable on your machine
(setenv DISPLAY <machine>:13.0 for example)
2. Type:
/vobs/IDPS-CAT3/ADL/script/runEBX.sh
3. Choose a location for your Eclipse workspace (Click OK)
4. Create an Eclipse Project and Import Product Definition XML Files:
Select File->New->Project. In the popup, select JPSS PRO Wizards->EBX Project Wizard.
Enter the name of the project
If you wish to import Product Definition XML files into the new project, check the box next to Import Product Definition XML Files and either type in the location manually or use the button labeled Browse to open a dialog to select a directory.
Note: The Product Definition XML Files are normally located at $ADL_HOME/xml/<Sensor Directories>/*.xml
Click Finish
5. Associate Binary Files with Product Definitions:
In the project explorer, right click on a product definition file, choose JPSS Tools->Associate with binary files menu item, choose the binary files you wish associated. Note: This will be the updated binary LUT files. The pairings will be displayed in the "Product Definition and Binary File Association" view. Double click on the association to open the product definition file and binary file together in one editor.

I believe the Product Definition XML file and the associated binary file will automatically update with the appropriate padded bytes. If this is the case, save the file(s) in Eclipse, copy them into their appropriate locations within ADL (if not modifying the files in place), and recompile ADL. If not, the files can be edited using the instructions listed in the Eclipse Cheat Sheets located within Eclipse:
Help->Cheat Sheets…->JPSS Toolkits->PRO Edit Binary & XML (EBX)->Getting Started (Select the “Editing Product Definitions” and/or “Editing Binary Files” sections)
Once edited, save the files, copy them as needed, and recompile ADL. If you need further assistance, please do not hesitate to respond back to this discussion thread, attaching the updated LUT XML files if possible (otherwise we can use email), and let me know which ADL idps version you are working with:
In a terminal window:
cat $ADL_HOME/.version_idps
Thanks,
Sara

Dear Sara,
My EBX did not work because of no eclipse installed on our server. It could take a while for me to figure that out with our IT. I will deeply appreciate your kind help in finding out what went wrong with my new LUT xml files if it is on your convenience.
The ADL version I am using is AD4.2/mx8.0.
The attached two files are two new LUT XML files that I made changes already. Since it only allows me to attaché 3 files, I sincerely hope you were able to find the original XML files in the mx8.0 or latest package in $ADL_HOME/xml/VIIRS.
Please let me know what went wrong the two files I attached here. Thank you very very much!
Yours sincerely,
Jingfeng

Neither of the structures posted are padded correctly. The compiler is therefore trying to be helpful and inserting additional padding behind the scenes to pad them correctly.

Here is an in depth article describing the issue: http://en.wikipedia.org/wiki/Data_structure_alignment . In general, a structure member should be aligned on an offset that is a multiple of its size. For example, an Int32 (4 bytes) should be on a structure offset that is a multiple of 4. An Int16 (2 bytes) should be on a structure offset that is a multiple of 2. Additionally, the size of a structure should be a multiple of the largest data type contained within it. For example, a structure containing only 2 and 4 byte data types should have an overall size that is a multiple of 4. If an 8 byte data type (e.g. Float64) is added to the structure, the structure size must be a multiple of 8. If it is not a multiple of 8, the compiler will insert padding at the end of the structure. Another user had a padding relating issue a while back: viewtopic.php?f=27&t=351&p=1296&hilit=padding#p1296

If you open the .xml files with a web browser, (e.g. Firefox), they should be displayed using a style sheet. At the bottom is a table which shows the structure field sizes and offsets. For each field, its offset must be a multiple of its size.

Regarding VIIRS_AOT_LUT.xml, for the field "aot", the offset is 328. This is correct because aot is a 4 byte data type and the offset of 328 is a multiple of 4. However, the field "scat_ang_incr" has an offset of 49730116. This is *not* divisible by 8. Therefore the compiler will insert a few bytes before scat_ang_incr to ensure that it is on an offset of 4 bytes. It looks like it would insert 4 bytes of padding before scat_ang_incr.

Regarding VIIRS_AOT_SUNGLINT_LUT.xml, all the fields are on the correct offset. However, the structure size is 32006588. Because the structure contains 8 byte data types, the size must be a multiple of 8. The size of 32006588 is *not* a multiple of 8. Therefore the compiler will insert an extra 4 bytes at the end of the structure to make it a multiple of 8. However, you'll notice that the last field is already a pad field named "pad" (however, the compiler doesn't know this). If that last field named "pad" is removed, the size of the structure becomes 32006584 which is a multiple of 8 and therefore not padding will need to be added.

In summary, I would recommend copying the pad field from VIIRS_AOT_SUNGLINT_LUT.xml and inserting it before scat_ang_incr in VIIRS_AOT_LUT.xml. I would then delete the pad field from VIIRS_AOT_SUNGLINT_LUT.xml. In each XML, you need to update the <NumberOfFields> tag and the <FieldOffset> tag.

Summary of changes in VIIRS_AOT_LUT.xml:
--Add a pad field of size 4 before scat_ang_incr
--Set the field offset of the new pad field correctly
--Set the field offset of scat_ang_incr correctly
--Set NumberOfFields correctly

Summary of changes in VIIRS_AOT_SUNGLINT_LUT.xml
--Remove the existing pad field
--Update NumberOfFields correctly

FYI, EBX will perform all the adding and removing of pad fields for you behind the scenes. It also makes sure that FieldOffset and NumberOfField tags are updated. It would be helpful to get it installed. Technically, it is very easy to install Eclipse and use the EBX plugin. Just download "Eclipse IDE for Java Developers" version 3.7 Indigo from http://www.eclipse.org/downloads/packag ... /indigosr2. Then use the runEBX.sh script. While it's not technically difficult, I understand that sometimes there are IT policies that can make it more challenging.

After you update the XML, you'll need to do a "make clean src library" in $ADL_HOME/xml. Then do a "make clean src library" in $ADL_HOME/CMN/Utilities/Dictionary/Entries/VIIRS/src. Then do a "make clean src library program" in your algorithm's directory. Alternatively, you can rebuild all of ADL.

Dear Sara and Kevin,
Thank you very much for all your detailed and instructive explanations and guidance. I am happy reporting back to you that the fix is working magically!
Thanks!!!
Jingfeng