idlcr8ascii.pro 148.08 KiB
;Main Program Version: idlcr8ascii.pro v4.0b27, 20240927
; Written by Ian Boyd for the EVDC/AVDC - iboyd@bryanscientific.org
;
;Sub-versions:
; v3.01, 20081020 - If the HDF4 file is created using the HDF4.2r3 library, then extra information
; regarding the dimensions of the VAR_DEPEND values may be included in the file
; (which show up as extra datasets in the HDF_SD_FILEINFO call). A check for
; this is carried out and, if found, the information is excluded from the output.
; v3.02, 20090311 - Improve HDF4.2r3 library checks by comparing the number of datasets recorded
; under DATA_VARIABLES, with the number returned by the HDF_SD_FILEINFO call;
; Change 'WARNING' to 'INFORMATION; Add procedure to handle 'INFORMATION' text
; output; Add 'INFORMATION' text if anything irregular found with the datasets;
; Change 'Data dimensions exceed 8' and 'Data type not allowable' ERROR messages
; to 'INFORMATION' messages. Make log file output to the same directory as the
; first HDF file to be read.
; v3.03, 20091208 - Incorporate HDF_SD_ISCOORDVAR to differentiate between datasets and dimension
; variable names. Add check for allowable VAR_DATA_TYPE=STRING. Add all
; INFORMATION text messages to INFOTXT_OUTPUT_A procedure to avoid duplication
; of code that creates the messages; Add INFORMATION messages when reading HDF5
; files; Rename AVDC Button to FORMAT and replace keyword option /AVDC with
; /FORMAT. Improve checks on parameter and keyword inputs; Improve checks for
; non-standard (i.e. non-groundbased) HDF files.
; v4.0b1, 20101122 - Adopt GEOMS metadata standard; Because FORMAT attribute(s) have been dropped
; can no longer use defined formatting values when outputing data to files;
; Default ASCII formatting adopted - scientific notation for REAL and DOUBLE,
; except for DATETIME and related values, which have formatting dependent on
; the data type; Numeric metadata values remain in their original data type
; when writing to session memory, meaning: variable attribute labels and values
; now written to separate structure variables sds.va_l and sds.va_v (previously
; sds.va), and structure has 2 dimensions (n_sds, n_atts) instead of 1
; (previously n_sds); The dataset is written to the initial n_atts index (i.e.
; sds[n_sds,0].data); Rename saved SDS datatype according to GEOMS rules
; (affects INT, LONG, and FLOAT); Add JDF_2_DATETIME and JULIAN_DATE routines
; to the program; Include coordinate variables if they and the non-coordinate
; variables add up to the number of saved DATA_VARIABLES. Add return error code
; option so that program returns to the calling program if an error is
; generated.
; v4.0b2, 20110323 - If required reverse the order of VAR_SIZE and VAR_DEPEND values so that they
; match the order of the multi-dimensional dataset. Note that the HDF read programs
; automatically transpose the multi-dimensional datasets to conform to the IDL
; convention of fastest changing index being listed first (opposite for HDF).
; v4.0b3, 20111014 - Add netCDF support - writes variable attributes to GEOMS standard attributes
; where possible, otherwise writes to VAR_NOTES
; v4.0b4, 20111208 - Test for dimension ordering of the netCDF files, and reverse if required (same
; rules as for HDF). Create separate Dimension Ordering Routine
; v4.0b5, 20120328 - Add options to convert HDF/netCDF files to other HDF/netCDF formats e.g. H4 to
; H5 etc. Requires IDLcr8HDF in the search path to do this.
; v4.0b6, 20120426 - Bug fix when a netCDF file only contains datasets with single dimensions -
; VAR_SIZE information was not being extracted during the read process.
; v4.0b7 20130114 - Add check for IDL being run in DEMO mode by using LMGR command. Replace PRINTF,-1
; statements (i.e. when dux[0] eq -1) with PRINT (o/w causes DEMO mode to stop). Also
; disable /LOG, /FORMAT and /DUMP keywords in DEMO mode.
; v4.0b8 20140325 - Fix bug that meant VAR_SIZE and VAR_DEPEND values were not being automatically
; reversed if the array sizes were the same and VAR_DEPEND did not include a DATETIME
; value. Now defaults to the 'reverse' option in the TEST_DIM_ORDER routine if no
; other conditions are satisfied; Fix bug that caused crash when VAR_SIZE values were
; not of type string.
; v4.0b9 20141110 - Fix bug that caused multi-dimensional array ordering to not be correctly identified
; if the first dataset to be checked in the file had the same number of elements in
; the array (e.g. was a set of Averaging Kernels).
; v4.0b10 20150127 - Allow for VAR_FILL_VALUE values for string datasets to be written and saved as an
; (empty) string of the correct length. Ensure string datasets are all written and
; saved to the correct length (that of the longest string in the dataset).
; v4.0b12 20151109 - Convert DATETIME (and related) max, min and fill values to LONG64 data type before
; determining the format for ASCII output to account for very large values.
; v4.0b15 20160725 - Fix bug causing program to crash when the number of DATA_VARIABLES values is greater
; than the number of datasets in the file.
; v4.0b19 20180218 - Fix bug that caused multi-dimensional array ordering to not be correctly identified
; if the datasets have the same VAR_SIZE i.e. rev_vd_vs stays as 2 through all the
; checks. Now rev_vd_vs changes to 1 (VAR_SIZE, VAR_DEPEND and dataset ordering is ; reversed) if rev_vd_vs=2 after all the checks.
; v4.0b20 20181116 - Fix bug that caused multi-dimensional array ordering to not be correctly written if
; not consistent through the file i.e. rev_vd_vs eq 3. Now rev_vd_vs changes to 1
; (VAR_SIZE, VAR_DEPEND and dataset ordering is reversed), but information message
; advising of the issue is still reported.
; v4.0b21 20190514 - Fix bug that occurred when GEOMS variable attribute information is missing from an
; HDF4 file but written to the heap structure based on the contents of the file. Previously
; the variable attribute label was not written correctly.
; v4.0b22 20190806 - Provide support for reading netCDF4 files. Currently uses the HDF5 routines to do this
; so need to account for features unique to netCDF4 such as; the inclusion of dimension
; names; standard netCDF attribute names; empty attribute values that do not include a
; null termination character (which causes an error in the H5A_Read routine). New
; sub-routines include is_a_number_ascii, alpha_numeric_underscore, nc_dimension_chk,
; file_format_a.
; v4.0b23 20190821 - Fixed bugs associated with H5 TAG_NAMES checks introduced in v4.0b22.
; v4.0b24 20200830 - Add INFORMATION messages (20-22) that identify issues with the VAR_DEPEND and VAR_SIZE
; attributes, that were previously fixed 'quietly' by the program; Check input GEOMS file
; for zero filesize before doing checks.
; v4.0b25 20220805 - For NetCDF3 or 4, if SDS_NAME NE VAR_NAME then converts non-alphanumeric VAR_NAME
; characters to '_' and compares with SDS_NAME. If there is a match then program quietly
; converts SDS_NAME to VAR_NAME for testing, otherwise generates an error; If input file has
; an HDF5 file signature then checks for .nc .nc4 .netcdf and .netcdf4 extension and also
; calls NCDF_PARSE (if ge 8.5.2) to check for NC4 compatibility; Create DATA_TYPE_CHECKS
; procedure to do all the data type checks. Unable to check for unsigned 8-bit in netCDF3
; files at this time; Generate an INFORMATION/ERROR message if VAR_UNITS is numeric
; (same as VAR_SIZE); INFORMATION/ERROR message added if a mis-match is found between
; the input file name and the FILE_NAME attribute value.
; v4.0b26 20240912 - For NetCDF3 files only output WARNING to check for signed BYTE entries when the dataset
; is of type BYTE (infotxt_output_a message 23).
; v4.0b27 20240927 - For netCDF4 files allow for additional standard netCDF attribute names.
PRO idlcr8ascii_common
;Procedure to define the data COMMON block WIDGET_WIN_A, containing common variables
;associated with the Graphical User Interface
; ----------
; Written by Ian Boyd for the EVDC/AVDC - iboyd@bryanscientific.org
;
; History:
; 20061004: Introduced to IDLCR8ASCII - Version 2.0
; 20080302: 'cwidg' variable removed, and 'dux' array added to identify requested
; output options - Version 3.0
; 20101122: Add rerr - Version 4.0b1
;
;
; Input: Nil
;
; Output: Nil
;
; Called by: N/A
;
; Subroutines Called: None
COMMON WIDGET_WIN_A,wtxt,lineno,base,o3,b3,dux,rerr
END ; Procedure idlcr8ascii_common
PRO intro_a_event, ev
;Procedure to define how Events from the Start-up Introduction Window are handled
; ----------
;Written by Ian Boyd for the EVDC/AVDC - iboyd@bryanscientific.org
;
; History:
; 20061004: Introduced to IDLCR8ASCII - Version 2.0
; 20080302: Code added to define the Pop-up Window and Log Output events added to
; the Introduction Window - Version 3.0
;
; Input: ev - Selected widget event structure
;; Output: o3 - Common string array defining the various options for program output
;
; Called by: XMANAGER in INTRO_A
;
; Subroutines Called: None
COMMON WIDGET_WIN_A
;The uservalue is retrieved from a widget when an event occurs
WIDGET_CONTROL,ev.id,GET_UVALUE=uv
IF (uv EQ 'C') OR (uv EQ 'P') OR (uv EQ 'idlcr8ascii.log') OR $
(uv EQ 'H4') OR (uv EQ 'H5') OR (uv EQ 'N3') THEN BEGIN
CASE 1 OF
uv EQ 'C': BEGIN
uvi=1 & eadd=12L
END
uv EQ 'P': uvi=3
uv EQ 'H4': BEGIN
uvi=5 & eadd=6L
END
uv EQ 'H5': BEGIN
uvi=6 & eadd=5L
END
uv EQ 'N3': BEGIN
uvi=7 & eadd=4L
END
ELSE: uvi=4
ENDCASE
IF o3[uvi] EQ uv THEN o3[uvi]='0' ELSE o3[uvi]=uv
IF (uv EQ 'C') OR (uv EQ 'H4') OR (uv EQ 'H5') OR (uv EQ 'N3') THEN BEGIN
IF (o3[1] EQ '0') AND (o3[5] EQ '0') AND (o3[6] EQ '0') AND (o3[7] EQ '0') THEN $
WIDGET_CONTROL,ev.id+eadd,Sensitive=0 $
ELSE BEGIN
WIDGET_CONTROL,ev.id+eadd,Sensitive=1
IF uv EQ 'C' THEN BEGIN
WIDGET_CONTROL,ev.id+1,/Set_Button
o3[3]='P'
ENDIF
ENDELSE
ENDIF
ENDIF ELSE IF (uv EQ 'F') OR (uv EQ 'D') THEN o3[0]=uv $ ;Assign button event to a variable name
ELSE IF uv EQ 'Cont' THEN o3[0]='0' ;changes o3[0] from -1 to 0
IF (uv NE 'C') AND (uv NE 'P') AND (uv NE 'idlcr8ascii.log') AND (uv NE 'H4') AND $
(uv NE 'H5') AND (uv NE 'N3') THEN WIDGET_CONTROL,ev.top,/DESTROY
END ;Intro_Event handler
PRO intro_a, intype
;Procedure which creates an Introduction Window at start-up when IDLCR8ASCII is called without parameters,
;or invalid parameters, or is called by IDL Virtual Machine. The user has a choice of continuing with
;the program after selecting input options, or stopping the program.
; ----------
;Written by Ian Boyd for the EVDC/AVDC - iboyd@bryanscientific.org
;
; History:
; 20061004: Introduced to IDLCR8ASCII - Version 2.0
; 20080302: Swapped the order of the option windows, so that the options indicated by the check
; boxes appear above the option boxes that close the Introduction Window and continue
; with the program; Added options to allow the user to send input/output log information
; to a Pop-Up Window and/or a Log File - Version 3.0
; 20091208: Change AVDC button to FORMAT and keyword option from /AVDC to /FORMAT - Version 3.03
; 20101122: Add vertxt array. Modify Introduction text to indicate that program accepts 'GEOMS
; Compliant' files, and also to show the new structure format - Version 4.0b1
;
; Input: intype - Integer set to -2 or -3: -2 indicates normal state; -3 indicates that input
; parameters or keywords at the IDLCR8ASCII call are invalid.
;
; Output: Nil;
; Called by: IDLCR8ASCII
;
; Subroutines Called: INTRO_A_EVENT (via XMANAGER)
COMMON WIDGET_WIN_A
;procedure which provides an introduction message before starting the program.
nhdr=46 & errtxt=STRARR(nhdr)
vertxt=['idlcr8ascii-v4.0_Readme.pdf','v4.0b27 September 2024']
errtxt[1]='Welcome to IDLcr8ASCII. This program reads GEOMS compliant HDF4, HDF5 and netCDF files and'
errtxt[2]='saves contents to either session memory, an output window (summary only) or to ASCII or formatted'
errtxt[3]='files (also refer to '+vertxt[0]+').'
errtxt[5]='Inputs to the program:'
errtxt[6]=' HDF/netCDF FILE(s) - GEOMS compliant HDF4, HDF5 or netCDF files. Input can be by DIALOG_BOX'
errtxt[7]=' (IDL VM or full version of IDL), or by command line (full version of IDL only).'
errtxt[9]='Choice of output is made by selecting options in this DIALOG_BOX (IDL VM or full version'
errtxt[10]=' of IDL), or by ''Keywords'' at the command line input (full version only), as follows:'
errtxt[11]=' /F or /FORMAT - generates two output files (with .META and .DATA extensions). The resulting'
errtxt[12]=' files can be used as inputs to GEOMS compliant write programs (IDLcr8HDF etc), OR'
errtxt[13]=' /D or /DUMP - generates two output files (with .META and .DATA extensions). Data values'
errtxt[14]=' will be shown as indicated in the array format defined by VAR_SIZE, AND/OR'
errtxt[15]=' /C or /CATALOG - sends variable index, variable name, data format, and dimension size'
errtxt[16]=' information to an output window and/or log file, AND/OR'
errtxt[17]=' /P or /POPUP - sends log input/output information to a Pop-up Dialog Box, AND/OR'
errtxt[18]=' /L or /LOG - sends log input/output information to a file named idlcr8ascii.log'
errtxt[19]=' /H4 - outputs contents of the input file as an HDF4 file'
errtxt[20]=' /H5 - outputs contents of the input file as an HDF5 file'
errtxt[21]=' /NC3 - outputs contents of the input file as a netCDF3 file'
errtxt[23]='Example of command line input: idlcr8ascii,DFSpec[,/F][,/D][,/C][,/P][,/L][,/H4][,/H5][,/NC3]'
errtxt[24]=' where ''DFSpec'' can either be a string array containing filenames or a scalar string containing a'
errtxt[25]=' filespec or a single file name.'
errtxt[26]='If running the full version of IDL, and input is a single HDF file, output can also be saved'
errtxt[27]=' to session memory. This can be done by calling idlcr8ascii with the following command line:'
errtxt[28]=' idlcr8ascii,DFFile,GA,SDS[,/F][,/D][,/C][,/P][,/L][,/H4][,/H5][,/NC3].'
errtxt[29]='DFFile can either be the name of an input file or '''' (in which case a DIALOG_BOX will open'
errtxt[30]=' and prompt for the input filename). ''GA'' is a returned string array containing the Global'
errtxt[31]=' Attributes, and ''SDS'' is a returned structure using pointers containing the Variable'
errtxt[32]=' Attribute Labels (sds.va_l), Variable Attribute Values (sds.va_v), and the Data (sds.data).
errtxt[34]='Contacts -'
errtxt[35]=' Ian Boyd, Bryan Scientific Consulting LLC (iboyd@bryanscientific.org)'
errtxt[36]=' 9 Cambridge Terrace'
errtxt[37]=' Masterton 5810, New Zealand'
errtxt[39]=' Ann Mari Fjaeraa, EVDC Project Manager (amf@nilu.no)'
errtxt[40]=' Norwegian Institute for Air Research, Instituttveien 18'
errtxt[41]=' Postbox 100, N-2027 KJELLER, NORWAY'
errtxt[43]='EVDC Website: Tools and documentation available from http://evdc.esa.int/'
errtxt[45]='To continue, please choose from the options below.'
errtxt=' '+errtxt
;Set-up text display widget
demomode=LMGR(/DEMO) & optsens=1
IF intype EQ -3 THEN xtxt=' - Command Line Input Error' ELSE xtxt=''
base=WIDGET_BASE(Title='idlcr8ascii '+vertxt[1]+xtxt,Tlb_Frame_Attr=1,/Column) ;,Tab_Mode=1)
wtxt=WIDGET_TEXT(base,xsize=90,ysize=25,/Scroll)
tip='Left Mouse Click or Tab to entry and hit <Spacebar>'
base2=WIDGET_BASE(base,/Nonexclusive)
cattxt='Catalog - Generate a summary of the contents of the input file(s) and send to an output window or file'
poptext='Open a Pop-Up Window to display log input/output (default if ''Catalog'' option is chosen)'
logtext='Append log input/output to the file ''idlcr8ascii.log'' '
IF demomode THEN BEGIN
logtext=logtext+'(No log or netCDF file output permitted in IDL DEMO Mode)' & optsens=0
ENDIF ELSE logtext=logtext+'(will create the file IF it doesn''t exist)'
h4text='Convert input file(s) to HDF4'
h5text='Convert input file(s) to HDF5'
nctext='Convert input file(s) to netCDF3'
fctext1='For the following 3 options, any existing files will be overwritten, IDLcr8HDF must be in the IDL Search Path,'
fctext2='and a valid Table Attribute Values (TAV) file must be in the same directory as the Input file(s).'
b1=WIDGET_BUTTON(base2,value=cattxt,uvalue='C',frame=3)b2=WIDGET_BUTTON(base2,value=poptext,uvalue='P',frame=3)
b3=WIDGET_BUTTON(base2,value=logtext,uvalue='idlcr8ascii.log',sensitive=optsens,frame=3)
base3=WIDGET_LABEL(base)
WIDGET_CONTROL,base3,set_value=fctext1
base4=WIDGET_LABEL(base)
WIDGET_CONTROL,base4,set_value=fctext2
base5=WIDGET_BASE(base,/Nonexclusive,ROW=1)
b4=WIDGET_BUTTON(base5,value=h4text,uvalue='H4',frame=3)
b5=WIDGET_BUTTON(base5,value=h5text,uvalue='H5',frame=3)
b6=WIDGET_BUTTON(base5,value=nctext,uvalue='N3',sensitive=optsens,frame=3)
base6=WIDGET_BASE(base,/Row)
b7=WIDGET_BUTTON(base6,value='ASCII Formatted',uvalue='F',sensitive=optsens,frame=3) ;,Tooltip=Tip)
b8=WIDGET_BUTTON(base6,value='ASCII Dump',uvalue='D',sensitive=optsens,frame=3) ;,Tooltip=Tip)
b9=WIDGET_BUTTON(base6,value='Continue',uvalue='Cont',frame=3,Sensitive=0) ;,Tooltip=Tip)
b10=WIDGET_BUTTON(base6,value='Stop',uvalue='S',frame=3) ;,ToolTip=Tip)
WIDGET_CONTROL,base,/Realize
WIDGET_CONTROL,b1,/Input_Focus
FOR i=0,N_ELEMENTS(errtxt)-1 do $
WIDGET_CONTROL,wtxt,set_value=errtxt[i],/Append
XMANAGER,'intro_a',base
END ;Intro_A
PRO idlcr8ascii_event, ev
;Procedure to close the pop-up logging window after user selects 'Finish'.
; ----------
;Written by Ian Boyd for the EVDC/AVDC - iboyd@bryanscientific.org
;
; History:
; 20061004: Introduced to IDLCR8ASCII - Version 2.0
;
; Input: Selected widget event structure
;
; Output: Nil
;
; Called by: XMANAGER in IDLCR8ASCII and STOP_WITH_ERROR_A
;
; Subroutines Called: None
WIDGET_CONTROL,ev.top,/DESTROY
RETALL & HEAP_GC
END ;Proc IDLcr8ASCII_Event
PRO stop_with_error_a, txt1, txt2, lu
;Procedure called when an error in the program is detected. An error message is displayed
;and the program stopped and reset. If necessary, open files are closed. The error message
;is displayed in one or more of the following: a Pop-up dialog window; the Pop-up logging
;window; the Output Logging window in the IDLDE.
; ----------
;Written by Ian Boyd for the EVDC/AVDC - iboyd@bryanscientific.org
;
; History:
; 20050729: Original IDLCR8ASCII Routine - Version 1.0
; 20061004: Set-up so that the error output is displayed in the output window dependent on the
; method that IDLCR8ASCII is called. If txt1 is preceeded by 'D_' or is null, the
; error output is to a Pop-up Dialog window. Otherwise output will be to a logging
; window. Renamed from STOP_WITH_ERROR to STOP_WITH_ERROR_A to avoid a name conflict
; with the equivalent procedure in IDLCR8HDF. Common variable definition WIDGET_WIN_A
; added - Version 2.0
; 20080302: Added code which sends the error message to the IDLDE output window and/or an external
; file (as determined by the dux array values) - Version 3.0
; 20101122: Allow routine to return to the calling program, rather than stop the application, if; a 'reterr' argument is included in the call to idlcr8ascii - Version 4.0b1
;
; Inputs: txt1 - the initial text line of the error message. Generally contains the name of the routine
; where the error was detected.
; txt2 - the second text line which generally describes the error state.
; lu - Where applicable, the file unit that needs to be closed at the termination of the program,
; otherwise set to -1.
;
; Output: Nil
;
; Called by: The routine in which the error was detected. The following routines call STOP_WITH_ERROR:
; READ_HDF_SDS; IDLCR8ASCII
;
; Subroutines Called: IDLCR8ASCII_EVENT (via XMANAGER)
COMMON WIDGET_WIN_A
IF lu NE -1L THEN FREE_LUN,lu
IF txt1 EQ '' THEN BEGIN ;<cancel> chosen on Intro box
res=DIALOG_MESSAGE('IDLcr8ASCII Stopped!',/Information,Title='EVDC/AVDC IDLcr8ASCII')
ENDIF ELSE BEGIN
IF STRMID(txt1,0,2) EQ 'D_' THEN txtx=STRMID(txt1,2) ELSE txtx=txt1
FOR i=dux[0],dux[1],dux[2] DO BEGIN
IF i EQ -1 THEN BEGIN
PRINT,' ERROR in '+txtx & PRINT,' '+txt2
PRINT,'' & PRINT,'IDLcr8ASCII stopped - Program Ended on '+SYSTIME(0)
ENDIF ELSE BEGIN
PRINTF,i,' ERROR in '+txtx & PRINTF,i,' '+txt2
PRINTF,i,'' & PRINTF,i,'IDLcr8ASCII stopped - Program Ended on '+SYSTIME(0)
ENDELSE
ENDFOR
IF dux[1] GT -1 THEN FREE_LUN,dux[1]
IF (STRMID(txt1,0,2) EQ 'D_') AND (rerr EQ 'NA') THEN BEGIN
;write error to DIALOG Box instead of Popup window
errtxt2=STRARR(4)
errtxt2[0]=STRMID(txt1,2) & errtxt2[1]=txt2 & errtxt2[3]='IDLcr8ASCII Stopped!'
res=DIALOG_MESSAGE(errtxt2,/Error,Title='EVDC/AVDC IDLcr8ASCII Error')
ENDIF ELSE IF rerr EQ 'NA' THEN BEGIN ;write error to Popup window
lineno=lineno+4L
WIDGET_CONTROL,wtxt,set_value=' ERROR in '+txt1,/Append
WIDGET_CONTROL,wtxt,set_value=' '+txt2,/Append
WIDGET_CONTROL,wtxt,set_value='',/Append,Set_text_top_line=lineno
WIDGET_CONTROL,wtxt,set_value='HDF file read stopped - hit <Finish> to close program',/Append
WIDGET_CONTROL,b3,Sensitive=1,/Input_Focus
XMANAGER,'stop_with_error_a',base,Event_Handler='idlcr8ascii_event'
ENDIF
ENDELSE
HEAP_GC
IF rerr EQ 'NA' THEN RETALL ELSE rerr='Unable to read HDF or NC file - '+txt2
END ;Procedure Stop_With_Error_A
FUNCTION is_a_number_ascii, value
ON_IOERROR, ConversionError
IF STRTRIM(value,2) EQ '' THEN RETURN, 0B
n=DOUBLE(value)
RETURN, 1B
ConversionError:
RETURN, 0B
END
FUNCTION alpha_numeric_underscore, str_value
;Function that returns the input string with non-alphanumeric characters replaced with '_'
; ----------;Written by Ian Boyd for the EVDC - iboyd@bryanscientific.org
;
; History:
; 20190806: Introduced - Version 4.0b22
;
; Input: str_value: string
;
; Returns: string with non-alphanumeric values replaced with '_'
;
; Called by: Read_HDF_SDS
;
ON_IOERROR, ConversionError
numb=BINDGEN(10)+48B ;0-9
ucb=BINDGEN(26)+65B ;A-Z
lcb=BINDGEN(26)+97B ;a-z
str_value=STRING(str_value)
new_str_value=str_value
n_char=STRLEN(str_value)
FOR i=0L,n_char-1L DO BEGIN
byt_value=BYTE(STRMID(new_str_value,i,1))
testn=(byt_value GE numb[0]) AND (byt_value LE numb[9])
testu=(byt_value GE ucb[0]) AND (byt_value LE ucb[25])
testl=(byt_value GE lcb[0]) AND (byt_value LE lcb[25])
IF (~testn) AND (~testu) AND (~testl) THEN $
STRPUT,new_str_value,'_',i
ENDFOR
RETURN, new_str_value
ConversionError:
RETURN, 'io_error'
END ;Function alpha_numeric_underscore
PRO infotxt_output_a, in0, in1
;Procedure called to report information relevant to the reading of the HDF file.
;This information can be reported to a Pop-up logging window, IDLDE output log window, and/or
;the log file. Code for this output was originally written directly in the affected procedures
; ----------
;Written by Ian Boyd for the EVDC/AVDC - iboyd@bryanscientific.org
;
; History:
; 20090311: Introduced to IDLCR8ASCII - Version 3.02
; 20091208: Added all the INFORMATION text messages to the routine, to avoid duplication
; in the HDF4 and HDF5 read sections of READ_HDF_SDS - Version 3.03
; 20160213: Update information message when in0[0] EQ 13 - Version 4.0b13
; 20161130: Added in0[0] messages 18 and 19 to cover non-GEOMS metadata attributes issues
; - Version 4.0b16
; 20220805: Modified message 2 (full message created in DATA_TYPE_CHECKS) and added messages
; 23 and 24 - Version 4.0b25
; 20240912: Changed WARNING message in 23 so that it is only generated when the NC3 dataset
; is of type BYTE - Version 4.0b26
;
; Inputs: in0 - Integer array containing input used to make correct text output
; in1 - String array containing input used to make correct text output
;
; Output: Nil
;
; Called by: The routine in which the request for INFORMATION text was made.
; The following routines call INFOTXT_OUTPUT_A: READ_HDF_SDS; OUTPUT_HDF_DATA;
; IDLCR8ASCII
;
; Subroutines Called: None
COMMON WIDGET_WIN_A
CASE 1 OF
in0[0] EQ 0: BEGIN
infotxt=STRARR(3)
IF LONG(in1[0]) EQ 0L THEN BEGIN infotxt[0]=' INFORMATION: Global Attribute DATA_VARIABLES not found or has no values.'
infotxt[1]=' Dataset listing order matches the order that the dataset is read in the GEOMS file.'
ENDIF ELSE BEGIN
infotxt[0]=' INFORMATION: Number of Global Attribute DATA_VARIABLES values ('+in1[0]+')'
infotxt[1]=' does not match the number of datasets saved to the GEOMS file ('+in1[1]+')'
ENDELSE
infotxt[2]=' Number of datasets determined from '+in1[2]+' call'
END
in0[0] EQ 1: infotxt=' INFORMATION: '+in1[0]+' has '+in1[1]+' data dimensions'
in0[0] EQ 2: infotxt=' INFORMATION: '+in1[0]
in0[0] EQ 3: BEGIN
infotxt=STRARR(4)
IF (STRPOS(STRUPCASE(in1[1]),STRUPCASE(in1[0])) NE -1) AND (in0[1] EQ 4) AND (in0[2] EQ 0) THEN $
infotxt[0]=' INFORMATION: Dataset Name is truncated (File created with HDF4.2r1 library or earlier):' $
ELSE infotxt[0]=' INFORMATION: Dataset Name does not match VAR_NAME value:'
infotxt[1]=' SDS_NAME: '+in1[0]
infotxt[2]=' VAR_NAME: '+in1[1]
IF in0[2] EQ 0 THEN infotxt[3]=' Output uses VAR_NAME from Metadata' $
ELSE infotxt[3]=' Output uses Dataset Name from the HDF file'
END
in0[0] EQ 4: BEGIN
infotxt=STRARR(2)
IF in0[1] EQ 0 THEN BEGIN
infotxt[0]=' INFORMATION: Metadata label VAR_NAME not found for dataset:'
infotxt[1]=' SDS_NAME: '+in1[0]
ENDIF ELSE IF (in0[2] EQ 0) AND (in0[3] NE 0L) THEN BEGIN
infotxt[0]=' INFORMATION: VAR_NAME value does not match any DATA_VARIABLES values:'
infotxt[1]=' VAR_NAME: '+in1[1]
ENDIF
END
in0[0] EQ 5: BEGIN
infotxt=STRARR(2)
infotxt[0]=' INFORMATION: No Variable Attributes returned after call to '+in1[1]
infotxt[1]=' SDS_NAME: '+in1[0]
END
in0[0] EQ 6: BEGIN
infotxt=' INFORMATION: Metadata Dataset listing order may not match the order of'
infotxt=infotxt+' the Global Attribute DATA_VARIABLES Values'
END
in0[0] EQ 7: BEGIN
infotxt=' INFORMATION: Can read one HDF or netCDF file into session memory.'
infotxt=infotxt+' Only first file in the array will be read'
END
in0[0] EQ 8: BEGIN
infotxt=' INFORMATION: '+in1[0]+' is a coordinate variable'
END
in0[0] EQ 9: BEGIN
infotxt=STRARR(3)
infotxt[0]=' INFORMATION: Calibrated data in dataset '+in1[0]
infotxt[1]=' has been converted back to its original form using the formula:'
infotxt[2]=' Orig_Data = Scale_Factor * (Cal_Data - Offset)'
END
in0[0] EQ 10: BEGIN
infotxt=STRARR(2)
infotxt[0]=' INFORMATION: VAR_DATA_TYPE='+in1[1]+' for Calibrated data in dataset '+in1[0]
infotxt[1]=' has been changed to the original datatype of '+in1[2]
END
in0[0] EQ 11: BEGIN
infotxt=STRARR(2)
CASE 1 OF
in0[1] EQ 20: BEGIN
infotxt[0]=' INFORMATION: Order of multi-dimensional datasets changed to match the programming'
infotxt[0]=infotxt[0]+' language convention:
infotxt[1]=' IDL/Fortran - fastest changing dimension first (datasets transposed when'
infotxt[1]=infotxt[1]+' reading/writing to the HDF file)'
END
in0[1] MOD 10 EQ 0: BEGIN
infotxt[0]=' INFORMATION: Listing order of multi-dimensional VAR_DEPEND and VAR_SIZE values'
infotxt[1]=' in the HDF file changed to match corresponding dataset dimension ordering'
END ELSE: BEGIN
infotxt[0]=' INFORMATION: Unable to determine the correct listing order of multi-dimensional'
infotxt[1]=' VAR_SIZE and VAR_DEPEND values in the HDF file'
END
ENDCASE
END
in0[0] EQ 12: BEGIN
infotxt=STRARR(2)
infotxt[0]=' INFORMATION: Table Attribute Values file must be in the same directory as the input file'
infotxt[1]=' if wanting to convert from one Data Format to another'
END
in0[0] EQ 13: BEGIN
infotxt=STRARR(2)
infotxt[0]=' INFORMATION: IDLcr8HDF called but it is not in the IDL Search Path or it encountered an'
infotxt[1]=' unexpected error. If the latter please contact Ian Boyd at iboyd@bryanscientific.org'
END
in0[0] EQ 14: BEGIN
infotxt=STRARR(2)
infotxt[0]=' INFORMATION: /POPUP keyword cannot be used together with the ''reterr'' argument.'
infotxt[1]=' The request for a POPUP window has been ignored'
END
in0[0] EQ 15: BEGIN
infotxt=STRARR(2)
infotxt[0]=' INFORMATION: Argument ''reterr'' must be of type string.' ;a returnable variable of type string.'
infotxt[1]=' idlcr8ascii will stop normally if an error is encountered'
END
in0[0] EQ 16: BEGIN
infotxt=STRARR(2)
infotxt[0]=' INFORMATION: /LOG, /FORMAT, and /DUMP keywords cannot be used in IDL DEMO mode and will be ignored.'
infotxt[1]=' To generate ASCII files please use the free IDL Virtual Machine or a licenced version of IDL'
END
in0[0] EQ 17: BEGIN
infotxt=STRARR(2)
infotxt[0]=' INFORMATION: NetCDF file create feature is disabled in IDL DEMO mode.'
infotxt[1]=' To generate files please use the free IDL Virtual Machine or a licenced version of IDL'
END
in0[0] EQ 18: BEGIN
infotxt=' INFORMATION: Value for Attribute Label '+in1[0]+' is an invalid Data Type'
END
in0[0] EQ 19: BEGIN
infotxt=' INFORMATION: '+in1[0]+' is not a valid GEOMS Metadata Attribute'
END
in0[0] EQ 20: BEGIN
infotxt=' INFORMATION: '+in1[0]+' entry must be written to the file as a STRING for '+in1[1]
END
in0[0] EQ 21: BEGIN
infotxt=' INFORMATION: '+in1[0]+' sub-values must be separated by '';'' for '+in1[1]
END
in0[0] EQ 22: BEGIN
infotxt=' INFORMATION: Spaces not permitted in the '+in1[0]+' entry for '+in1[1]
END
in0[0] EQ 23: BEGIN
infotxt=STRARR(2)
itxt1=' WARNING: This program can''t check that BYTE datasets and attributes contain only'
itxt2=' 8-bit unsigned values (0-255) in netCDF3 files.'
infotxt[0]=itxt1+itxt2
itxt1=' Please check that the '+in1[0]+' dataset and attributes'
itxt2=' with BYTE data type do not contain negative 8-bit signed values'
infotxt[1]=itxt1+itxt2
END
in0[0] EQ 24: BEGIN
infotxt=STRARR(2)
infotxt[0]=' INFORMATION: Input file name does not match the Global Attribute entry'
infotxt[1]=' FILE_NAME='+in1[0]
END
ENDCASE
dm=SIZE(infotxt,/N_Elements)
IF o3[3] EQ '' THEN lineno=lineno+dm
FOR n=0,dm-1 DO BEGIN
IF o3[3] EQ '' THEN BEGIN
IF n EQ dm THEN WIDGET_CONTROL,wtxt,set_value='',/Append $
ELSE WIDGET_CONTROL,wtxt,set_value=infotxt[n],/Append,Set_text_top_line=lineno
ENDIF
FOR m=dux[0],dux[1],dux[2] DO $
IF m EQ -1 THEN PRINT,infotxt[n] ELSE PRINTF,m,infotxt[n] ;write out to log(s)
ENDFOR
END ;Procedure InfoTxt_Output_A
FUNCTION jdf_2_datetime, jdf, MJD2000=mjd2000, SHORTISO8601=iso, LONGISO8601=isoms
;Computes the UT date/time from JDF/MJD2000.
; ----------
; Bojan R. Bojkov
; bojan.r.bojkov@nasa.gov
; 03/04/2004
; 03/11/2004 bug fix
; 05/27/2005 add ShortISO8601 and LongISO8601 switches,
; and change seconds value to decimal seconds (Ian Boyd)
;
; References ----------
; Explan. Supp. Astron. Almanac, p.604 (1992); through E. Celarier
;
; Caveats ----------
; Valid for all YYYY >= -4712 (i.e. for all JD .ge. 0)
; The true Gregorian calendar was only adopted on 15 October 1582,
; so any dates before this are "virtual" Gregorian dates.
;
; Input ----------
; jdf : double precision value
; mjd2000 : flag if input is MJD2000
; iso : flag if output is in ISO8601 (YYYYMMDDThhmmssZ)
; isoms : flag if output is in ISO8601ms (YYYYMMDDThhmmss.sssZ)
;
; Output ---------
; floating point array [YYYY, MM, DD, hh, mn, ss.sss],
; or ISO8601 string format
; External subroutines ---------
; NONE
jdf=DOUBLE(jdf)
j0=2451544.5D
IF KEYWORD_SET(mjd2000) THEN jdhold=jdf+j0 ELSE jdhold=jdf
jdi=LONG(jdhold)
df=jdhold-jdi
;Determine hh, mm, ss, ms
hh=(df+0.5)*24.D
q=WHERE(hh GE 24.D)
IF q[0] NE -1 THEN BEGIN
hh[q]=hh[q]-24.D
jdi[q]=jdi[q]+1
ENDIF
t1=hh
hh=LONG(t1)
t2=(t1-hh)*60.D
mn=LONG(t2)
t3=(t2-mn)*60.D
ss=t3
;ss=LONG(t3)
;ms=LONG((t3-ss)*1.d3)
;Determine YYYY, MM, DD
t1=jdi+68569L
t2=(4*t1)/146097L
t1=t1-(146097L*t2+3L)/4Lt3=(4000L*(t1+1L))/1461001L
t1=t1-(1461L*t3)/4L + 31L
t4=(80L*t1)/2447L
dd=t1-(2447L*t4)/80L
t1=t4/11L
mm=t4+2L-12L*t1
yyyy=100L*(t2-49L)+t3+t1
dt=TRANSPOSE([[yyyy],[mm],[dd],[hh],[mn],[ss]])
IF (KEYWORD_SET(iso)) OR (KEYWORD_SET(isoms)) THEN BEGIN
dts=STRARR(6)
IF (KEYWORD_SET(iso)) AND (ss-FIX(ss) GE 0.5) THEN BEGIN
;recalculate to get correct seconds value
jdhold=jdf+0.000008D ;add ~0.7 secs
IF KEYWORD_SET(mjd2000) THEN jdhold=jdhold+j0
CALDAT,jdhold,mm,dd,yyyy,hh,mn,ss
dt=TRANSPOSE([[yyyy],[mm],[dd],[hh],[mn],[ss]])
ENDIF
dt=LONG(dt)
FOR i=1,5 DO $
IF dt[i] LT 10L THEN dts[i]='0'+STRTRIM(dt[i],2) ELSE dts[i]=STRTRIM(dt[i],2)
IF KEYWORD_SET(isoms) THEN BEGIN
ssd=STRING(format='(f6.3)',ss)
IF FLOAT(ssd) LT 10.0 THEN dts[5]='0'+STRTRIM(ssd,2) ELSE dts[5]=STRTRIM(ssd,2)
ENDIF
RETURN,STRTRIM(dt[0],2)+dts[1]+dts[2]+'T'+dts[3]+dts[4]+dts[5]+'Z'
ENDIF ELSE RETURN, dt
END ;Function jdf_2_datetime
FUNCTION julian_date, date_ut, ISO8601=iso, MJD2000=mjd2000
;Computes the Julian date (jd) or date in MJD2000 format in double precision.
; ----------
; Bojan R. Bojkov
; bojan.r.bojkov@nasa.gov
; 06/03/2001
; 09/24/2001: Code cleanup - Function Version 1.0
; 10/16/2001: Mode corrections - Function Version 1.01
; 03/07/2002: Comment cleanup and variable change - Function Version 2.0
; 10/08/2002: Added jdf option - Function Version 2.1
; 07/21/2003: Converted to a function - Function Version 3.0
; 05/25/2005: Added ISO8601 and MJD2000 keywords.
; Input can either be a string or a numeric array
; (see below for input details). Returns -99999.d
; if the Input is invalid - Ian Boyd
;
; References ----------
; Hughes, D.W., Yallop, B.D. and Hohenkerk, C.Y., "The Equation of Time",
; Mon. Not. R. astr. Soc., 238, pp 1529-1535. 1989.
;
; Results verified using:
; Meeus, J, "Astronomical Algorithms", William-Bell, Inc., Richmond VA,
; p62, 1991.
;
; Caveats ----------
; NONE
;
; Input ----------
; date_ut: Numeric array (UT: YYYY, MM, DD, [hh, mm, ss]), or if ISO8601
; keyword set, a string containing datetime as YYYYMMDDThhmmssZ
;
; Output ---------
; jd: julian day, or if MJD2000 keyword set,
; a double precision day in MJD2000 format
;
; External subroutines ---------
; NONE
ON_IOERROR,TypeConversionError
valid=0 ;Type conversion check
inputerror=0 ;Check that input is as expected
;Check input format
IF KEYWORD_SET(iso) THEN BEGIN ;input is YYYYMMDDThhmmssZ
IF STRLEN(date_ut) EQ 16 THEN BEGIN
;test for numeric values (will return type conversion error value if not a number)
FOR i=0,7 DO check=FIX(STRMID(date_ut,i,1))
FOR i=9,14 DO check=FIX(STRMID(date_ut,i,1))
dt=INTARR(6)
dt[0]=FIX(STRMID(date_ut,0,4)) & dt[1]=FIX(STRMID(date_ut,4,2))
dt[2]=FIX(STRMID(date_ut,6,2)) & dt[3]=FIX(STRMID(date_ut,9,2))
dt[4]=FIX(STRMID(date_ut,11,2)) & dt[5]=FIX(STRMID(date_ut,13,2))
date_ut=dt & achk=[0,6] ;to add hhmmss component
ENDIF ELSE inputerror=1
ENDIF ELSE BEGIN
;check that input contains at least YMD (and at most YMDhms) information and is either
;an integer, long, float or double array
achk=SIZE(date_ut)
IF (achk[1] LT 3) OR (achk[1] GT 6) OR (achk[2] LT 2) OR (achk[2] GT 5) THEN inputerror=1
ENDELSE
IF inputerror EQ 0 THEN BEGIN ;can perform JD calculation
IF date_ut[1] GT 2 THEN BEGIN
y=DOUBLE(date_ut[0])
m=DOUBLE(date_ut[1]-3)
d=DOUBLE(date_ut[2])
ENDIF ELSE BEGIN
y=DOUBLE(date_ut[0]-1)
m=DOUBLE(date_ut[1]+9)
d=DOUBLE(date_ut[2])
ENDELSE
;Compute Julian date:
j=LONG(365.25D0*(y+4712.D0))+LONG(30.6D0*m+0.5D0)+59.D0+d-0.5D0
;Check for Julian or Gregorian calendar:
IF j LT 2299159.5D0 THEN jd=j $ ;If Julian calendar.
ELSE BEGIN ;If Gregorian calendar.
gn=38.D0-LONG(3.D0*LONG(49.D0+y/100.D0)/4.D0)
jd=j+gn
ENDELSE
;add hhmmss values if present
CASE 1 OF
achk[1] EQ 4: jd=jd+DOUBLE(date_ut[3])/24.D ;hh only
achk[1] EQ 5: jd=jd+(DOUBLE(date_ut[3])*3600.D + DOUBLE(date_ut[4])*60.D)/86400.D ;hhmm only
achk[1] EQ 6: jd=jd+(DOUBLE(date_ut[3])*3600.D + DOUBLE(date_ut[4])*60.D + $
DOUBLE(date_ut[5]))/86400.D ;hhmmss
ELSE:
ENDCASE
valid=1 ;Type conversion OK
;Set to MJD2000 if required
IF KEYWORD_SET(mjd2000) THEN jd=jd-2451544.5D
ENDIF
TypeConversionError:
IF valid EQ 0 THEN RETURN,-99999.D ELSE RETURN, jd
END ;Function Julian_Date
PRO nc_dimension_chk, ftype, sd_id, di, n_sds, dgi, n_sdsg
;Procedure to check for netCDF dimension datasets. These will be quietly ignored during the
;read process
; ----------;Written by Ian Boyd for the EVDC - iboyd@bryanscientific.org
;
; History:
; 20190806: Introduced - Version 4.0b22
;
; Inputs: ftype - format type, HDF5 or netCDF
; sd_id - GEOMS file identifier number
; di - index values for datasets
; n_sds - number of datasets determined by H5G_GET_NMEMBERS call
;
; Outputs: dgi - index values for GEOMS datasets (not generated by netCDF libraries)
; n_sdsg - number of GEOMS datasets
;
; Called by: Read_HDF_SDS
;
dimen_names=['constant','independent_','_strlen']
n_dn=N_ELEMENTS(dimen_names)
ds_names=STRARR(n_sds) & gdi=INTARR(n_sds)-1
FOR i=0,n_sds-1 DO BEGIN
IF ftype EQ 'H5' THEN sds_name=H5G_GET_MEMBER_NAME(sd_id,'/',di[i]) $
ELSE BEGIN ;netCDF file
varstruct=NCDF_VARINQ(sd_id,i)
sds_name=varstruct.name
ENDELSE
ds_names[i]=STRTRIM(STRLOWCASE(sds_name),2)
ENDFOR
FOR i=0,n_sds-1 DO BEGIN
dnicnt=0 & j=0
WHILE (dnicnt EQ 0) AND (j LE n_dn-1) DO $
IF STRPOS(ds_names[i],dimen_names[j]) NE -1 THEN dnicnt++ ELSE j++
IF dnicnt NE 0 THEN BEGIN
;Check that names are actually netCDF dimension names o/w will treat as a dataset name
CASE j OF
0: IF ds_names[i] NE dimen_names[j] THEN dnicnt=0 ;i.e. not a netCDF dimension name
1: BEGIN
;check that first part of the name is INDEPENDENT_ and the second part is a number
IF STRMID(ds_names[i],0,STRLEN(dimen_names[j])) NE dimen_names[j] $
THEN dnicnt=0 $
ELSE BEGIN
res=STRSPLIT(ds_names[i],'_',/EXTRACT,COUNT=cres)
res=[res,''] ;to ensure at least 2 extracted values in ds_names[i]
if ~IS_A_NUMBER_ASCII(res[1]) THEN dnicnt=0 ; i.e. not a netCDF dimension name
ENDELSE
END
2: BEGIN
;check that first part of the name is a named dataset
res=STRSPLIT(ds_names[i],'_',/EXTRACT)
ni=WHERE(res[0] EQ ds_names,ncnt)
IF ncnt EQ 0 THEN dnicnt=0 ;i.e. not a netCDF dimension name
END
ENDCASE
ENDIF
IF dnicnt EQ 0 THEN gdi[i]=i ;i.e. this is a valid dataset name and not a netCDF dimension name
ENDFOR
dgi=WHERE(gdi NE -1,n_sdsg) ;dgi and n_sdsg are returned by the procedure
END ;Procedure nc_dimension_chk
FUNCTION file_format_a, infile
;Function to identify the format of the input file - tests for HDF4, HDF5/netCDF4 and netCDF3.
;netCDF4 files will be identified as HDF5 and read in using the HDF5 library
; ----------
;Written by Ian Boyd for the EVDC - iboyd@bryanscientific.org
;
; History:
; 20190806: Introduced - Version 4.0b22; 20220805: Additional checks for netCDF4 files if the file signature ia HDF5 - Version 4.0b25
;
; Inputs: infile - Input GEOMS compliant file
;
; Outputs: string identifying the type of file (H4, H5, NC) or XX for unidentified file
;
; Called by: IDLCR8ASCII
;
;Markers
;HDF4: 14 3 19 (1 0 16 0 1 95 98 0 30)
;HDF5: 137 72 68 70 (13 10 26 10 0 0 0 0)
;nc3: 67 68 70 (1 0 0 0 0 0 0 0 10)
;nc4: 137 72 68 70 (13 10 26 10 (2 8 8 0))
h4id=[14B, 3B, 19B]
h5id=[137B, 72B, 68B, 70B] ;HDF
nc3id=[67B, 68B, 70B] ;CDF
;First test using start of file byte markers
fid=BYTARR(4)
openr,fu,infile,/GET_LUN
readu,fu,fid
FREE_LUN,fu
idfound='XX'
CASE 1 OF
ARRAY_EQUAL(fid[0:2],h4id): idfound='H4'
ARRAY_EQUAL(fid,h5id): idfound='H5'
ARRAY_EQUAL(fid[0:2],nc3id): idfound='N3'
ELSE:
ENDCASE
;Second test, if required, using IDL commands
IF idfound EQ 'XX' THEN BEGIN
IF HDF_ISHDF(infile) EQ 1 THEN idfound='H4' $
ELSE IF FLOAT(!Version.Release) GE 5.6 THEN BEGIN
IF H5F_IS_HDF5(infile) EQ 1 THEN idfound='H5' ;netCDF4 or HDF5
ENDIF
IF idfound EQ 'XX' THEN BEGIN
;final test for netCDF3
validname=1
CATCH, ncdferror ;To catch the error when attempting to open the file as a netCDF
;Error Handler
IF ncdferror NE 0 THEN BEGIN ;not a netCDF file
validname=0
CATCH, /CANCEL
ENDIF
IF validname EQ 1 THEN BEGIN
fileid=NCDF_OPEN(infile)
;If get to here then file is netCDF
CATCH, /CANCEL
NCDF_CLOSE,fileid
idfound='N3'
ENDIF
ENDIF
ENDIF
IF idfound EQ 'H5' THEN BEGIN
;For H5, try to identify whether it is H5 or N4
nc4ext=['nc','nc4','netcdf','netcdf4']
fileext=STRMID(infile,STRPOS(infile,'.',/REVERSE_SEARCH)+1)
nci=WHERE(STRLOWCASE(fileext) EQ nc4ext,nccnt)
vr=!Version.Release ;IDL version check (needs to be 8.5.2 or greater for NCDF_PARSE check
vrlast=STRMID(vr,STRPOS(vr,'.',/REVERSE_SEARCH)+1)
IF (FLOAT(vr) GT 8.5) OR ((STRMID(vr,0,3) EQ '8.5') AND (FIX(vrlast GE 2))) THEN parsechk=1B ELSE parsechk=0B
IF (nccnt NE 0) AND (parsechk) THEN BEGIN
;Catch error from attempting to parse non-NetCDF-4 compliant HDF5 file
validname=1
CATCH, ncdferror ;To catch the error when attempting to open the file as a netCDF ;Error Handler
IF ncdferror NE 0 THEN BEGIN ;not a netCDF file
validname=0
CATCH, /CANCEL
ENDIF
IF validname EQ 1 THEN BEGIN
fileid=NCDF_PARSE(infile)
;If get to here then file can be opened using N4 libraries, so most likely a netcdf4 file
CATCH, /CANCEL
idfound='N4'
ENDIF
ENDIF ELSE IF nccnt NE 0 THEN idfound='N4' ;Unable to do NCDF_PARSE check but still likely nc4
ENDIF
RETURN, idfound
END ;File_Format_A
PRO test_dim_order, va_name, va_value, va_type, sds_dim, sds_name, rev_vd_vs
;Procedure to test the dimension ordering of any multi-dimensional datasets in the input DF file, and
;return correct ordering code (dimension ordering for idlcr8ascii uses IDL/Fortran convention)
; ----------
;Written by Ian Boyd for the EVDC/AVDC - iboyd@bryanscientific.org
;
; History:
; 20111208: Introduced - Version 4.0b4
; 20140325: Fix bug that meant VAR_SIZE and VAR_DEPEND values were not being automatically
; reversed if the array sizes were the same and VAR_DEPEND did not include a DATETIME
; value. Now defaults to the 'reverse' option if no other conditions are satisfied;
; Fix bug that caused crash when VAR_SIZE values were not of type string - Version 4.0b8
; 20141110: Fix bug that caused multi-dimensional array ordering to not be correctly identified
; if the first dataset to be checked in the file had the same number of elements in
; the array (e.g. was a set of Averaging Kernels) - Version 4.0b9
; 20200930: Add INFORMATION messages (20-22) that identify issues with the VAR_DEPEND and VAR_SIZE
; attributes that were previously fixed 'quietly' by the program; Add sds_name to the
; variables required by the procedure - Version 4.0b24
;
; Inputs: va_name - An abbreviated version of the Variable Name; either 'VD' (VAR_DEPEND) or 'VS'
; (VAR_SIZE)
; va_value - The corresponding Variable Value
; sds_dim - Dimension information for the DF Dataset being tested
; sds_name - Dataset name, required if an INFORMATION message is generated
;
; Outputs: rev_vd_vs - Scalar to indicate whether VAR_DEPEND and VAR_SIZE values (and data, in the case
; of netCDF measurements) need to be reversed
;
; Called by: Read_HDF_SDS
IF va_type NE 'STRING' THEN BEGIN ;VAR_SIZE contains numeric values instead of in the form of a string
vavhold='' & n_vav=N_ELEMENTS(va_value)
FOR k=0,n_vav-1 DO BEGIN
IF k EQ n_vav-1 THEN vtxt='' ELSE vtxt=';'
vavhold=vavhold+STRTRIM(va_value[k],2)+vtxt
ENDFOR
IF va_name EQ 'VD' THEN vtxt='VAR_DEPEND' ELSE vtxt='VAR_SIZE'
INFOTXT_OUTPUT_A,[20],[vtxt,STRTRIM(sds_name,2)]
ENDIF ELSE vavhold=va_value
vs_v=STRCOMPRESS(STRSPLIT(vavhold,';, ',/EXTRACT,COUNT=rcnt),/REMOVE_ALL)
IF (rcnt GT 1) AND ((STRPOS(vavhold,';') EQ -1) OR (STRPOS(vavhold,',') NE -1)) THEN BEGIN
IF va_name EQ 'VD' THEN itxt='VAR_DEPEND' ELSE itxt='VAR_SIZE'
INFOTXT_OUTPUT_A,[21],[itxt,STRTRIM(sds_name,2)] ;sub-values should be separated by ';'
ENDIF
IF (rcnt EQ 1) OR ((rcnt GT 1) AND (STRPOS(vavhold,';') NE -1)) THEN BEGIN
IF STRCOMPRESS(vavhold,/REMOVE_ALL) NE vavhold THEN BEGIN
IF va_name EQ 'VD' THEN itxt='VAR_DEPEND' ELSE itxt='VAR_SIZE'
INFOTXT_OUTPUT_A,[22],[itxt,STRTRIM(sds_name,2)] ;no spaces permitted ENDIF
ENDIF
IF va_name EQ 'VD' THEN BEGIN ;VAR_DEPEND tests
dti=WHERE(STRUPCASE(vs_v) EQ 'DATETIME',dticnt)
;Note: if DATETIME has VAR_SIZE=1 then the ordering is dependent on the order of the VAR_SIZE values only
IF (rcnt GT 1) AND (dticnt NE 0) THEN BEGIN ;multi-dimensions including DATETIME
test1=(dti[0] EQ rcnt-1) AND (sds_dim[0] GT 1)
;DATETIME is the last value and present only once and has VAR_SIZE GT 1
test2=(dti[0] EQ 0) AND (dticnt EQ 1) AND (sds_dim[0] GT 1)
;DATETIME is the first value and present only once and has VAR_SIZE GT 1
IF (test1) AND (rev_vd_vs EQ 2) THEN rev_vd_vs=10 $
ELSE IF (test2) AND (rev_vd_vs EQ 2) THEN rev_vd_vs=11 $
ELSE IF ((test1) AND (rev_vd_vs MOD 10 EQ 1)) OR ((test2) AND (rev_vd_vs EQ 10)) THEN $
rev_vd_vs=3 $ ;inconsistent rules regarding dimension ordering in file
ELSE IF (test2) AND (rev_vd_vs EQ 0) THEN rev_vd_vs=20
;Dimension ordering is slowest changing first so change dataset ordering as well as
;VAR_DEPEND and VAR_SIZE ordering
ENDIF
ENDIF ELSE BEGIN ;VAR_SIZE tests
IF rcnt GT 1 THEN BEGIN ;multi-dimensions
;check all values are numeric
numchk=STRJOIN(vs_v,/SINGLE) & valok=1
FOR k=0,STRLEN(numchk)-1 DO BEGIN
bchar=BYTE(STRMID(numchk,k,1))
IF (bchar LT 48) OR (bchar GT 57) THEN valok=0 ;i.e. non-numeric character
ENDFOR
IF valok EQ 1 THEN BEGIN ;all values are numeric so do checks
vs_vl=LONG(vs_v)
test1=ARRAY_EQUAL(vs_vl,sds_dim)
test2=ARRAY_EQUAL(REVERSE(vs_vl),sds_dim) ;test in case values are the same e.g. 40;40
;If arrays are the same size then the default will be to reverse the VS and VD values
;unless a different dimension ordering has already been identified
IF (test1 EQ 1) AND (test2 EQ 0) AND (rev_vd_vs EQ 2) THEN rev_vd_vs=0 $
ELSE IF (test1 EQ 0) AND (test2 EQ 1) AND (rev_vd_vs EQ 2) THEN rev_vd_vs=1 $
ELSE IF ((test1 EQ 1) AND (test2 EQ 0) AND (rev_vd_vs MOD 10 EQ 1)) OR $
((test1 EQ 0) AND (test2 EQ 1) AND (rev_vd_vs MOD 10 EQ 0)) THEN $
rev_vd_vs=3 $ ;inconsistent VAR_SIZE and sds_dim agreement from dataset to dataset
ELSE IF (test1 EQ 1) AND (test2 EQ 1) AND (rev_vd_vs EQ 2) THEN $
rev_vd_vs=2 $ ;first set of measurements tested have same dimensions so do not do anything
ELSE IF rev_vd_vs EQ 2 THEN rev_vd_vs=1 ;default if no other criteria are found as this
;will then automatically reverse the VD and VS values
ENDIF
ENDIF
ENDELSE
END ;Procedure Test_Dim_Order
PRO data_type_checks, ftype, dt_chk_labels, hdftype, idltype, sds_name, var_name
;VAR_DATA_TYPE corresponds to the dataset data type and the VAR_VALID_MIN, VAR_VALID_MAX
;and VAR_FILL_VALUES also match the dataset data type. Note limitation that 8-bit
;unsigned integer (BYTE) values cannot be identified in netCDF3 files
; ----------
;Written by Ian Boyd for the EVDC/AVDC - iboyd@bryanscientific.org
;
; History:
; 20220805: Introduced to IDLCR8ASCII - Version 4.0b25
;
; Inputs: ftype - a string identifying the file type (H4, H5, N3 or N4)
; dt_chk_labels - string array containing labels of attributes (plus 'Dataset')
; upon which the data type checks are done
; hdftype - string array containing the data type returned by the respective
; ftype specific calls
; idltype - string array containing the data types as IDL codes or names
; sds_name - string holding dataset name returned by the respective ftype specific
; cals
; var_name - string holding the VAR_NAME value (should be the same as sds_name)
;; Outputs: N/A
; Called by: READ_HDF_SDS
;
; Subroutines Called: INFOTXT_OUTPUT_A (if program discovers an issue with the data type)
; Information Conditions:
; 1. VAR_DATA_TYPE is not an allowable data type
; 2. Dataset, Valid Min, Valid Max and/or Fill Value data types do not match the VAR_DATA_TYPE (if applicable)
; 3. Valid Min, Valid Max and/or Fill Value data types do not match the Dataset data type (if applicable)
; 3. Dataset, Valid Min, Valid Max and/or Fill value data types are not allowable
;Allowable VAR_DATA_TYPEs
avdt=['','BYTE','SHORT','INTEGER','REAL','DOUBLE','','STRING','','','','','','',''] ;'LONG'] ;corresponding to IDL codes 1,2,3,4,5,7,14 (LONG/14 not used)
idl_name=['UNDEFINED','BYTE','SHORT','INTEGER','REAL','DOUBLE','COMPLEX','STRING','STRUCT','DCOMPLEX', $
'POINTER','OBJREF','UINT','ULONG','LONG64','ULONG64'] ;Note: Integer, Long and Float renamed Short, Integer and Real for sds_type
hdf_dt=['','DFNT_UINT8','DFNT_INT16','DFNT_INT32','DFNT_FLOAT32','DFNT_FLOAT64','','DFNT_CHAR8'] ;allowable HDF4 datatypes (note BYTE=8-bit unsigned)
;hdf_all=['DFNT_NONE','DFNT_UINT8','DFNT_INT16','DFNT_INT32','DFNT_FLOAT32','DFNT_FLOAT64','','DFNT_CHAR8', $
; '','','','','DFNT_UINT16','DFNT_UINT32','DFNT_INT8']
h5_dt=['H5T_INTEGER_0','H5T_INTEGER_1','H5T_FLOAT','H5T_STRING'] ;allowable HDF5/NC4 datatypes (0=unsigned, 1=signed)
;dt_chk_labels=['Dataset','VAR_VALID_MIN','VAR_VALID_MAX','VAR_FILL_VALUE','VAR_DATA_TYPE'] ;dataset and attribute labels for data type checks
n_lab=N_ELEMENTS(dt_chk_labels)
dterr=STRARR(n_lab) ;non-zero indicates error
IF var_name NE '' THEN usevn=var_name ELSE usevn=sds_name ;determine VAR_NAME value to use for error messages
IF (ftype EQ 'H5') OR (ftype EQ 'N4') THEN BEGIN
;need to convert hdftype and idltype values to equivalent idl_name and hdf_dt values
FOR i=0,n_lab-2 DO BEGIN
idltype[i]=idl_name[idltype[i]] ;convert IDL type code to type name
CASE 1 OF
hdftype[i] EQ 'H5T_FLOAT': BEGIN
IF idltype[i] EQ 'DOUBLE' THEN hdftype[i]='DFNT_FLOAT64' $
ELSE hdftype[i]='DFNT_FLOAT32'
END
hdftype[i] EQ 'H5T_STRING': hdftype[i]='DFNT_CHAR8'
hdftype[i] EQ 'H5T_INTEGER_0': BEGIN
CASE idltype[i] OF
'BYTE': hdftype[i]='DFNT_UINT8'
('SHORT') OR ('UINT'): hdftype[i]='DFNT_UINT16'
('INTEGER') OR ('ULONG'): hdftype[i]='DFNT_UINT32'
'ULONG64': hdftype[i]='DFNT_UINT64'
ELSE: hdftype[i]='DFNT_UNDF'
ENDCASE
END
hdftype[i] EQ 'H5T_INTEGER_1': BEGIN
CASE idltype[i] OF
'BYTE': hdftype[i]='DFNT_INT8'
'SHORT': hdftype[i]='DFNT_INT16'
'INTEGER': hdftype[i]='DFNT_INT32'
'LONG64': hdftype[i]='DFNT_INT64'
ELSE: hdftype[i]='DFNT_UNDF'
ENDCASE
END
ELSE:
ENDCASE
ENDFOR
ENDIF ELSE IF ftype EQ 'N3' THEN BEGIN
;need to convert hdftype and idltype values to equivalent idl_name and hdf_dt values
writeonce=0B
FOR i=0,n_lab-2 DO BEGIN
idltype[i]=idl_name[idltype[i]] ;convert IDL type code to type name
CASE 1 OF
hdftype[i] EQ 'BYTE_0': hdftype[i]='DFNT_UINT8'
hdftype[i] EQ 'BYTE_1': hdftype[i]='DFNT_INT8'
hdftype[i] EQ 'INT': hdftype[i]='DFNT_INT16'
hdftype[i] EQ 'LONG': hdftype[i]='DFNT_INT32'
hdftype[i] EQ 'FLOAT': hdftype[i]='DFNT_FLOAT32'
hdftype[i] EQ 'DOUBLE': hdftype[i]='DFNT_FLOAT64'
hdftype[i] EQ 'CHAR': hdftype[i]='DFNT_CHAR8'
ELSE: hdftype[i]='DFNT_UNDF' ENDCASE
IF (idltype[i] EQ 'BYTE') AND (idltype[0] EQ 'BYTE') AND (~writeonce) THEN BEGIN
;For VAR_VALID_MIN/MAX and VAR_FILL_VALUE only output if the dataset is of type BYTE as well
;o/w probably means that the value was written as a character string instead of as numbers
INFOTXT_OUTPUT_A,[23],[usevn]
writeonce=1B
ENDIF
ENDFOR
ENDIF
;Check VAR_DATA_TYPE value
hdti=WHERE((idltype[n_lab-1] EQ avdt) AND (idltype[n_lab-1] NE ''),hdtcnt)
IF hdtcnt EQ 0 THEN BEGIN
IF idltype[n_lab-1] NE '' THEN $
dterr[n_lab-1]=usevn+' '+dt_chk_labels[n_lab-1]+'='+idltype[n_lab-1]+' is not a valid GEOMS data type'
ENDIF
;Check dataset, min, max and fill data types are valid
FOR i=0,n_lab-2 DO BEGIN
IF i EQ 0 THEN itxt=' value(s) ' ELSE itxt=' value '
hdti=WHERE((hdftype[i] EQ hdf_dt) AND (hdftype[i] NE ''),hdtcnt)
IF hdtcnt EQ 0 THEN BEGIN ;identify the actual data type of the dataset
hdti=WHERE(idltype[i] EQ idl_name,hdtcnt)
IF hdtcnt EQ 0 THEN IF idltype[i] NE '' THEN hdti[0]=0
CASE hdti[0] OF
-1: IF i EQ 0 THEN dterr[i]=usevn+' '+dt_chk_labels[i]+' data type is not identifiable'
1: dterr[i]=usevn+' '+dt_chk_labels[i]+itxt+'must be 8-bit unsigned INTEGER (BYTE)'
2: dterr[i]=usevn+' '+dt_chk_labels[i]+itxt+'must be 16-bit signed INTEGER'
3: dterr[i]=usevn+' '+dt_chk_labels[i]+itxt+'must be 32-bit signed INTEGER'
ELSE: dterr[i]=usevn+' '+dt_chk_labels[i]+itxt+'not a valid GEOMS data type: '+idl_name[hdti[0]]
ENDCASE
ENDIF
ENDFOR
;If valid VAR_DATA_TYPE is present check that the dataset and min, max and fill data types match
IF dterr[n_lab-1] EQ '' THEN BEGIN
;Use error generated in SET_UP_STRUCTURE in idlcr8hdf instead
;FOR i=0,n_lab-2 DO BEGIN
; IF (idltype[n_lab-1] NE idltype[i]) AND (idltype[i] NE '') THEN BEGIN
; dterr[i]=usevn+' '+dt_chk_labels[i]+' data type ('+hdftype[i]+') does not match VAR_DATA_TYPE='+idltype[n_lab-1]
; ;note: overwrites any existing reported error
; ENDIF
;ENDFOR
ENDIF ELSE IF dterr[0] EQ '' THEN BEGIN
;VAR_DATA_TYPE not valid so check min, max and fill data types against the dataset data type
FOR i=1,n_lab-2 DO BEGIN
IF (idltype[0] NE idltype[i]) AND (idltype[i] NE '') THEN BEGIN
dterr[i]=usevn+' '+dt_chk_labels[i]+' data type ('+idl_name[i]+') does not match Dataset data type ('+idl_name[0]+')'
;note: overwrites any existing reported error
ENDIF
ENDFOR
ENDIF
;Output information messages
FOR i=0,n_lab-1 DO $
IF dterr[i] NE '' THEN INFOTXT_OUTPUT_A,[2],[dterr[i]]
END ;Data_Type_Checks
PRO read_hdf_sds, ifile, ga, sds, catinfo
;Procedure to read the contents of a GEOMS standard HDF or netCDF compatible file
;into session memory
; ----------
;Written by Ian Boyd for the EVDC/AVDC - iboyd@bryanscientific.org
;
; History:
; 20050729: Original IDLCR8ASCII Routine - Version 1.0; 20050912: Removed Common variable definition CATALOGINFO and made the variable a parameter passed
; to the procedure; Checks that the attribute variable names read by HDF_SD_GETINFO, match
; those recorded in DATA_VARIABLES in the Global Attributes. If not use the variable name
; listed in DATA_VARIABLES - Version 1.1
; 20061004: Common variable definition WIDGET_WIN_A added for Error calls; Add option to read HDF5
; files (needs IDL5.6 or newer); Add check that the HDF4 file has valid information on the
; number of global and variable attributes; Checks added in the event that the HDF4 or
; HDF5 file has been created by the NCSA utility programs H5toH4 or H4toH5 from an original
; AVDC/Envisat/NDACC HDF4 or HDF5 file; Ensures variable attributes and data are listed in
; the same order as that given under DATA_VARIABLES - Version 2.0
; 20081020 - If the HDF4 file is created using the HDF4.2r3 library, then extra information
; regarding the dimensions of the VAR_DEPEND values may be included in the file (which
; show up as extra datasets in the HDF_SD_FILEINFO call). A check for this is carried out
; and, if found, the information is excluded from the output - Version 3.01
; 20090311: Improve HDF4.2r3 library checks by comparing the number of datasets recorded under
; DATA_VARIABLES, with the number returned by the HDF_SD_FILEINFO call; Change 'WARNING' to
; 'INFORMATION and make calls to procedure INFOTXT_OUTPUT_A to avoid duplication. Add
; 'INFORMATION' text if anything irregular found with the datasets; Change 'Data dimensions
; exceed 8' and 'Data type not allowable' ERROR messages to 'INFORMATION' messages
; - Version 3.02
; 20091208: Incorporate HDF_SD_ISCOORDVAR to differentiate between datasets and dimension variable
; names. Add check for allowable VAR_DATA_TYPE=STRING. Add all INFORMATION text messages to
; INFOTXT_OUTPUT_A procedure to avoid duplication of code that creates the messages;
; Add INFORMATION messages when reading HDF5 files; Improve checks for non-standard (i.e.
; non-groundbased) HDF input file - Version 3.03
; 20101122: Numeric metadata values remain in their original data type when transferring to session
; memory, meaning: variable attribute labels and values now written to separate structure
; variables sds.va_l and sds.va_v (previously sds.va), and structure has 2 dimensions
; (n_sds, n_atts) instead of 1 (previously n_sds); The dataset is written to the initial
; n_atts index (i.e. sds[n_sds,0].data); Rename saved SDS datatype according to GEOMS rules
; (affects INT, LONG, and FLOAT); Compare HDF dimensions with VAR_SIZE values and transpose
; VAR_SIZE and VAR_DEPEND values if required - Version 4.0b1
; 20111014: Add netCDF read capability - writes variable attributes to standard GEOMS variable
; attribute labels where possible, otherwise appends information to VAR_NOTES - Version 4.0b3
; 20120426: Bug fix when a netCDF file only contains datasets with single dimensions, then VAR_SIZE
; information was not being extracted during the read process - Version 4.0b6
; 20140325: Fix bug that caused crash when VAR_SIZE values were not of type string - Version 4.0b8
; 20150127: Allow for VAR_FILL_VALUE values for string datasets to be written and saved as an
; (empty) string of the correct length. Ensure string datasets are all written and
; saved to the correct length (that of the longest string in the dataset) - Version 4.0b10
; 20150217: Do not remove whitespace of any variable attribute values that are written as strings
; - Version 4.0b11
; 20160614: Some HDF4 string datasets may not have 2 dimensions, so do not need to remove the first
; dimension (= maximum number of characters) - Version 4.0b14
; 20160725: Fix bug causing program to crash when the number of DATA_VARIABLES values is greater
; than the number of datasets in the file - Version 4.0b15
; 20161130: Assign SDS_name to catinfo for attributes when the VAR_NAME is not present; rewrite
; netCDF section to align it with HDF5 section and to account for dataset ordering that
; does not match that in DATA_VARIABLES - Version 4.0b16
; 20161213: Fix bug when reading in netCDF files to stop all dataset variable attribute values to
; be converted to string format (now matches HDF4 and HDF5 checks). Fix bug that caused
; the program to crash when VAR_DEPEND=CONSTANT in netCDF files (sds_ndim=0) - Version 4.0b17
; 20161218: Fix bug when reading netCDF attribute values that are of type byte rather than string -
; Version 4.0b18
; 20180218 Fix bug that caused multi-dimensional array ordering to not be correctly identified
; if the datasets have the same VAR_SIZE. Now rev_vd_vs changes to 1 (VAR_SIZE and
; VAR_DEPEND ordering is reversed) if rev_vd_vs=2 after all the checks - Version 4.0b19
; 20181116 Fix bug that caused multi-dimensional array ordering to not be correctly written if
; not consistent through the file i.e. rev_vd_vs eq 3. Now rev_vd_vs changes to 1
; (VAR_SIZE, VAR_DEPEND and dataset ordering is reversed), but information message
; advising of the issue is still reported - Version 4.0b20
; 20190514 Fix bug that occurred when GEOMS variable attribute information is missing from an
; HDF4 file but written to the heap structure based on the contents of the file. Previously
; the variable attribute label was not written correctly - Version 4.0b21
; 20190806 Provide support for reading netCDF4 files. Currently uses the HDF5 routines to do this
; so need to account for features unique to netCDF4 such as; the inclusion of dimension
; names; standard netCDF attribute names; empty attribute values that do not include a
; null termination character (which causes an error in the H5A_Read routine)
; - Version 4.0b22
; 20190821 Fixed bugs associated with H5 TAG_NAMES checks introduced in v4.0b22 - Version 4.0b23; 20200930 Add sds_name to TEST_DIM_ORDER procedure call - Version 4.0b24
; 20220805 For NetCDF3 or 4, if SDS_NAME NE VAR_NAME then converts non-alphanumeric VAR_NAME
; characters to '_' and compares with SDS_NAME. If there is a match then program quietly
; converts SDS_NAME to VAR_NAME for testing, otherwise generates an error; Add code to
; collect information for the call to the DATA_TYPE_CHECKS procedure; Generate an
; INFORMATION/ERROR message if VAR_UNITS is numeric (same as VAR_SIZE); INFORMATION/ERROR
; message added if a mis-match is found between the input file name and the FILE_NAME attribute value.
;
; Inputs: ifile - a string containing the name of the input file to be read in.
; catinfo - a string array identifying the type of input file ('H4','H5','N4','N3')
;
; Outputs: ga - a string array containing the global attribute labels and values extracted from the HDF
; file.
; sds - a structure using pointers, of size [n_sds,n_atts], containing the variable attribute
; labels and values (sds[n,m].va_l and sds[n,m].va_v) and the data (sds[n,0].data)
; extracted from the HDF file.
; catinfo - a string array of size [n_sds,4] (where n_sds is the number of datasets in the HDF
; file), containing information on the variable name, data type, data dimension, and
; number of attributes.
;
; Called by: IDLCR8ASCII
;
; Subroutines Called: STOP_WITH_ERROR_A (if error state detected)
; INFOTXT_OUTPUT_A (if program can make a change)
; DATA_TYPE_CHECKS (to perform data type checks on the datasets and some attributes)
; Possible Conditions for STOP_WITH_ERROR call (plus [line number] where called):
; 1. No global attributes or datasets present in the root group of an HDF5 file.
; 2. The number of global attributes or datasets is zero or not able to be determined
; in an HDF4 file.
;
; Information Conditions (when the program is able to make changes):
; 1. DATA_VARIABLES global attribute not found or has no values.
; 2. Number of datasets given under DATA_VARIABLES is not equal to the number saved to the
; file.
; 3. The number of dimensions in a dataset exceeds 8.
; 4. The data type of a dataset is not BYTE, SHORT, INTEGER, LONG, REAL, DOUBLE, or STRING.
; 5. Dataset name is truncated or does not match VAR_NAME value.
; 6. Metadata label VAR_NAME not found for the dataset.
; 7. VAR_NAME does not match any DATA_VARIABLES values.
; 8. No Variable Attributes returned after HDF call.
; 9. Metadata Dataset listing order may not match the order of the Global Attribute
; DATA_VARIABLES Values.
; 10. Variable is written to the HDF file as a coordinate variable.
COMMON WIDGET_WIN_A
;Note any calibrated (scaled) HDF4 dataset is corrected using the formula specified in
;UG_print42r3.pdf pg. 3-107: orig = cal * (cal_val - offset)
ncsa_cal=['scale_factor','scale_factor_err','add_offset','add_offset_err','calibrated_nt']
;Initialize scalar to indicate whether VAR_DEPEND and VAR_SIZE values need to be reversed
rev_vd_vs=2 ;1 = reverse attribute values; 0 = no reverse (and will generate message);
;2 = no action required; 3 = rev_vd_vs changed between 0 and 1 during checking, therefore error
;Note if rev_vd_vs is 2 or 3 after all the checks then it will be changed to 1 so that dimensions
;are swapped by default
;List of Standard GEOMS Variable Attributes
attr_arr_data=['VAR_NAME','VAR_DESCRIPTION','VAR_NOTES','VAR_SIZE','VAR_DEPEND',$
'VAR_DATA_TYPE','VAR_UNITS','VAR_SI_CONVERSION','VAR_VALID_MIN',$
'VAR_VALID_MAX','VAR_FILL_VALUE']
n_aad=N_ELEMENTS(attr_arr_data)
;Possible error messages for this procedure
proname='Read_HDF_SDS procedure: '
errtxt=STRARR(2)
errtxt[0]=' found in the Root Group of the HDF5 File.'
errtxt[1]=' is zero or not able to be determined (check for corrupted file).'
lu=-1
slabel=['VAR_UNITS','VAR_SI_CONVERSION','VAR_VALID_MIN','VAR_VALID_MAX','VAR_FILL_VALUE'] ;save values as is if dataset is of type STRING
dt_chk_labels=['Dataset','VAR_VALID_MIN','VAR_VALID_MAX','VAR_FILL_VALUE','VAR_DATA_TYPE'] ;dataset and attribute labels for data type checks
n_lab=N_ELEMENTS(dt_chk_labels)
;If necessary, free up memory by destroying the heap variables pointed at by its pointer arguments
;from previous calls to this program
IF N_ELEMENTS(sds) NE 0 THEN PTR_FREE,sds.va_l,sds.va_v,sds.data
;Define the HDF SDS storage structure
sds_set={va_l: PTR_NEW(), $ ;Variable Attribute Labels
va_v: PTR_NEW(), $ ;Variable Attribute Values
data: PTR_NEW()} ;SD data array
ftype=catinfo[0,0]
ifilechk=STRLOWCASE(STRTRIM(FILE_BASENAME(ifile),2)) ;for FILE_NAME check
IF ftype EQ 'H4' THEN BEGIN
;The HDF_SD_START function opens an HDF file and initializes the SD interface.
sd_id=HDF_SD_START(ifile,/READ)
;Determine the number of SDS (n_sds) and global attributes (n_ga) found
;in the current file.
HDF_SD_FILEINFO,sd_id,n_sds,n_ga
IF n_sds EQ 0L THEN ntxt='Number of SDS datasets' $
ELSE IF n_ga EQ 0L THEN ntxt='Number of Global Attributes' $
ELSE ntxt=''
IF ntxt NE '' THEN BEGIN
STOP_WITH_ERROR_A,o3[3]+proname,ntxt+errtxt[1],lu & RETURN
ENDIF
ga=STRARR(n_ga) ;set the Global Attribute dimensions
vn=[''] ;initialize vn array (to hold Dataset names)
n_sds_hold=n_sds & n_vn=0L & do_coordvar=0
;Read the file's Global Attributes and determine number of DATA_VARIABLES
FOR i=0L,n_ga-1L DO BEGIN
;The HDF_SD_ATTRINFO procedure reads or retrieves info about an SD attribute.
HDF_SD_ATTRINFO,sd_id,i,NAME=ga_name,DATA=ga_data,HDF_TYPE=ga_hdftype
ga_name=STRTRIM(ga_name,2) & ga_data=STRTRIM(ga_data,2)
;Assign global attributes to ga
ga[i]=ga_name+'='+ga_data
;read list of DATA_VARIABLES into array
IF STRUPCASE(ga_name) EQ 'DATA_VARIABLES' THEN vn=STRSPLIT(ga_data,' ;',/Extract,COUNT=n_vn)
;String labels for GA not checked at this time as extra Global Attributes could be numeric
;IF (ga_hdftype NE 'DFNT_NONE') AND (ga_hdftype NE 'DFNT_CHAR8') THEN $
; INFOTXT_OUTPUT_A,[20],['Global Attribute',STRTRIM(ga_name,2)]
IF STRUPCASE(ga_name) EQ 'FILE_NAME' THEN BEGIN
;check that the actual file name matches the FILE_NAME entry
IF STRLOWCASE(STRTRIM(ga_data,2)) NE ifilechk THEN INFOTXT_OUTPUT_A,[24],[ga_data]
ENDIF
ENDFOR
IF n_sds_hold NE n_vn THEN BEGIN
;Do check for sds being a Dimension attribute
do_coordvar=1
FOR i=0L,n_sds-1L DO BEGIN
;The HDF_SD_SELECT function returns an SD dataset ID given the current
;SD interface ID, and the zero-based SD dataset index.
sds_id=HDF_SD_SELECT(sd_id,i)
IF HDF_SD_ISCOORDVAR(sds_id) THEN n_sds_hold=n_sds_hold-1L
;Closes the SDS interface.
HDF_SD_ENDACCESS,sds_id
ENDFOR
IF n_sds_hold EQ 0L THEN BEGIN
STOP_WITH_ERROR_A,o3[3]+proname,'Number of SDS datasets'+errtxt[1],lu
RETURN
ENDIF
IF n_sds_hold NE n_vn THEN $
INFOTXT_OUTPUT_A,[0],[STRTRIM(n_vn,2),STRTRIM(n_sds_hold,2),'HDF_SD_FILEINFO']
;DATA_VARIABLES values not read successfully, or number of datasets given under DATA_VARIABLES ;is not equal to the number saved to the file
ENDIF
;Determine maximum number of attributes, and match the dataset order to DATA_VARIABLES list if possible
max_atts=0L & oi=LONARR(n_sds_hold)
dv_order=1 & c_sds=0L
catinfo=STRARR(n_sds_hold,4) ;output info for catalog output
FOR i=0L,n_sds-1L DO BEGIN
vnv='' & sds_name=''
hdftype=STRARR(n_lab-1) & idltype=STRARR(n_lab) ;to hold data types for dataset, min, max and fill values, and VAR_DATA_TYPE (idltype only)
;The HDF_SD_SELECT function returns an SD dataset ID given the current
;SD interface ID, and the zero-based SD dataset index.
sds_id=HDF_SD_SELECT(sd_id,i)
IF (do_coordvar EQ 0) OR (NOT HDF_SD_ISCOORDVAR(sds_id)) THEN BEGIN
;The HDF_SD_GETINFO procedure retrieves information about an SD dataset.
HDF_SD_GETINFO,sds_id,NATTS=sds_natts, $ ;no. attributes
HDF_TYPE=sds_hdftype, $ ;HDF data type
TYPE=sds_type, $ ;data type
DIMS=sds_dim, $ ;dimension information (automatically reversed by IDL)
NAME=sds_name ;dataset name
;Check for string attribute and, if so, remove the first dimension
IF (sds_type EQ 'STRING') AND (N_ELEMENTS(sds_dim) NE 1) THEN sds_dim=sds_dim[1:N_ELEMENTS(sds_dim)-1]
;Check for multi-dimensional dataset
n_sds_dim=N_ELEMENTS(sds_dim)
IF n_sds_dim GT 1 THEN multi_dim=1
;Check number of dimensions
IF n_sds_dim gt 8 THEN INFOTXT_OUTPUT_A,[1],[sds_name,STRTRIM(n_sds_dim,2)]
;Check for coordinate variable
IF HDF_SD_ISCOORDVAR(sds_id) THEN INFOTXT_OUTPUT_A,[8],[sds_name]
;Check that a dataset name has been successfully read
sds_name=STRTRIM(sds_name,2)
IF sds_name EQ '' THEN sds_name='N/A'
;Rename data type to be compatible with the Metadata guidelines
sds_type=STRTRIM(STRUPCASE(sds_type),2)
IF sds_type EQ 'INT' THEN sds_type='SHORT' $
ELSE IF sds_type EQ 'LONG' THEN sds_type='INTEGER' $
ELSE IF sds_type EQ 'FLOAT' THEN sds_type='REAL'
hdftype[0]=sds_hdftype & idltype[0]=sds_type
IF sds_natts NE 0L THEN BEGIN
IF sds_natts GT max_atts THEN max_atts=sds_natts
vnf=0 & lcnt=0 & vcnt=sds_natts
;Extract the variable attributes
FOR j=0L,sds_natts-1L DO BEGIN
;The HDF_SD_ATTRINFO procedure reads or retrieves information about an SD attribute.
HDF_SD_ATTRINFO,sds_id,j,NAME=va_name,DATA=va_value,HDF_TYPE=va_hdftype,TYPE=va_type
va_name=STRTRIM(va_name,2)
schk=WHERE(STRUPCASE(va_name) EQ slabel,schkcnt)
;Rename data type to be compatible with the Metadata guidelines
va_type=STRTRIM(STRUPCASE(va_type),2)
IF va_type EQ 'INT' THEN va_type='SHORT' $
ELSE IF va_type EQ 'LONG' THEN va_type='INTEGER' $
ELSE IF va_type EQ 'FLOAT' THEN va_type='REAL'
IF ((sds_type EQ 'STRING') AND (schkcnt EQ 0)) OR ((sds_type NE 'STRING') AND (va_type EQ 'STRING')) THEN $
va_value=STRTRIM(va_value,2)
;Check returned dataset attributes (sds_) with saved attributes (va_)
IF (STRUPCASE(va_name) EQ 'VAR_NAME') AND (vnf EQ 0) THEN BEGIN
;check that the dataset name matches the VAR_NAME
vnf=1
IF STRUPCASE(va_value) NE STRUPCASE(sds_name) THEN BEGIN ;Try and determine which is wrong - Dataset name or VAR_NAME - default is Dataset name
li=WHERE((STRUPCASE(sds_name) EQ STRUPCASE(vn)) AND (vn[0] NE ''),lcnt)
;If lcnt NE 0 then Dataset name matches DATA_VARIABLES value so assume that VAR_NAME is wrong
IF lcnt NE 0 THEN usesds=1 ELSE usesds=0
INFOTXT_OUTPUT_A,[3,4,usesds],[sds_name,va_value]
IF usesds EQ 0 THEN sds_name=va_value ELSE va_value=sds_name
ENDIF
vnv=va_value
;match the order of the dataset to the DATA_VARIABLES list
li=WHERE(STRUPCASE(va_value) EQ STRUPCASE(vn),lcnt)
ENDIF
IF STRUPCASE(va_name) EQ 'VAR_DATA_TYPE' THEN idltype[n_lab-1]=STRUPCASE(STRTRIM(va_value,2)) $
ELSE IF STRUPCASE(va_name) EQ 'VAR_UNITS' THEN BEGIN
;check that VAR_UNITS is a string value (in event that VAR_UNITS=1)
IF va_type NE 'STRING' THEN $
INFOTXT_OUTPUT_A,[20],['VAR_UNITS',STRTRIM(sds_name,2)]
ENDIF ELSE IF STRUPCASE(va_name) EQ 'VAR_DEPEND' THEN $ ;do dimension ordering checks
TEST_DIM_ORDER,'VD',va_value,va_type,sds_dim,sds_name,rev_vd_vs $
ELSE IF STRUPCASE(va_name) EQ 'VAR_SIZE' THEN $ ;do dimension ordering checks
TEST_DIM_ORDER,'VS',va_value,va_type,sds_dim,sds_name,rev_vd_vs $
ELSE IF (STRUPCASE(va_name) EQ 'VAR_VALID_MIN') THEN BEGIN
hdftype[1]=va_hdftype & idltype[1]=va_type
ENDIF ELSE IF (STRUPCASE(va_name) EQ 'VAR_VALID_MAX') THEN BEGIN
hdftype[2]=va_hdftype & idltype[2]=va_type
ENDIF ELSE IF (STRUPCASE(va_name) EQ 'VAR_FILL_VALUE') THEN BEGIN
hdftype[3]=va_hdftype & idltype[3]=va_type
ENDIF
ENDFOR
ENDIF ELSE BEGIN ;No Variable Attributes found
vcnt=3
;Can the SDS_NAME be matched with a DATA_VARIABLES value?
li=WHERE(STRUPCASE(sds_name) EQ STRUPCASE(vn),lcnt)
IF lcnt NE 0 THEN vnf=1 ELSE vnf=0
INFOTXT_OUTPUT_A,[5],[sds_name,'HDF_SD_GETINFO']
ENDELSE
;Check for possible problems with determining array index
IF lcnt NE 0 THEN BEGIN
IF li[0] GE n_sds_hold THEN dv_order=0 $ ;can occur when number of DATA_VARIABLE values is greater than number of datasets
ELSE IF catinfo[li[0],0] NE '' THEN dv_order=0 ;This array has already been written to
ENDIF ELSE IF (lcnt EQ 0) OR (vnf EQ 0) THEN dv_order=0
;Write out information text if sds_name cannot be matched and determine array index
IF dv_order EQ 0 THEN BEGIN
li=WHERE(catinfo[*,0] EQ '') ;determine lowest available array index to write info to
in0=[4,vnf,lcnt,n_vn]
IF (vnf EQ 0) OR ((lcnt EQ 0) AND (n_vn NE 0L)) THEN INFOTXT_OUTPUT_A,in0,[sds_name,vnv]
ENDIF
oi[c_sds]=li[0] ;Attribute order index
c_sds=c_sds+1L
catinfo[li[0],0]=sds_name & catinfo[li[0],1]=sds_type
catinfo[li[0],2]=STRTRIM(sds_dim[0],2) & catinfo[li[0],3]=vcnt
IF N_ELEMENTS(sds_dim) GT 1 THEN $
FOR j=1,N_ELEMENTS(sds_dim)-1 DO catinfo[li[0],2]=catinfo[li[0],2]+';'+STRTRIM(sds_dim[j],2)
ENDIF
;Do data type checks on the Dataset, VAR_DATA_TYPE value and data types of VAR_VALID_MIN, VAR_VALID_MAX and VAR_FILL_VALUES
DATA_TYPE_CHECKS,ftype,dt_chk_labels,hdftype,idltype,sds_name,vnv
;Closes the SDS interface.
HDF_SD_ENDACCESS,sds_id
ENDFOR
IF max_atts EQ 0L THEN max_atts=3 ;Use information from HDF_SD_GETINFO call only
rev_vd_vsh=rev_vd_vs ;keep original value in hold variable (required if rev_vd_vs eq 3)
IF (rev_vd_vs EQ 2) OR (rev_vd_vs EQ 3) THEN rev_vd_vs=1 ;change default so that VAR_DEPEND and VAR_SIZE
;are reversed if not o/w changed in TEST_DIM_ORDER
;Dimension the structure to the number of datasets x number of attributes sds=REPLICATE(sds_set, n_sds_hold, max_atts)
;Write attributes to structure
c_sds=0L
!QUIET=1 ;suppress system error and information messages (used for CALDATA call in HDF_SD_GETINFO)
FOR i=0L,n_sds-1L DO BEGIN
notranspose=1 ;0/1 will change to 0 if the dataset needs to be transposed
;The HDF_SD_SELECT function returns an SD dataset ID given the current
;SD interface ID, and the zero-based SD dataset index.
sds_id=HDF_SD_SELECT(sd_id,i)
nocal=1 ;Boolean to identify dataset which has been calibrated
IF (do_coordvar EQ 0) OR (NOT HDF_SD_ISCOORDVAR(sds_id)) THEN BEGIN
;The HDF_SD_GETINFO procedure retrieves information about an SD dataset.
;Note - any saved pre-defined attributes will be called with HDF_SD_ATTRINFO call
HDF_SD_GETINFO,sds_id,NATTS=sds_natts, $ ;no. attributes
HDF_TYPE=sds_hdftype, $ ;HDF data type
TYPE=sds_type, $ ;data type
DIMS=sds_dim, $ ;dimension information
NAME=sds_name, $ ;dataset name
CALDATA=sds_cal ;pre-defined calibration info
;Check to see whether data has had scale factor and offset applied
IF (sds_cal.Cal NE 0.D) OR (sds_cal.Offset NE 0.D) THEN BEGIN
nocal=0 ;Dataset has been calibrated
;identify datatype of the original dataset
CASE 1 OF
sds_cal.Num_Type EQ 21L: vdt_val='BYTE'
sds_cal.Num_Type EQ 22L: vdt_val='SHORT'
sds_cal.Num_Type EQ 24L: vdt_val='INTEGER'
sds_cal.Num_Type EQ 5L: vdt_val='REAL'
sds_cal.Num_Type EQ 6L: vdt_val='DOUBLE'
ELSE: vdt_val='' ;invalid or cannot determine original datatype
ENDCASE
ENDIF
IF sds_natts NE 0L THEN BEGIN
;Extract the variable attributes
jh=0 ;will only loop if attribute is not a pre-defined calibration attribute
FOR j=0L,sds_natts-1L DO BEGIN
;The HDF_SD_ATTRINFO procedure reads or retrieves information about an SD attribute.
HDF_SD_ATTRINFO,sds_id,j,NAME=va_name,DATA=va_value,TYPE=va_type
va_name=STRTRIM(va_name,2)
schk=WHERE(STRUPCASE(va_name) EQ slabel,schkcnt)
IF ((sds_type EQ 'STRING') AND (schkcnt EQ 0)) OR ((sds_type NE 'STRING') AND (va_type EQ 'STRING')) THEN $
va_value=STRTRIM(va_value,2)
IF (nocal EQ 0) AND (STRUPCASE(va_name) EQ 'VAR_DATA_TYPE') THEN BEGIN
;If required prepare INFORMATION text (written after DATA has been corrected)
IF (vdt_val NE STRUPCASE(va_value)) AND (vdt_val NE '') THEN BEGIN
io_arr=[STRTRIM(sds_name,2),va_value,vdt_val]
va_value=vdt_val
ENDIF ELSE BEGIN
io_arr=[''] & vdt_val=va_value
ENDELSE
ENDIF
IF ((STRUPCASE(va_name) EQ 'VAR_DEPEND') OR (STRUPCASE(va_name) EQ 'VAR_SIZE')) AND $
((rev_vd_vs MOD 10 EQ 1) OR (rev_vd_vs EQ 20)) THEN BEGIN ;need to reverse the va_values
IF va_type NE 'STRING' THEN BEGIN ;VAR_SIZE contains numeric values instead of in the form of a string
vavhold='' & n_vav=N_ELEMENTS(va_value)
FOR k=0,n_vav-1 DO BEGIN
IF k EQ n_vav-1 THEN vtxt='' ELSE vtxt=';'
vavhold=vavhold+STRTRIM(va_value[k],2)+vtxt
ENDFOR
ENDIF ELSE vavhold=va_value
vs_v=STRCOMPRESS(STRSPLIT(vavhold,';, ',/EXTRACT,COUNT=rcnt),/REMOVE_ALL)
IF rcnt GT 1 THEN BEGIN ;multi-dimensions
IF (rev_vd_vs EQ 20) THEN notranspose=0 ;need to transpose dataset as well vs_v=REVERSE(vs_v)
FOR k=0,rcnt-1 DO $
IF k EQ 0 THEN va_value=vs_v[k] ELSE va_value=va_value+';'+vs_v[k]
ENDIF
ENDIF
;Test for pre-defined calibration attribute and do not save as the data have
;been coverted back to original values
pi=WHERE(STRLOWCASE(va_name) EQ ncsa_cal,pcnt)
IF pcnt EQ 0 THEN BEGIN
sds[oi[c_sds],jh].va_l=PTR_NEW(va_name)
sds[oi[c_sds],jh].va_v=PTR_NEW(va_value)
jh=jh+1
ENDIF
ENDFOR
ENDIF ELSE BEGIN ;No Variable Attributes found
va_lab=['VAR_NAME','VAR_SIZE','VAR_DATA_TYPE']
FOR j=0,2 DO sds[oi[c_sds],j].va_l=PTR_NEW(va_lab[j])
sds[oi[c_sds],0].va_v=PTR_NEW(sds_name)
sds[oi[c_sds],1].va_v=PTR_NEW(sds_dim)
;Rename format to be compatible with the Metadata guidelines
IF sds_type EQ 'INT' THEN sds_type='SHORT' $
ELSE IF sds_type EQ 'LONG' THEN sds_type='INTEGER' $
ELSE IF sds_type EQ 'FLOAT' THEN sds_type='REAL'
sds[oi[c_sds],2].va_v=PTR_NEW(sds_type)
ENDELSE
;Extract the data
HDF_SD_GETDATA,sds_id,datasize
;Test to see if dataset dimension ordering needs to be changed
IF notranspose EQ 0 THEN datasize=TRANSPOSE(datasize)
IF nocal EQ 0 THEN BEGIN
;apply scale factor and offset corrections to data and write text to log file
CASE 1 OF
vdt_val EQ 'BYTE': datasize=BYTE(sds_cal.Cal*(datasize-sds_cal.Offset))
vdt_val EQ 'SHORT': datasize=FIX(sds_cal.Cal*(datasize-sds_cal.Offset))
vdt_val EQ 'INTEGER': datasize=LONG(sds_cal.Cal*(datasize-sds_cal.Offset))
vdt_val EQ 'REAL': datasize=FLOAT(sds_cal.Cal*(datasize-sds_cal.Offset))
ELSE: datasize=DOUBLE(sds_cal.Cal*(datasize-sds_cal.Offset))
ENDCASE
INFOTXT_OUTPUT_A,[9],[STRTRIM(sds_name,2)]
IF io_arr[0] NE '' THEN INFOTXT_OUTPUT_A,[10],io_arr
ENDIF
sds[oi[c_sds],0].data=PTR_NEW(datasize)
c_sds=c_sds+1L
ENDIF
;Closes the SDS interface.
HDF_SD_ENDACCESS,sds_id
ENDFOR
!QUIET=0 ;Allow system error and information messages
;The HDF_SD_END function closes the SD interface to an HDF file.
HDF_SD_END,sd_id
ENDIF ELSE IF (ftype EQ 'H5') OR (ftype EQ 'N4') THEN BEGIN ;HDF5 or netCDF4 format file
stdfm=1 ;Boolean indicating standard AVDC/EVDC/NDACC H5 format or not
dv_order=1 ;Boolean indicating whether Metadata Variable listing order matches DATA_VARIABLES values
n_vn=0L ;Initialize the number of Variable Names in the standard format h5 file
;labels to quietly ignore if the file is netCDF4
nc4_ignore=['reference_list','dimension_list','class','name','_netcdf4dimid','_netcdf4coordinates','_ncproperties']
;Do bulk read so errors generated by netCDF4 inconsistencies can be identified without
;crashing the program e.g. null string attribute values
h5p=H5_PARSE(ifile)
hdf_file_id=H5F_OPEN(ifile)
sd_id=H5G_OPEN(hdf_file_id,'/')
;Get number of attributes within the root group (these should be the Global Attributes)
n_ga=H5A_GET_NUM_ATTRS(sd_id) ;Get the number of objects within the root group (including datasets, groups etc)
n_obj=H5G_GET_NMEMBERS(sd_id,'/')
IF n_obj NE 0L THEN BEGIN
d_obj=INTARR(n_obj) & g_obj=INTARR(n_obj)
;Determine number of datasets. Note: if H5 file created using H4toH5 then an extra
;group will have been created which will not be used. Also datasets may be listed
;alphabetically. This section also determines whether the H5 file is a standard
;AVDC/EVDC/NDACC groundbased file or otherwise.
FOR i=0L,n_obj-1L DO BEGIN
sds_name=H5G_GET_MEMBER_NAME(sd_id,'/',i)
h5fstat=H5G_GET_OBJINFO(sd_id,sds_name)
IF h5fstat.type EQ 'DATASET' THEN d_obj[i]=1 $
ELSE IF h5fstat.type EQ 'GROUP' THEN g_obj[i]=1
ENDFOR
di=WHERE(d_obj EQ 1,n_sds)
gi=WHERE(g_obj EQ 1,n_grp)
IF (n_ga EQ 0) AND (n_sds EQ 0) AND (n_grp NE 0) THEN stdfm=0
ENDIF ELSE n_sds=0L
IF stdfm EQ 0 THEN catinfo[0,0]='HE5' ;non-standard dataset so need to call READ_NONSTD_H5
IF n_ga NE 0 THEN BEGIN
ga=STRARR(n_ga) ;set the Global Attribute dimensions
vn=[''] ;initialize vn array (to hold Dataset names)
;Read in the Global Attribute labels and values
FOR i=0L,n_ga-1L DO BEGIN
ga_id=H5A_OPEN_IDX(sd_id,i)
ga_name=H5A_GET_NAME(ga_id) & ga_name=STRTRIM(ga_name,2)
;Check for netCDF4 specific attributes and ignore
nc4i=WHERE(STRLOWCASE(ga_name) EQ nc4_ignore,nc4icnt)
IF nc4icnt EQ 0 THEN BEGIN
;test for valid attribute value - can be invalid if it is netCDF4 and equal to ''
gni=WHERE(TAG_NAMES(h5p) EQ STRUPCASE(ga_name))
attest=h5p.(gni[0])
IF STRTRIM(attest._data[0],2) EQ '<read error>' THEN ga_data='' $
ELSE BEGIN
dt_id=H5A_GET_TYPE(ga_id)
ga_data=H5A_READ(ga_id) & ga_data=STRTRIM(ga_data,2)
ga_datatype=H5T_GET_CLASS(dt_id)
H5T_CLOSE,dt_id
;String labels for GA not checked at this time as extra Global Attributes could be numeric
;IF ga_datatype NE 'H5T_STRING' THEN $
; INFOTXT_OUTPUT_A,[20],['Global Attribute',STRTRIM(ga_name,2)]
ENDELSE
;Check for _GLOSDS suffix and strip (added for compatibility with H4toH5 utility)
IF STRMID(STRUPCASE(ga_name),STRLEN(ga_name)-7) EQ '_GLOSDS' THEN $
ga_name=STRMID(ga_name,0,STRPOS(ga_name,'_',/Reverse_Search))
WHILE STRMID(ga_name,STRLEN(ga_name)-1) EQ '_' DO ga_name=STRMID(ga_name,0,STRLEN(ga_name)-1)
ga[i]=ga_name+'='+ga_data
ENDIF
H5A_CLOSE,ga_id
;Extract the list of the DATA_VARIABLES
IF STRUPCASE(ga_name) EQ 'DATA_VARIABLES' THEN vn=STRSPLIT(ga_data,' ;',/Extract,COUNT=n_vn)
IF STRUPCASE(ga_name) EQ 'FILE_NAME' THEN BEGIN
;check that the actual file name matches the FILE_NAME entry
IF STRLOWCASE(STRTRIM(ga_data,2)) NE ifilechk THEN INFOTXT_OUTPUT_A,[24],[ga_data]
ENDIF
ENDFOR
;recalculate n_ga and ga in case of netCDF4 name which is ignored
gai=WHERE(ga NE '',n_ga)
IF n_ga NE 0 THEN ga=ga[gai]
ENDIF ELSE IF stdfm EQ 1 THEN BEGIN
STOP_WITH_ERROR_A,o3[3]+proname,'No Global Attributes'+errtxt[0],lu
RETURN
ENDIF
IF n_sds NE 0L THEN BEGIN
NC_DIMENSION_CHK,'H5',sd_id,di,n_sds,dgi,n_sdsg ;check for netCDF dimension names (will quietly ignore)
di=dgi & n_sds=n_sdsg IF n_vn NE n_sds THEN $
INFOTXT_OUTPUT_A,[0],[STRTRIM(n_vn,2),STRTRIM(n_sds,2),'H5G_GET_NMEMBERS']
;DATA_VARIABLES values not read successfully, or number of datasets given under DATA_VARIABLES
;is not equal to the number saved to the file
;Determine maximum number of Attributes, correct dataset order, and dimension order
max_atts=0L & oi=LONARR(n_sds)
dv_order=1
catinfo=STRARR(n_sds,4) ;output info for catalog output
FOR i=0,n_sds-1 DO BEGIN
vnv='' & sds_name=''
hdftype=STRARR(n_lab-1) & idltype=STRARR(n_lab) ;to hold data types for dataset, min, max and fill values, and VAR_DATA_TYPE (idltype only)
sds_name=H5G_GET_MEMBER_NAME(sd_id,'/',di[i])
ds_id=H5D_OPEN(sd_id,sds_name)
;need to swap non-alphanumeric characters to '_' to match names from h5_parse for attribute values test
h5p_name=ALPHA_NUMERIC_UNDERSCORE(STRUPCASE(sds_name))
IF sds_name EQ '' THEN sds_name='N/A'
;determine dataset datatype
dt_id=H5D_GET_TYPE(ds_id)
dt_datatype=H5T_GET_CLASS(dt_id)
IF dt_datatype EQ 'H5T_INTEGER' THEN BEGIN
dt_sign=H5T_GET_SIGN(dt_id) ;0=unsigned; 1=signed
dt_datatype=dt_datatype+'_'+STRTRIM(dt_sign,2)
;Byte needs to be unsigned, Short and Integer are signed
ENDIF
H5T_CLOSE,dt_id
datasize=H5D_READ(ds_id)
sds_dim=SIZE(datasize,/DIMENSIONS) & n_sds_dim=N_ELEMENTS(sds_dim)
FOR sdl=0,n_sds_dim-1 DO IF sds_dim[sdl] EQ 0 THEN sds_dim[sdl]=1 ;constant scalar h5
IF (ftype EQ 'N4') AND (dt_datatype EQ 'H5T_STRING') THEN BEGIN
;need to do char_dim checks for STRING datasets and remove the string length dimension
char_dim=sds_dim[0]
IF n_sds_dim EQ 1 THEN sds_dim[0]=1 $
ELSE BEGIN
sds_dim=sds_dim[1:n_sds_dim-1] & n_sds_dim=n_sds_dim-1
ENDELSE
dschk=STRARR(sds_dim)
FOR sdl=0L,char_dim-1L DO dschk=dschk+datasize[sdl,*]
datasize=dschk
ENDIF
sds_type=SIZE(datasize,/TYPE)
hdftype[0]=dt_datatype & idltype[0]=STRTRIM(sds_type,2)
;Determine the number of attributes associated with the dataset
sds_natts=H5A_GET_NUM_ATTRS(ds_id)
IF sds_natts GT max_atts THEN max_atts=sds_natts
IF sds_natts NE 0L THEN BEGIN
vnf=0 & lcnt=0
ci=STRARR(4) ;array containing catalog attributes
;Read in Dataset Attributes
jj=0 ;use this index in the event of invalid attribute names (e.g. dimension_list from netCDF4 file)
FOR j=0L,sds_natts-1L DO BEGIN
da_id=H5A_OPEN_IDX(ds_id,j)
da_name=H5A_GET_NAME(da_id) & da_name=STRTRIM(da_name,2)
;determine attribute datatype
at_id=H5A_GET_TYPE(da_id)
at_datatype=H5T_GET_CLASS(at_id)
IF at_datatype EQ 'H5T_INTEGER' THEN BEGIN
at_sign=H5T_GET_SIGN(at_id) ;0=unsigned; 1=signed
at_datatype=at_datatype+'_'+STRTRIM(at_sign,2)
;Byte needs to be unsigned, Short and Integer are signed
ENDIF
H5T_CLOSE,at_id
nc4i=WHERE(STRLOWCASE(da_name) EQ nc4_ignore,nc4icnt)
IF nc4icnt EQ 0 THEN BEGIN
;test for valid attribute value - can be invalid if it is netCDF4 and equal to ''
gni=WHERE(TAG_NAMES(h5p) EQ STRUPCASE(h5p_name)) ;Tag index from h5_parse read
attest=h5p.(gni[0])
gni=WHERE(TAG_NAMES(attest) EQ STRUPCASE(da_name))
attestx=attest.(gni[0])
IF STRTRIM(attestx._data[0],2) EQ '<read error>' THEN va_value='' $
ELSE va_value=H5A_READ(da_id)
IF SIZE(va_value,/TYPE) EQ 7 THEN va_type='STRING' ELSE va_type='NON_STRING'
idltypehold=STRTRIM(SIZE(va_value,/TYPE),2)
schk=WHERE(STRUPCASE(da_name) EQ slabel,schkcnt)
IF ((sds_type EQ 7) AND (schkcnt EQ 0)) OR ((sds_type NE 7) AND (va_type EQ 'STRING')) THEN $
va_value=STRTRIM(va_value,2)
CASE 1 OF
(STRUPCASE(da_name) EQ 'VAR_NAME') AND (vnf EQ 0):BEGIN
vnf=1
;check that the dataset name matches the VAR_NAME
IF STRUPCASE(va_value) NE STRUPCASE(sds_name) THEN BEGIN
test1=ftype EQ 'N4'
test2=STRUPCASE(ALPHA_NUMERIC_UNDERSCORE(va_value)) EQ STRUPCASE(sds_name)
;For nc4 files check if '.'s have been replaced by '_'s in the SDS_NAME. If so then it is OK
;and quietly change SDS_NAME to the VAR_NAME value
IF (test1) AND (test2) THEN sds_name=va_value $
ELSE BEGIN
;Try and determine which is wrong - Dataset name or VAR_NAME - default is Dataset name
li=WHERE((STRUPCASE(sds_name) EQ STRUPCASE(vn)) AND (vn[0] NE ''),lcnt)
;If lcnt NE 0 then Dataset name matches DATA_VARIABLES value so assume that VAR_NAME is wrong
IF lcnt NE 0 THEN usesds=1B ELSE usesds=0B
INFOTXT_OUTPUT_A,[3,5,usesds],[sds_name,va_value]
IF usesds THEN va_value=sds_name else sds_name=va_value
ENDELSE
ENDIF
;Determine actual list order from DATA_VARIABLE listing (H5 lists datasets alphabetically)
li=WHERE(STRUPCASE(va_value[0]) EQ STRUPCASE(vn),lcnt)
ci[0]=va_value & vnv=va_value
END
STRUPCASE(da_name) EQ 'VAR_DATA_TYPE': BEGIN
ci[1]=va_value & idltype[n_lab-1]=STRUPCASE(STRTRIM(va_value,2))
END
STRUPCASE(da_name) EQ 'VAR_UNITS': BEGIN
;check that VAR_UNITS is a string value (in event that VAR_UNITS=1)
IF va_type NE 'STRING' THEN $
INFOTXT_OUTPUT_A,[20],['VAR_UNITS',STRTRIM(sds_name,2)]
END
STRUPCASE(da_name) EQ 'VAR_DEPEND': BEGIN
TEST_DIM_ORDER,'VD',va_value,va_type,sds_dim,sds_name,rev_vd_vs
END
STRUPCASE(da_name) EQ 'VAR_SIZE': BEGIN
TEST_DIM_ORDER,'VS',va_value,va_type,sds_dim,sds_name,rev_vd_vs
END
STRUPCASE(da_name) EQ 'VAR_VALID_MIN': BEGIN
hdftype[1]=at_datatype & idltype[1]=idltypehold
END
STRUPCASE(da_name) EQ 'VAR_VALID_MAX': BEGIN
hdftype[2]=at_datatype & idltype[2]=idltypehold
END
STRUPCASE(da_name) EQ 'VAR_FILL_VALUE': BEGIN
hdftype[3]=at_datatype & idltype[3]=idltypehold
END
ELSE:
ENDCASE
jj++
ENDIF ;ELSE IF nc4i[0] EQ 4 THEN catinfo[0,0]='H5N' ;netCDF4 file - not implemented yet
H5A_CLOSE,da_id
ENDFOR
ci[3]=STRTRIM(jj,2)
IF ci[0] EQ '' THEN ci[0]=sds_name
ENDIF ELSE BEGIN ;No Attributes, so can only write dataset name to Metadata ci=[STRUPCASE(sds_name),'','','1'] & vnv=sds_name
;Can the SDS_NAME be matched with a DATA_VARIABLES value?
li=WHERE(STRUPCASE(sds_name) EQ STRUPCASE(vn),lcnt)
IF lcnt NE 0 THEN vnf=1 ELSE vnf=0
INFOTXT_OUTPUT_A,[5],[sds_name,'H5A_GET_NUM_ATTRS']
ENDELSE
FOR k=0,N_ELEMENTS(sds_dim)-1 DO BEGIN
IF k EQ 0 THEN ci[2]=STRTRIM(sds_dim[k],2) ELSE ci[2]=ci[2]+';'+STRTRIM(sds_dim[k],2)
ENDFOR
;Check for possible problems with determining array index
IF lcnt NE 0 THEN BEGIN
IF li[0] GE n_sds THEN dv_order=0 $ ;can occur when number of DATA_VARIABLE values is greater than number of datasets
ELSE IF catinfo[li[0],0] NE '' THEN dv_order=0 ;This array has already been written to
ENDIF ELSE IF (lcnt EQ 0) OR (vnf EQ 0) THEN dv_order=0
;Write out information text if sds_name cannot be matched and determine array index
IF dv_order EQ 0 THEN BEGIN
li=WHERE(catinfo[*,0] EQ '') ;determine lowest available array index to write info to
in0=[4,vnf,lcnt,n_vn]
IF (vnf EQ 0) OR ((lcnt EQ 0) AND (n_vn NE 0L)) THEN INFOTXT_OUTPUT_A,in0,[sds_name,vnv]
ENDIF
H5D_CLOSE,ds_id
oi[i]=li[0] ;to put datasets in the correct order
catinfo[li[0],*]=ci[*]
;Do data type checks on the Dataset, VAR_DATA_TYPE value and data types of VAR_VALID_MIN, VAR_VALID_MAX and VAR_FILL_VALUES
DATA_TYPE_CHECKS,ftype,dt_chk_labels,hdftype,idltype,sds_name,vnv
ENDFOR
ENDIF ELSE IF stdfm EQ 1 THEN BEGIN
STOP_WITH_ERROR_A,o3[3]+proname,'No Datasets'+errtxt[0],lu & RETURN
ENDIF
rev_vd_vsh=rev_vd_vs ;keep original value in hold variable (required if rev_vd_vs eq 3)
IF (rev_vd_vs EQ 2) OR (rev_vd_vs EQ 3) THEN rev_vd_vs=1 ;change default so that VAR_DEPEND and VAR_SIZE
;are reversed if not o/w changed in TEST_DIM_ORDER
;Dimension the structure to the size of the SDS datasets (with dimension n_sds)
sds=REPLICATE(sds_set, n_sds, max_atts)
;Put datasets and attributes into sds structure
FOR i=0,n_sds-1 DO BEGIN
notranspose=1 ;0/1 will change to 0 if the dataset needs to be transposed
sds_name=H5G_GET_MEMBER_NAME(sd_id,'/',di[i])
ds_id=H5D_OPEN(sd_id,sds_name)
;need to swap non-alphanumeric characters to '_' to match names from h5_parse for attribute values test
h5p_name=ALPHA_NUMERIC_UNDERSCORE(STRUPCASE(sds_name))
IF sds_name EQ '' THEN sds_name='N/A'
datasize=H5D_READ(ds_id)
sds_type=SIZE(datasize,/TYPE)
sds_dim=SIZE(datasize,/DIMENSIONS) & n_sds_dim=N_ELEMENTS(sds_dim)
FOR sdl=0,n_sds_dim-1 DO IF sds_dim[sdl] EQ 0 THEN sds_dim[sdl]=1 ;constant scalar h5
IF (ftype EQ 'N4') AND (sds_type EQ 7) THEN BEGIN
;need to do char_dim checks for STRING datasets and remove the string length dimension
char_dim=sds_dim[0]
IF n_sds_dim EQ 1 THEN sds_dim[0]=1 $
ELSE BEGIN
sds_dim=sds_dim[1:n_sds_dim-1] & n_sds_dim=n_sds_dim-1
ENDELSE
dschk=STRARR(sds_dim)
FOR sdl=0L,char_dim-1L DO dschk=dschk+datasize[sdl,*]
datasize=dschk
ENDIF
;Determine the number of attributes associated with the dataset
sds_natts=H5A_GET_NUM_ATTRS(ds_id)
IF sds_natts NE 0L THEN BEGIN
;Read in Dataset Attributes jj=0 ;use this index in the event of invalid attribute names (e.g. dimension_list fro netCDF4 file)
FOR j=0L,sds_natts-1L DO BEGIN
da_id=H5A_OPEN_IDX(ds_id,j)
da_name=H5A_GET_NAME(da_id) & da_name=STRTRIM(da_name,2)
nc4i=WHERE(STRLOWCASE(da_name) EQ nc4_ignore,nc4icnt)
IF nc4icnt EQ 0 THEN BEGIN
;test for valid attribute value - can be invalid if it is netCDF4 and equal to ''
gni=WHERE(TAG_NAMES(h5p) EQ STRUPCASE(h5p_name)) ;Tag index from h5_parse read
attest=h5p.(gni[0])
gni=WHERE(TAG_NAMES(attest) EQ STRUPCASE(da_name))
attestx=attest.(gni[0])
IF STRTRIM(attestx._data[0],2) EQ '<read error>' THEN va_value='' $
ELSE va_value=H5A_READ(da_id)
IF SIZE(va_value,/TYPE) EQ 7 THEN va_type='STRING' ELSE va_type='NON_STRING'
schk=WHERE(STRUPCASE(da_name) EQ slabel,schkcnt)
IF ((sds_type EQ 7) AND (schkcnt EQ 0)) OR ((sds_type NE 7) AND (va_type EQ 'STRING')) THEN $
va_value=STRTRIM(va_value,2)
H5A_CLOSE,da_id
IF ((STRUPCASE(da_name) EQ 'VAR_DEPEND') OR (STRUPCASE(da_name) EQ 'VAR_SIZE')) AND $
((rev_vd_vs MOD 10 EQ 1) OR (rev_vd_vs EQ 20)) THEN BEGIN ;need to reverse the va_values
IF va_type NE 'STRING' THEN BEGIN ;VAR_SIZE contains numeric values instead of in the form of a string
vavhold='' & n_vav=N_ELEMENTS(va_value)
FOR k=0,n_vav-1 DO BEGIN
IF k EQ n_vav-1 THEN vtxt='' ELSE vtxt=';'
vavhold=vavhold+STRTRIM(va_value[k],2)+vtxt
ENDFOR
ENDIF ELSE vavhold=va_value
vs_v=STRCOMPRESS(STRSPLIT(vavhold,';',/EXTRACT,COUNT=rcnt),/REMOVE_ALL)
IF rcnt GT 1 THEN BEGIN ;multi-dimensions
IF (rev_vd_vs EQ 20) THEN $ ;AND (STRPOS(STRUPCASE(va_value),'DATETIME') NE -1) THEN $
notranspose=0 ;need to transpose dataset as well
vs_v=REVERSE(vs_v)
FOR k=0,rcnt-1 DO $
IF k EQ 0 THEN va_value=vs_v[k] ELSE va_value=va_value+';'+vs_v[k]
ENDIF
ENDIF
sds[oi[i],jj].va_l=PTR_NEW(da_name)
sds[oi[i],jj].va_v=PTR_NEW(va_value)
jj++
ENDIF
ENDFOR
ENDIF ELSE BEGIN ;No Attributes, so can only write dataset name to structure
val='VAR_NAME'
sds[oi[i],0].va_l=PTR_NEW(val)
sds[oi[i],0].va_v=PTR_NEW(sds_name)
ENDELSE
H5D_CLOSE,ds_id
;Test to see if dataset dimension ordering needs to be changed
IF notranspose EQ 0 THEN datasize=TRANSPOSE(datasize)
sds[oi[i],0].data=PTR_NEW(datasize)
ENDFOR
H5G_CLOSE,sd_id
H5F_CLOSE,hdf_file_id
ENDIF ELSE BEGIN ;netCDF3 file (N3)
ncdf_desc=['long_name']
ncdf_std=['units','valid_range','_fillvalue']
ncdf_dimen=['constant','independent']
dv_order=1 ;Boolean indicating whether Metadata Variable listing order matches DATA_VARIABLES values
n_vn=0L ;Initialize the number of Variable Names in the standard format netCDF file
fileid=NCDF_OPEN(ifile)
;Get number of global atts and variables
filestruct=NCDF_INQUIRE(fileid) n_dim=filestruct.ndims
n_ga=filestruct.ngatts
n_sds=filestruct.nvars
IF n_sds EQ 0L THEN ntxt='Number of SDS datasets' $
ELSE IF n_ga EQ 0L THEN ntxt='Number of Global Attributes' $
ELSE ntxt=''
IF ntxt NE '' THEN BEGIN
STOP_WITH_ERROR_A,o3[3]+proname,ntxt+errtxt[1],lu & RETURN
ENDIF
;read global attributes
ga=STRARR(n_ga) & vn=['']
FOR j=0,n_ga-1 DO BEGIN
ga_name=NCDF_ATTNAME(fileid,j,/GLOBAL)
NCDF_ATTGET,fileid,ga_name,ga_data,/GLOBAL
ga_data=STRTRIM(ga_data,2)
ga[j]=STRTRIM(STRUPCASE(ga_name),2)+'='+ga_data
;read list of DATA_VARIABLES into array
IF STRUPCASE(ga_name) EQ 'DATA_VARIABLES' THEN vn=STRSPLIT(ga_data,' ;',/Extract,COUNT=n_vn)
IF STRUPCASE(ga_name) EQ 'FILE_NAME' THEN BEGIN
;check that the actual file name matches the FILE_NAME entry
IF STRLOWCASE(STRTRIM(ga_data,2)) NE ifilechk THEN INFOTXT_OUTPUT_A,[24],[ga_data]
ENDIF
ENDFOR
;read dimension information
IF n_dim NE 0 THEN BEGIN
mv_dim=STRARR(n_dim,2)
FOR j=0,n_dim-1 DO BEGIN
NCDF_DIMINQ,fileid,j,dimname,dimsize
mv_dim[j,0]=STRTRIM(dimname,2)
mv_dim[j,1]=STRTRIM(dimsize,2)
ENDFOR
ENDIF ELSE mv_dim=['']
di=INDGEN(n_sds)
NC_DIMENSION_CHK,'N3',fileid,di,n_sds,dgi,n_sdsg
di=dgi & n_sds=n_sdsg ;di contains index values of valid sds datasets
IF n_vn NE n_sds THEN $
INFOTXT_OUTPUT_A,[0],[STRTRIM(n_vn,2),STRTRIM(n_sds,2),'NCDF_INQUIRE']
;DATA_VARIABLES values not read successfully, or number of datasets given under DATA_VARIABLES
;is not equal to the number saved to the file
;Determine maximum number of Attributes, correct dataset order, and dimension order
max_atts=0L & oi=LONARR(n_sds)
catinfo=STRARR(n_sds,4) ;output info for catalog output
FOR j=0,n_sds-1 DO BEGIN
vnv='' & sds_name=''
hdftype=STRARR(n_lab-1) & idltype=STRARR(n_lab) ;to hold data types for dataset, min, max and fill values, and VAR_DATA_TYPE (idltype only)
;Extract data from a netCDF dataset and get size information
NCDF_VARGET,fileid,di[j],sds_data
sds_type=SIZE(sds_data,/TYPE)
varstruct=NCDF_VARINQ(fileid,di[j])
sds_natts=varstruct.natts
IF sds_natts GT max_atts THEN max_atts=sds_natts
sds_name=varstruct.name
sds_datatype=varstruct.datatype ;'BYTE', 'CHAR', 'INT', 'LONG', 'FLOAT', or 'DOUBLE'
sds_dimid=varstruct.dim
sds_ndim=varstruct.ndims
;determine the actual dimension values from the dim IDs given in varstruct.dim
;IF sds_ndim NE 0 THEN BEGIN
; sds_dim=LONARR(sds_ndim)
; FOR k=0,sds_ndim-1 DO sds_dim[k]=mv_dim[sds_dimid[k],1]
;ENDIF ELSE sds_dim=[1L]
IF (sds_datatype EQ 'CHAR') AND (sds_type EQ 1) THEN sds_type=7
sds_dim=SIZE(sds_data,/DIMENSIONS) & n_sds_dim=N_ELEMENTS(sds_dim)
FOR sdl=0,n_sds_dim-1 DO IF sds_dim[sdl] EQ 0 THEN sds_dim[sdl]=1 ;constant scalar h5
IF sds_type EQ 7 THEN BEGIN
;need to do char_dim checks for STRING datasets and remove the string length dimension
char_dim=sds_dim[0]
IF n_sds_dim EQ 1 THEN sds_dim[0]=1 $
ELSE BEGIN
sds_dim=sds_dim[1:n_sds_dim-1] & n_sds_dim=n_sds_dim-1
ENDELSE
ENDIF
IF sds_datatype EQ 'BYTE' THEN $
IF MIN(sds_data) LT 0 THEN sds_datatype='BYTE_1' ELSE sds_datatype='BYTE_0'
;This doesn't work as NCDF_VARGET converts BYTE values to 8-bit unsigned if they are negative signed values
hdftype[0]=sds_datatype & idltype[0]=STRTRIM(sds_type,2)
;Read Variable Attributes
IF sds_natts NE 0L THEN BEGIN
vnf=0 & lcnt=0 & vnv=''
ci=STRARR(4) ;array containing catalog attributes
std_fnd=INTARR(n_aad)
;Read in Dataset Attributes
FOR k=0,sds_natts-1 DO BEGIN
varattsname=NCDF_ATTNAME(fileid,di[j],k)
NCDF_ATTGET,fileid,di[j],varattsname,va_value
attstruct=NCDF_ATTINQ(fileid,di[j],varattsname)
va_name=STRUPCASE(varattsname)
vavtype=SIZE(va_value,/TYPE)
IF (vavtype EQ 7) OR (vavtype EQ 1) THEN va_type='STRING' ELSE va_type='NON-STRING'
hdftypehold=attstruct.datatype
IF hdftypehold EQ 'BYTE' THEN $
IF va_value LT 0 THEN hdftypehold='BYTE_1' ELSE hdftypehold='BYTE_0'
;This doesn't work as NCDF_VARGET converts BYTE values to 8-bit unsigned if they are negative signed values
idltypehold=vavtype
schk=WHERE(va_name EQ slabel,schkcnt)
test1=(sds_datatype EQ 'CHAR') AND (schkcnt EQ 0) ;string dataset but not one of the slabel values
test2=(sds_datatype NE 'CHAR') AND (va_type EQ 'STRING') ;not a string dataset but string attribute value
test3=(sds_datatype EQ 'CHAR') AND (schkcnt NE 0) AND (vavtype EQ 1) ;string dataset, slabel and attribute value is of type byte
IF (test1) OR (test2) THEN va_value=STRTRIM(va_value,2) $
ELSE IF test3 THEN BEGIN
va_value=STRING(va_value) & idltypehold=7
ENDIF
CASE 1 OF
(STRUPCASE(va_name) EQ 'VAR_NAME') AND (vnf EQ 0):BEGIN
vnf=1
;check that the dataset name matches the VAR_NAME
IF STRUPCASE(va_value) NE STRUPCASE(sds_name) THEN BEGIN
test1=ftype EQ 'N3'
test2=STRUPCASE(ALPHA_NUMERIC_UNDERSCORE(va_value)) EQ STRUPCASE(sds_name)
;For nc3 files check if '.'s have been replaced by '_'s in the SDS_NAME. If so then it is OK
;and quietly change SDS_NAME to the VAR_NAME value
IF (test1) AND (test2) THEN sds_name=va_value $
ELSE BEGIN
;Try and determine which is wrong - Dataset name or VAR_NAME - default is Dataset name
li=WHERE((STRUPCASE(sds_name) EQ STRUPCASE(vn)) AND (vn[0] NE ''),lcnt)
;If lcnt NE 0 then Dataset name matches DATA_VARIABLES value so assume that VAR_NAME is wrong
IF lcnt NE 0 THEN usesds=1 ELSE usesds=0
INFOTXT_OUTPUT_A,[3,5,usesds],[sds_name,va_value]
IF usesds EQ 0 THEN sds_name=va_value ELSE va_value=sds_name
ENDELSE
ENDIF
;Determine actual list order from DATA_VARIABLE listing
li=WHERE(STRUPCASE(va_value[0]) EQ STRUPCASE(vn),lcnt)
ci[0]=va_value & vnv=va_value END
STRUPCASE(va_name) EQ 'VAR_DATA_TYPE': BEGIN
ci[1]=va_value & idltype[n_lab-1]=STRUPCASE(STRTRIM(va_value,2))
END
STRUPCASE(va_name) EQ 'VAR_UNITS': BEGIN
;check that VAR_UNITS is a string value (in event that VAR_UNITS=1)
IF va_type NE 'STRING' THEN $
INFOTXT_OUTPUT_A,[20],['VAR_UNITS',STRTRIM(sds_name,2)]
END
STRUPCASE(va_name) EQ 'VAR_DEPEND': BEGIN
TEST_DIM_ORDER,'VD',va_value,va_type,sds_dim,sds_name,rev_vd_vs
END
STRUPCASE(va_name) EQ 'VAR_SIZE': BEGIN
TEST_DIM_ORDER,'VS',va_value,va_type,sds_dim,sds_name,rev_vd_vs
END
STRUPCASE(va_name) EQ 'VAR_VALID_MIN': BEGIN
hdftype[1]=hdftypehold & idltype[1]=idltypehold
END
STRUPCASE(va_name) EQ 'VAR_VALID_MAX': BEGIN
hdftype[2]=hdftypehold & idltype[2]=idltypehold
END
STRUPCASE(va_name) EQ 'VAR_FILL_VALUE': BEGIN
hdftype[3]=hdftypehold & idltype[3]=idltypehold
END
ELSE:
ENDCASE
ENDFOR
ci[3]=STRTRIM(sds_natts,2)
IF ci[0] EQ '' THEN ci[0]=sds_name
ENDIF ELSE BEGIN ;No Attributes, so can only write dataset name to Metadata
ci=[STRUPCASE(sds_name),'','','1'] & vnv=sds_name
;Can the SDS_NAME be matched with a DATA_VARIABLES value?
li=WHERE(STRUPCASE(sds_name) EQ STRUPCASE(vn),lcnt)
IF lcnt NE 0 THEN vnf=1 ELSE vnf=0
INFOTXT_OUTPUT_A,[5],[sds_name,'NCDF_VARINQ']
ENDELSE
FOR k=0,N_ELEMENTS(sds_dim)-1 DO BEGIN
IF k EQ 0 THEN ci[2]=STRTRIM(sds_dim[k],2) ELSE ci[2]=ci[2]+';'+STRTRIM(sds_dim[k],2)
ENDFOR
;Check for possible problems with determining array index
IF lcnt NE 0 THEN BEGIN
IF li[0] GE n_sds THEN dv_order=0 $ ;can occur when number of DATA_VARIABLE values is greater than number of datasets
ELSE IF catinfo[li[0],0] NE '' THEN dv_order=0 ;This array has already been written to
ENDIF ELSE IF (lcnt EQ 0) OR (vnf EQ 0) THEN dv_order=0
;Write out information text if sds_name cannot be matched and determine array index
IF dv_order EQ 0 THEN BEGIN
li=WHERE(catinfo[*,0] EQ '') ;determine lowest available array index to write info to
in0=[4,vnf,lcnt,n_vn]
IF (vnf EQ 0) OR ((lcnt EQ 0) AND (n_vn NE 0L)) THEN INFOTXT_OUTPUT_A,in0,[sds_name,vnv]
ENDIF
oi[j]=li[0] ;to put datasets in the correct order
catinfo[li[0],*]=ci[*]
;Do data type checks on the Dataset, VAR_DATA_TYPE value and data types of VAR_VALID_MIN, VAR_VALID_MAX and VAR_FILL_VALUES
DATA_TYPE_CHECKS,ftype,dt_chk_labels,hdftype,idltype,sds_name,vnv
ENDFOR
rev_vd_vsh=rev_vd_vs ;keep original value in hold variable (required if rev_vd_vs eq 3)
IF (rev_vd_vs EQ 2) OR (rev_vd_vs EQ 3) THEN rev_vd_vs=1 ;change default so that VAR_DEPEND and VAR_SIZE
;are reversed if not o/w changed in TEST_DIM_ORDER
;Dimension the structure to the size of the SDS datasets (with dimension n_sds)
sds=REPLICATE(sds_set, n_sds, max_atts)
;Put datasets and attributes into sds structure
FOR j=0,n_sds-1 DO BEGIN
notranspose=1 ;0/1 will change to 0 if the dataset needs to be transposed
varstruct=NCDF_VARINQ(fileid,di[j]) sds_name=varstruct.name
IF sds_name EQ '' THEN sds_name='N/A'
sds_natts=varstruct.natts
sds_datatype=varstruct.datatype
sds_ndim=varstruct.ndims
;Extract data from a netCDF dataset and get size information
NCDF_VARGET,fileid,di[j],sds_data
IF sds_natts NE 0L THEN BEGIN
;Read in Dataset Attributes
FOR k=0L,sds_natts-1L DO BEGIN
varattsname=NCDF_ATTNAME(fileid,di[j],k)
NCDF_ATTGET,fileid,di[j],varattsname,va_value
va_name=STRUPCASE(varattsname)
vavtype=SIZE(va_value,/TYPE)
vavhold=va_value
IF (vavtype EQ 7) OR (vavtype EQ 1) THEN va_type='STRING' ELSE va_type='NON-STRING'
schk=WHERE(va_name EQ slabel,schkcnt)
test1=(sds_datatype EQ 'CHAR') AND (schkcnt EQ 0) ;string dataset but not one of the slabel values
test2=(sds_datatype NE 'CHAR') AND (va_type EQ 'STRING') ;not a string dataset but string attribute value
test3=(sds_datatype EQ 'CHAR') AND (schkcnt NE 0) AND (vavtype EQ 1) ;string dataset, slabel and attribute value is of type byte
IF (test1) OR (test2) THEN va_value=STRTRIM(va_value,2) $
ELSE IF test3 THEN va_value=STRING(va_value)
IF ((STRUPCASE(va_name) EQ 'VAR_DEPEND') OR (STRUPCASE(va_name) EQ 'VAR_SIZE')) AND $
((rev_vd_vs MOD 10 EQ 1) OR (rev_vd_vs EQ 20)) THEN BEGIN ;need to reverse the va_values
IF va_type NE 'STRING' THEN BEGIN ;VAR_SIZE contains numeric values instead of in the form of a string
vavhold='' & n_vav=N_ELEMENTS(va_value)
FOR i=0,n_vav-1 DO BEGIN
IF i EQ n_vav-1 THEN vtxt='' ELSE vtxt=';'
vavhold=vavhold+STRTRIM(va_value[i],2)+vtxt
ENDFOR
ENDIF ELSE vavhold=va_value
vs_v=STRCOMPRESS(STRSPLIT(vavhold,';',/EXTRACT,COUNT=rcnt),/REMOVE_ALL)
IF rcnt GT 1 THEN BEGIN ;multi-dimensions
IF (rev_vd_vs EQ 20) THEN $ ;AND (STRPOS(STRUPCASE(va_value),'DATETIME') NE -1) THEN $
notranspose=0 ;need to transpose dataset as well
vs_v=REVERSE(vs_v)
FOR i=0,rcnt-1 DO $
IF i EQ 0 THEN va_value=vs_v[i] ELSE va_value=va_value+';'+vs_v[i]
ENDIF
ENDIF
IF ((STRUPCASE(va_name) EQ 'VAR_VALID_MIN') OR (STRUPCASE(va_name) EQ 'VAR_VALID_MAX') OR $
(STRUPCASE(va_name) EQ 'VAR_FILL_VALUE')) AND (sds_datatype EQ 'BYTE') AND $
(vavtype EQ 1) THEN va_value=vavhold
sds[oi[j],k].va_l=PTR_NEW(va_name)
sds[oi[j],k].va_v=PTR_NEW(va_value)
ENDFOR
;This section commented out for now - Version 4.0b16
;Check attributes again in case there are missing GEOMS Attributes that can be filled in
;vnt=''
;FOR k=0,sds_natts-1 DO BEGIN
; varattsname=NCDF_ATTNAME(fileid,j,k)
; NCDF_ATTGET,fileid,j,varattsname,va_value
; va_name=STRUPCASE(varattsname)
; va_type=SIZE(va_value,/TYPE)
; schk=WHERE(va_name EQ slabel,schkcnt)
; IF ((sds_datatype EQ 'CHAR') AND (schkcnt EQ 0)) OR ((sds_datatype NE 'CHAR') AND $
; ((va_type EQ 1) OR (va_type EQ 7))) THEN va_value=STRTRIM(va_value,2) $
; ELSE IF (va_type EQ 1) OR (va_type EQ 7) THEN va_value=STRING(va_value)
; ;Check for VAR_NOTES entry
; lcname=STRLOWCASE(varattsname)
; ni=WHERE(lcname EQ ncdf_desc,ncnt)
; IF ncnt EQ 0 THEN ni=WHERE(lcname EQ ncdf_std,ncnt) ; IF (ncnt EQ 0) AND (lcname NE 'units') THEN BEGIN ;append label and value to VAR_NOTES
; mi=WHERE(attr_arr_data EQ 'VAR_NOTES')
; IF vnt NE '' THEN BEGIN
; IF STRMID(vnt,STRLEN(vnt)-1,1) EQ '.' THEN itxt=' ' ELSE itxt='. '
; ENDIF ELSE itxt=''
; vnt=vnt+itxt+STRTRIM(varattsname,2)+': '+STRTRIM(va_value,2)
; ENDIF
; mcnt=0
; ;Check for VAR_DESCRIPTION entry (long_name)
; IF lcname EQ 'long_name' THEN mi=WHERE(attr_arr_data EQ 'VAR_DESCRIPTION',mcnt)
; ;Check for VAR_UNITS entry
; IF lcname EQ 'units' THEN mi=WHERE(attr_arr_data EQ 'VAR_UNITS',mcnt)
; IF (mcnt NE 0) AND (std_fnd[mi[0]] EQ 0) THEN sds[oi[j],mi[0]].va_v=PTR_NEW(STRTRIM(va_value,2))
; ;Check for valid_range and _FillValue attributes
; ni=WHERE(lcname EQ ncdf_std,ncnt)
; IF ncnt NE 0 THEN BEGIN
; vdt=SIZE(varattsval,/TYPE)
; IF lcname EQ 'valid_range' THEN BEGIN
; mi=WHERE(attr_arr_data EQ 'VAR_VALID_MIN')
; IF std_fnd[mi[0]] EQ 0 THEN sds[oi[j],mi[0]].va_v=PTR_NEW(varattsval[0])
; mi=WHERE(attr_arr_data EQ 'VAR_VALID_MAX')
; IF std_fnd[mi[0]] EQ 0 THEN sds[oi[j],mi[0]].va_v=PTR_NEW(varattsval[1])
; ENDIF ELSE BEGIN
; mi=WHERE(attr_arr_data EQ 'VAR_FILL_VALUE')
; IF std_fnd[mi[0]] EQ 0 THEN sds[oi[j],mi[0]].va_v=PTR_NEW(varattsval)
; ENDELSE
; ENDIF
;ENDFOR
;IF vnt NE '' THEN BEGIN
; mi=WHERE(attr_arr_data EQ 'VAR_NOTES')
; IF std_fnd[mi[0]] EQ 0 THEN sds[oi[j],mi[0]].va_v=PTR_NEW(vnt)
;ENDIF
; ;Write VAR_NAME to metadata
;mi=WHERE(attr_arr_data EQ 'VAR_NAME')
;IF std_fnd[mi[0]] EQ 0 THEN sds[oi[j],mi[0]].va_v=PTR_NEW(sds_name)
;catinfo[oi[j],0]=sds_name
;;Write VAR_DATA_TYPE to metadata
;mi=WHERE(attr_arr_data EQ 'VAR_DATA_TYPE')
;CASE 1 OF
; sds_datatype EQ 'BYTE':catinfo[oi[j],1]='BYTE'
; sds_datatype EQ 'CHAR':catinfo[oi[j],1]='STRING'
; (sds_datatype EQ 'INT') OR (sds_datatype EQ 'SHORT'):catinfo[oi[j],1]='SHORT'
; sds_datatype EQ 'LONG':catinfo[oi[j],1]='INTEGER'
; sds_datatype EQ 'FLOAT':catinfo[oi[j],1]='REAL'
; sds_datatype EQ 'DOUBLE':catinfo[oi[j],1]='DOUBLE'
;ENDCASE
;sds[oi[j],mi[0]].va_v=PTR_NEW(catinfo[oi[j],1])
;;Write VAR_DEPEND to metadata
;mi=WHERE(attr_arr_data EQ 'VAR_DEPEND')
;IF std_fnd[mi[0]] EQ 0 THEN BEGIN
; vd=''
; FOR k=0,sds_ndim-1 DO BEGIN
; IF k EQ 0 THEN vd=vd+STRTRIM(mv_dim[sds_dim[k],0],2) $
; ELSE vd=vd+';'+STRTRIM(mv_dim[sds_dim[k],0],2)
; ENDFOR
; sds[oi[j],mi[0]].va_v=PTR_NEW(vd)
;ENDIF
;;Determine VAR_SIZE from dataset attributes
;mi=WHERE(attr_arr_data EQ 'VAR_SIZE')
;IF std_fnd[mi[0]] EQ 0 THEN BEGIN
; sds_dim=SIZE(sds_data,/DIMENSIONS)
; IF sds_dim[0] EQ 0 THEN sds_dim[0]=1
; vs=''
; FOR k=0,N_ELEMENTS(sds_dim)-1 DO $
; IF k EQ 0 THEN vs=vs+STRTRIM(sds_dim[k],2) ELSE vs=vs+';'+STRTRIM(sds_dim[k],2)
; sds[oi[j],mi[0]].va_v=PTR_NEW(vs)
;ENDIF
ENDIF ELSE BEGIN ;No Attributes, so can only write dataset name to structure
val='VAR_NAME' sds[oi[j],0].va_l=PTR_NEW(val)
sds[oi[j],0].va_v=PTR_NEW(sds_name)
ENDELSE
;Test to see if dataset dimension ordering needs to be changed
IF notranspose EQ 0 THEN sds_data=TRANSPOSE(sds_data)
IF sds_datatype EQ 'CHAR' THEN sds_data=STRING(sds_data)
sds[oi[j],0].data=PTR_NEW(sds_data)
ENDFOR
NCDF_CLOSE,fileid
;Add DATA_VARIABLES to global attributes if not already present
gi=WHERE(STRMID(ga,0,14) EQ 'DATA_VARIABLES',gcnt)
IF gcnt EQ 0 THEN BEGIN
FOR j=0,n_sds-1 DO BEGIN
IF j EQ 0 THEN dv=catinfo[j,0] ELSE dv=dv+';'+catinfo[j,0]
ENDFOR
ga=[ga,'DATA_VARIABLES='+dv]
ENDIF
dv_order=-1 & n_vn=n_sds
ENDELSE
;Array listing order may not be correct
IF (dv_order EQ 0) AND (n_vn NE 0L) THEN INFOTXT_OUTPUT_A,[6]
;If necessary write comment regarding VAR_SIZE and VAR_DEPEND dimension ordering
rev_vd_vs=rev_vd_vsh ;used for when rev_vd_vs eq 3
IF (rev_vd_vs MOD 10 EQ 0) OR (rev_vd_vs EQ 3) THEN INFOTXT_OUTPUT_A,[11,rev_vd_vs]
END ;Procedure Read_HDF_SDS
PRO output_hdf_data, ifile, ga, sds, catinfo
;Procedure to output the contents of an HDF file in ASCII form, either in a format compatible for
;conversion back to HDF, or in Row and Column format, or as a summary list
; ----------
;Written by Ian Boyd for the EVDC/AVDC - iboyd@bryanscientific.org
;
; History:
; 20050729: Original IDLCR8ASCII Routine - Version 1.0
; 20050912: Removed Common variable definition CATALOGINFO and made the variable a parameter passed
; to the procedure; Make output filenames lower case; If data type is double and the
; accompanying VIS_FORMAT value starts with an 'F' (e.g. F9.4), then, change the 'F' to a
; 'D' when specifying the format for output - Version 1.1
; 20061004: Common variable definition WIDGET_WIN_A added for Error calls; Display input/output
; information in a pop-up display window if the program is opened using IDL VM - Version 2.0
; 20080302: Added code which sends the Catalog and log output to the IDLDE output window and/or an
; external file (as determined by the dux array values); Ensure all data values are
; separated by a space when the Dump output option is chosen - Version 3.0
; 20101122: Adopt GEOMS metadata standard; Because FORMAT attribute(s) have been dropped can no longer
; use defined formatting values when outputing data to files; Default ASCII formatting
; adopted - scientific notation for REAL and DOUBLE, except for DATETIME and related values,
; which have formatting dependent on the data type - Version 4.0b1
; 20150127: Ensure all string datasets are written and saved as left justified strings (previously
; defaulted to right justified) - Version 4.0b10
; 20151109: Convert DATETIME (and related) max, min and fill values to LONG64 data type before
; determining the format for ASCII output to account for very large values - Version 4.0b12
; 20161130: Add checks for Metadata values written in an invalid data type (e.g. structure) and for
; invalid attributes - Version 4.0b16
;
; Inputs: ifile - a string containing the name of the HDF file to be read in.
; ga - a string array containing the global attribute labels and values extracted from the HDF
; file.
; sds - a structure using pointers, of size [n_sds,n_atts], containing the variable attribute
; labels and values (sds[n,m].va_l and sds[n,m].va_v) and the data (sds[n,0].data)
; extracted from the HDF file.
; catinfo - a string array of size [n_sds,4] (where n_sds is the number of datasets in the HDF
; file), containing information on the variable name, data type, data dimension, and
; number of attributes;
; Outputs: Nil
;
; Called by: IDLCR8ASCII
;
; Subroutines Called: None
COMMON WIDGET_WIN_A
;determine number of datasets and attributes (from sds.va)
as=SIZE(catinfo)
n_sds=as[1]
indir=FILE_DIRNAME(ifile,/MARK_DIRECTORY)
infile=FILE_BASENAME(ifile)
vdtype=[1,2,3,4,5,7,12,13,14,15] ;valid IDL data types
badlabel=[''] ;list of attribute labels where the values are not valid (to avoid repeated log messages)
IF o3[1] EQ 'C' THEN BEGIN
IF o3[3] EQ '' THEN BEGIN
lineno=lineno+n_sds+1
wintxt='Listing of Var_Name Index; Var_Name; Var_Data_Type; Var_Size'
WIDGET_CONTROL,wtxt,set_value=wintxt,/Append,Set_text_top_line=lineno
ENDIF
FOR i=dux[0],dux[1],dux[2] DO BEGIN
IF i EQ -1 THEN PRINT,'Listing of Var_Name Index; Var_Name; Var_Data_Type; Var_Size' $
ELSE PRINTF,i,'Listing of Var_Name Index; Var_Name; Var_Data_Type; Var_Size'
ENDFOR
FOR j=0,n_sds-1 DO BEGIN
IF o3[3] EQ '' THEN BEGIN
wintxt=STRTRIM(j,2)+'; '+catinfo[j,0]+'; '+catinfo[j,1]+'; '+catinfo[j,2]
WIDGET_CONTROL,wtxt,set_value=wintxt,/Append
ENDIF
FOR i=dux[0],dux[1],dux[2] DO $
IF i EQ -1 THEN PRINT,STRTRIM(j,2),'; ',catinfo[j,0],'; ',catinfo[j,1],'; ',catinfo[j,2] $
ELSE PRINTF,i,STRTRIM(j,2),'; ',catinfo[j,0],'; ',catinfo[j,1],'; ',catinfo[j,2]
ENDFOR
ENDIF
IF o3[0] NE '0' THEN BEGIN
;create output filenames
outf1=STRMID(infile,0,STRPOS(infile,'.',/Reverse_Search))+'.meta'
outf2=STRMID(infile,0,STRPOS(infile,'.',/Reverse_Search))+'.data'
OPENW,lu1,indir+outf1,/GET_LUN
OPENW,lu2,indir+outf2,/GET_LUN
;HDF4 and NETCDF pre-defined attributes
ncsa=['long_name','units','format','coordsys','valid_range','_FillValue','scale_factor', $
'scale_factor_err','add_offset','add_offset_err','calibrated_nt']
PRINTF,lu1,'! Output from IDLcr8ASCII application v4.0'
PRINTF,lu1,'! '+ifile & PRINTF,lu1,'!'
PRINTF,lu1,'! Global Attributes'
FOR i=0,N_ELEMENTS(ga)-1 DO PRINTF,lu1,ga[i]
FOR j=0,n_sds-1 DO BEGIN
n_atts=FIX(catinfo[j,3])
IF n_atts NE 0 THEN BEGIN
PRINTF,lu1,'!' & PRINTF,lu1,'! Variable Attributes'
PRINTF,lu2,catinfo[j,0]
FOR i=0,n_atts DO BEGIN ;number of attributes. Last loop is for Data
IF i EQ n_atts THEN vavt=SIZE(*sds[j,0].data,/TYPE) $
ELSE vavt=SIZE(*sds[j,i].va_v,/TYPE)
gti=WHERE(vavt EQ vdtype,g_vavt) ;test to see if the data type is valid
IF g_vavt EQ 1 THEN BEGIN ;it is OK to transfer the attribute value to a variable
IF i EQ n_atts THEN vav=*sds[j,0].data ELSE vav=*sds[j,i].va_v
ENDIF
IF (STRPOS(STRUPCASE(catinfo[j,0]),'DATETIME') NE -1) AND ((vavt EQ 4) OR (vavt EQ 5)) THEN BEGIN
ns=MAX(STRLEN(STRTRIM(LONG64(vav),2)))
IF vavt EQ 4 THEN dp=6 ELSE dp=9 ;indicates the number of decimal places to use in the output
;If the number is very large then output as an exponent IF ns LT 8 THEN fmt='(D'+STRTRIM(ns+dp+1,2)+'.'+STRTRIM(dp,2)+')' ELSE fmt='(E0)'
ENDIF ELSE IF (vavt EQ 4) OR (vavt EQ 5) THEN fmt='(E0)' $
ELSE IF vavt EQ 7 THEN fmt='(A-)' $
ELSE fmt='(I0)'
IF i NE n_atts THEN BEGIN ;write out metadata in form LABEL=VALUE
nvav=N_ELEMENTS(vav)
vat=*sds[j,i].va_l
ni=WHERE(STRLOWCASE(vat) EQ STRLOWCASE(ncsa),ncnt) ;check for pre-defined attributes and exclude
IF (ncnt EQ 0) AND (g_vavt EQ 1) THEN BEGIN
IF nvav EQ 1 THEN vat=vat+'='+STRING(FORMAT=fmt,vav[0]) $ ;'(A-'+STRTRIM(STRLEN(vav[0]),2)+')',vav[0]) $
ELSE BEGIN
FOR k=0,N_ELEMENTS(vav)-1 DO BEGIN
IF k EQ 0 THEN vat=vat+'='+STRTRIM(STRING(FORMAT=fmt,vav[k]),2) $
ELSE vat=vat+';'+STRTRIM(STRING(FORMAT=fmt,vav[k]),2)
ENDFOR
ENDELSE
PRINTF,lu1,vat
ENDIF ELSE IF g_vavt NE 1 THEN BEGIN
bli=WHERE(vat EQ badlabel,blcnt)
IF blcnt EQ 0 THEN BEGIN
badlabel=[badlabel,vat]
INFOTXT_OUTPUT_A,[18],[vat]
ENDIF
ENDIF
ENDIF
ENDFOR
;Write out data
IF o3[0] EQ 'F' THEN BEGIN
PRINTF,lu2,format=fmt,vav
ENDIF ELSE IF o3[0] EQ 'D' THEN BEGIN
;Ensure there is a single character between all values
sdssize=SIZE(vav,/Dimension)
sdshold=STRTRIM(STRING(FORMAT=fmt,vav),2)
maxstr=MAX(STRLEN(sdshold))
;use default IDL formatting rules
sdshold=STRING(format='(A-'+STRTRIM(maxstr,2)+')',sdshold)
;Uncomment line below if format described by VIS_FORMAT is required
;sdshold=STRING(format=p_f,*sds[j].data)
IF sdssize[0] NE 0 THEN sdshold=REFORM(sdshold,sdssize)
PRINTF,lu2,sdshold
ENDIF
ENDIF ELSE BEGIN ;Variable attributes do not have an associated dataset
INFOTXT_OUTPUT_A,[19],[catinfo[j,0]]
ENDELSE
ENDFOR
PRINTF,lu1,'!' & PRINTF,lu1,'! End of output file created by IDLcr8ASCII'
FREE_LUN,lu1 & FREE_LUN,lu2
IF o3[3] EQ '' THEN BEGIN
lineno=lineno+2L
WIDGET_CONTROL,wtxt,set_value=outf1+' created!',/Append,Set_text_top_line=lineno
WIDGET_CONTROL,wtxt,set_value=outf2+' created!',/Append,Set_text_top_line=lineno
ENDIF
FOR i=dux[0],dux[1],dux[2] DO BEGIN
PRINTF,i,' '+outf1+' created!' & PRINTF,i,' '+outf2+' created!'
ENDFOR
ENDIF
END ;Procedure Output_HDF_Data
PRO idlcr8ascii, ifile, ga, sds, reterr, $
FORMAT=o1, DUMP=o2, CATALOG=o4, POPUP=o5, LOG=o6, H4=o7, H5=o8, NC=o9
;Main IDL procedure to convert GEOMS compliant netCDF, HDF4 and HDF5 files to ASCII, Session Memory
;or another recognized Data format.
;
;Program documentation, idlcr8ascii-v4.0_Readme.pdf, available on http://avdc.gsfc.nasa.gov.
;
;Program sub-version 4.0b27 (20240927)
; ----------;Written by Ian Boyd for the EVDC/AVDC - iboyd@bryanscientific.org
;
; History:
; 20050729: Original Release - Version 1.0
; 20050912: Removed Common variable definition CATALOGINFO; If the format of the
; DATA_START_DATE and FILE_GENERATION_DATE values is MJD2000, then change to ISO8601,
; unless the output option is /Dump - Version 1.1
; 20061004: Common variable definition WIDGET_WIN_A added for Error calls; Make the code suitable
; for running on a licensed version of IDL (using the .pro and .sav versions of the code)
; and on IDL Virtual Machine (using the .sav version of the code); Change the command
; line keyword options. Remove the /Help option (window now opened if there are no command
; line parameters), and /Catalog can now be called together with /AVDC or /Dump; The
; program can now handle multiple HDF files as input (either as a string array or as a
; file spec); Display input/output information as well as errors and warnings in a pop-up
; display window if the program is called using IDL VM; Help window becomes an
; Introduction window, and the user has the option of continuing to run the program with
; file inputs; Include option to read HDF5 versions of the ground-based files
; - Version 2.0
; 20080302: Remove HELP,/TRACEBACK call, which identified how the program was called. This was used
; to stop output to the IDLDE output window in the event that IDL VM was used, but it is
; not required as IDL VM ignores these print calls; Add options to send Catalog and logged
; input/output information to a Pop-up Box (/Popup) and/or to an external file (/Log). Add
; DIALOG_BOX to show completion of the program if program inputs are via the INTRO box, and
; logging window option isn't requested - Version 3.0
; 20091208: Add INFORMATION text message to INFOTXT_OUTPUT_A procedure; Replace /AVDC keyword with
; /FORMAT (note /AVDC also retained for backward compatibility purposes); Improve checks
; on parameter and keyword inputs - Version 3.03
; 20101122: Changes made to account for new GEOMS conventions, including changing the format of the
; structure so that numeric metadata values can be saved in their actual datatype, rather
; than string; Add return error code option so that program returns to the calling program
; if an error is generated - Version 4.0b1
; 20120328: Add options to convert HDF/netCDF files to other HDF/netCDF formats e.g. H4 to
; H5 etc. Requires IDLcr8HDF in the search path to do this - Version 4.0b5
; 20130114: Add check for IDL being run in DEMO mode by using LMGR command. Replace PRINTF,-1
; statements (i.e. when dux[0] eq -1) with PRINT (o/w causes DEMO mode to stop). Also disable
; /LOG, /FORMAT and /DUMP keywords and stop ASCII file output in DEMO mode - Version 4.0b7
; 20150127: Add extra condition for the file extension when checking whether an input file is netCDF
; due to the program crashing when incorrectly identifying an HDF5 file as netCDF when
; using IDL8.3 (OK for IDL6.4) - Version 4.0b10
; 20160213: Add CATCH, /CANCEL after idlcr8hdf call - Version 4.0b13
; 20190806: Call routine FILE_FORMAT_A to determine the correct format of the input file(s)
; - Version 4.0b22
; 20200930: Check that GEOMS file does not have filesize of zero before calling FILE_FORMAT_A
; - Version 4.0b24
;
; Inputs: ifile - a string array or filespec containing the name of the HDF file(s) to be read in
;
; OPTIONS
; FORMAT - Program generates two output files per input file (with .meta and .data
; extensions). The resulting files are compatible with programs that can write HDF
; files using these two files as input.
; DUMP - Program generates two output files per input file (with .meta and .data extensions).
; Data values will be shown as indicated by the array format defined by VAR_SIZE.
; CATALOG - Program sends the variable index, variable name, format, and dimension(s)
; information to a pop-up window or the IDL DE output log window.
; POPUP - to append input/output information as well as warnings and errors to a pop-up
; display window
; LOG - to append input/output/catalog information as well as errors to a log file named
; idlcr8ascii.log
; H4 - outputs contents of the input file as an HDF4 file
; H5 - outputs contents of the input file as an HDF4 file
; NC - outputs contents of the input file as a netCDF3 file
;
; Outputs: ga - If included as a parameter in the command line, a string array containing the global
; attribute labels and values extracted from a single HDF file.
; sds - If included as a parameter in the command line, a structure using pointers, of size
; [n_sds,n_atts], containing the variable attribute labels and values (sds[n,m].va_l
; and sds[n.m].va_v) and the data (sds[n,0].data) extracted from a single HDF file.
; The data is saved in the form defined by the variable attributes
; reterr - optional string input that, if included, returns an error message to the calling; program rather than stop the program. Note that the Pop-up option will be
; deselected if present together with this argument.
; Two ASCII files per input file containing the attributes and datasets extracted from the HDF
; file(s), if the FORMAT or DUMP options are chosen. Summary metadata information if the
; CATALOG option is chosen. Other Hierachical data format files if H4, H5, or netCDF are chosen.
;
; Called by: Main Routine
;
; Subroutines Called: INTRO_A (if program called without command line parameters)
; FILE_FORMAT_A (to determine the format of the file(s))
; READ_HDF_SDS
; JDF_2_DATETIME
; OUTPUT_HDF_DATA (if output is not just to session memory only)
; STOP_WITH_ERROR_A (if error state detected)
; INFOTXT_OUTPUT_A (if program can make a change)
; Possible Conditions for STOP_WITH_ERROR call (plus [line number] where called):
; 1. No valid HDF4, HDF5 or netCDF file selected.
;
; Information Conditions (when the program is able to make changes):
; 1. If Session Memory option is chosen can only read in one HDF file.
; 2. /POPUP keyword cannot be used together with the 'reterr' argument.
; 3. Argument 'reterr' must be a scalar variable of type string.
; 4. /LOG, /FORMAT, and /DUMP keywords cannot be used in IDL DEMO mode
; 5. NetCDF file create feature is disabled in IDL DEMO mode
COMMON WIDGET_WIN_A
;Possible error messages for this procedure
proname='IDLCR8ASCII procedure: '
IF FLOAT(!Version.Release) GE 5.6 THEN errtxt='No valid HDF or netCDF file selected.' $
ELSE BEGIN
errtxt='No valid HDF or netCDF file selected '
errtxt=errtxt+'(Note: Input file can only be HDF4 or netCDF3 for IDL'+!Version.Release+').'
ENDELSE
lu=-1
ON_IOERROR,TypeConversionError ;used when checking Datetime format
demomode=LMGR(/DEMO) ;Boolean to check for IDL being run in demo mode (disable /LOG option)
IF N_PARAMS() EQ 1 THEN BEGIN ;File input only
;check to see what output keyword options are set - if none then intype=-3 (i.e. invalid)
IF (~ KEYWORD_SET(o1)) AND (~ KEYWORD_SET(o2)) AND (~ KEYWORD_SET(o4)) AND $
(~ KEYWORD_SET(o7)) AND (~ KEYWORD_SET(o8)) AND (~ KEYWORD_SET(o9)) THEN $
intype=-3 ELSE intype=SIZE(ifile,/Type)
ENDIF ELSE IF N_PARAMS() GE 1 THEN intype=SIZE(ifile,/Type) $ ;Check that infile is a string
ELSE IF (KEYWORD_SET(o1)) OR (KEYWORD_SET(o2)) OR (KEYWORD_SET(o4)) OR $
(KEYWORD_SET(o7)) OR (KEYWORD_SET(o8)) OR (KEYWORD_SET(o9)) THEN $
intype=-1 $ ;need to open dialog box for input file
ELSE IF (KEYWORD_SET(o5)) OR (KEYWORD_SET(o6)) THEN intype=-3 $ ;cannot have these keyword options only
ELSE intype=-2 ;no parameter or keyword inputs in idlcr8ascii call
IF (intype NE 7) AND (intype GE 0) THEN intype=-3 ;first command line parameter not a string type
dux=[-1,-1,1] ;initialize flags for output log to IDLDE Output Window and/or to file
;dux[0] default allows output to the IDLDE output window (ignored by IDL VM)
;dux[1] is the program assigned file unit for sending log output to file
;dux[2] is the step between dux[0] and dux[1]
rerr='NA' ;initialize return error string
o3=['-1','0','0','0','0','0','0','0']
IF intype LT -1 THEN BEGIN ;either no or invalid input parameters and/or keywords
INTRO_A,intype ;Open Intro Box and determine output options (FORMAT/DUMP, Catalog, etc)
IF o3[0] EQ '-1' THEN BEGIN
STOP_WITH_ERROR_A,'','',lu & RETURN
ENDIF
IF o3[3] EQ 'P' THEN o3[3]='' ELSE o3[3]='D_'
ENDIF ELSE BEGIN ;Set options (FORMAT/DUMP/CONVERT, Catalog, Session Memory, Popup Window, Log Output)
IF (KEYWORD_SET(o1)) THEN o3=['F','0','0','D_','0','0','0','0'] $
ELSE IF KEYWORD_SET(o2) THEN o3=['D','0','0','D_','0','0','0','0'] $
ELSE o3=['0','0','0','D_','0','0','0','0']
IF N_PARAMS() GE 3 THEN o3[2]='M'
IF KEYWORD_SET(o4) THEN o3[1]='C' IF KEYWORD_SET(o5) THEN o3[3]=''
IF KEYWORD_SET(o6) THEN o3[4]='idlcr8ascii.log'
IF KEYWORD_SET(o7) THEN o3[5]='H4'
IF KEYWORD_SET(o8) THEN o3[6]='H5'
IF KEYWORD_SET(o9) THEN o3[7]='N3'
ENDELSE
;Check for reterr parameter
infoval=INTARR(4)
IF (N_PARAMS() EQ 2) OR (N_PARAMS() GE 4) THEN BEGIN
;If reterr included allows error output to return to a calling program
IF N_PARAMS() EQ 2 THEN reterr=ga ;session memory option not wanted
;IF (ARG_PRESENT(reterr)) AND (SIZE(reterr,/Type) EQ 7) THEN BEGIN
IF SIZE(reterr,/Type) EQ 7 THEN BEGIN
rerr=''
IF o3[3] EQ '' THEN BEGIN
;Deselect POPUP option and write INFORMATION message
o3[3]='D_' & infoval[0]=14
ENDIF
ENDIF ELSE infoval[1]=15
;reterr input included but is of the incorrect type, or parameter cannot be returned to the calling program
ENDIF
;Do check for /LOG, /FORMAT, /DUMP, and/or /NC keywords in IDL DEMO Mode and disable if necessary
IF (demomode) AND ((o3[4] EQ 'idlcr8ascii.log') OR (o3[0] NE '0') OR (o3[7] EQ 'N3')) THEN BEGIN
IF (o3[4] EQ 'idlcr8ascii.log') OR (o3[0] NE '0') THEN infoval[2]=16
IF o3[7] EQ 'N3' THEN infoval[3]=17
o3[0]='0' & o3[4]='0' & o3[7]='0'
ENDIF
IF N_PARAMS() GE 1 THEN BEGIN
;Check that input HDF/netCDF files exist. If not then prompt the user for the missing files
arorfs=SIZE(ifile,/Dimensions) ;Is file input Filespec or String Array?
IF (arorfs[0] EQ 0) THEN BEGIN ;input is filespec
IF ifile NE '' THEN ifile=FILE_SEARCH(ifile)
isize=SIZE(ifile,/Dimensions)
IF isize[0] EQ 0 THEN ifile=[''] ;no files found matching filespec
ENDIF
IF N_ELEMENTS(arorfs) GT 1 THEN ifile=REFORM(ifile,N_ELEMENTS(ifile),/Overwrite) ;Convert to 1-D array
file_exist=FILE_TEST(ifile,/Read)
gi=WHERE(file_exist EQ 1,gcnt)
IF gcnt NE 0 THEN ifile=ifile[gi] ;contains all valid filenames
ENDIF ELSE ifile=['']
gi=WHERE(ifile NE '',nfile)
IF nfile EQ 0L THEN BEGIN
IF o3[2] EQ 'M' THEN $ ;pick one file only
ifile=DIALOG_PICKFILE(Filter=['*.hdf','*.h5','*.nc;*nc4'],/Must_Exist, $
Title='Select HDF4, HDF5, or netCDF GEOMS Compatible Format File') $
ELSE $ ;multiple selection permissable
ifile=DIALOG_PICKFILE(Filter=['*.hdf','*.h5','*.nc;*.nc4'],/Must_Exist, $
/Multiple_Files,Title='Select HDF4, HDF5, or netCDF GEOMS Compatible Format File(s)')
ENDIF
;now check to see that files were actually selected, if not STOP!
gi=WHERE(ifile NE '',nfile)
IF nfile EQ 0L THEN BEGIN
STOP_WITH_ERROR_A,'D_'+proname,errtxt,lu
IF STRLEN(rerr) GT 2 THEN BEGIN
IF N_PARAMS() EQ 2 THEN ga=rerr ELSE reterr=rerr
ENDIF
RETURN
ENDIF ELSE BEGIN
ifile=ifile[gi] & dsort=SORT(ifile) & ifile=ifile[dsort]
ifile=FILE_SEARCH(ifile,/FULLY_QUALIFY_PATH)
ENDELSE
;Set dux values according to output options
IF o3[4] NE '0' THEN BEGIN ;open idlcr8ascii.log
o3[4]=FILE_DIRNAME(ifile[0],/Mark_Directory)+o3[4]
res=FILE_TEST(o3[4],/Write)
IF res EQ 0 THEN openw,du,o3[4],/GET_LUN $ ELSE BEGIN
OPENW,du,o3[4],/Append,/GET_LUN & FOR i=0,1 DO PRINTF,du,''
ENDELSE
dux[1]=du & dux[2]=dux[1]-dux[0]
ENDIF
FOR i=dux[0],dux[1],dux[2] DO $
IF i EQ -1 THEN PRINT,'HDF/netCDF File Input/Output Log - Program Started on '+SYSTIME(0) $
ELSE PRINTF,i,'HDF/netCDF File Input/Output Log - Program Started on '+SYSTIME(0)
IF o3[3] EQ '' THEN BEGIN
;Set-up output window display widget
base=WIDGET_BASE(Title='HDF/netCDF File Input/Output Log',Units=2,yoffset=3,xoffset=3,Tlb_Frame_Attr=1,/Column) ;,Tab_Mode=1)
wtxt=WIDGET_TEXT(base,/Scroll,xsize=100,ysize=12)
base2=WIDGET_BASE(base)
b3=WIDGET_BUTTON(base2,value='Finish',uvalue='DONE',frame=3,Sensitive=0) ;,Accelerator='Return')
WIDGET_CONTROL,wtxt,/Realize
WIDGET_CONTROL,base,/Realize
lineno=0L
ENDIF
IF (o3[2] EQ 'M') AND (N_ELEMENTS(ifile) GT 1) THEN BEGIN
ifile=[ifile[0]] & nfile=1L
INFOTXT_OUTPUT_A,[7]
ENDIF
FOR i=0,N_ELEMENTS(infoval)-1 DO IF infoval[i] NE 0 THEN INFOTXT_OUTPUT_A,[infoval[i]]
FOR ndf=0L,nfile-1L DO BEGIN
catinfo=STRARR(1,4) ;initialize catinfo array
ifile[ndf]=FILE_SEARCH(ifile[ndf],/Fully_Qualify_Path)
IF o3[3] EQ '' THEN BEGIN
lineno=lineno+2L
IF ndf NE 0 THEN WIDGET_CONTROL,wtxt,set_value='',/Append
WIDGET_CONTROL,wtxt,set_value='Reading '+ifile[ndf],/Append,Set_text_top_line=lineno
ENDIF
FOR i=dux[0],dux[1],dux[2] DO BEGIN
IF i EQ -1 THEN BEGIN
IF ndf NE 0 THEN PRINT,''
PRINT,'Reading '+ifile[ndf] & PRINT,''
ENDIF ELSE BEGIN
IF ndf NE 0 THEN PRINTF,i,''
PRINTF,i,'Reading '+ifile[ndf] & PRINTF,i,''
ENDELSE
ENDFOR
ftype='XX'
IF (FILE_TEST(ifile[ndf])) AND (~FILE_TEST(ifile[ndf],/ZERO_LENGTH)) THEN BEGIN
;Identify the file format
ftype=FILE_FORMAT_A(ifile[ndf])
IF ftype NE 'XX' THEN catinfo[0,0]=ftype
ENDIF
IF ftype EQ 'XX' THEN BEGIN
;Either file doesn't exist or is not a valid file format
STOP_WITH_ERROR_A,o3[3]+proname,errtxt,lu
IF STRLEN(rerr) GT 2 THEN BEGIN
IF N_PARAMS() EQ 2 THEN ga=rerr ELSE reterr=rerr
ENDIF
RETURN
ENDIF
;Open and read the file
READ_HDF_SDS,ifile[ndf],ga,sds,catinfo
IF STRLEN(rerr) GT 2 THEN BEGIN
reterr=rerr & RETURN
ENDIF
IF catinfo[0,0] EQ 'HE5' THEN BEGIN
STOP_WITH_ERROR_A,o3[3]+proname,errtxt,lu
IF STRLEN(rerr) GT 2 THEN BEGIN
IF N_PARAMS() EQ 2 THEN ga=rerr ELSE reterr=rerr ENDIF
RETURN
ENDIF ELSE BEGIN ;Do checks and output standard format file
;Convert format of DATA_START/STOP_DATE and FILE_GENERATION_DATE if required
;Note: for /Dump option output will still be original format
isodates=['DATA_START_DATE','DATA_STOP_DATE','FILE_GENERATION_DATE']
niso=N_ELEMENTS(isodates)
gaiso=STRARR(niso) & gih=INTARR(niso)
FOR i=0,niso-1 DO BEGIN
gi=WHERE(STRPOS(STRUPCASE(ga),isodates[i]) NE -1,gcnt)
IF gcnt EQ 1 THEN BEGIN
res=STRSPLIT(ga[gi[0]],' =',/EXTRACT)
IF N_ELEMENTS(res) EQ 2 THEN BEGIN
IF STRPOS(STRUPCASE(res[1]),'Z') EQ -1 THEN BEGIN ;i.e. not ISO8601
valid=0 ;used for Type Conversion Check
mjd=DOUBLE(res[1]) ;will jump to TypeConversionError if Res[1] is not a number
valid=1 ;got to here so valid conversion (presumably it is MJD2000)
TypeConversionError:
IF valid EQ 1 THEN BEGIN
iso='' & iso=JDF_2_DATETIME(mjd,/MJD2000,/S)
gaiso[i]=res[0]+'='+iso
IF o3[0] EQ 'F' THEN ga[gi[0]]=gaiso[i]
gih[i]=gi[0]
ENDIF
ENDIF
ENDIF
ENDIF
ENDFOR
;Create ASCII Output
IF (o3[0] NE '0') OR (o3[1] NE '0') THEN OUTPUT_HDF_DATA,ifile[ndf],ga,sds,catinfo
IF (o3[5] NE '0') OR (o3[6] NE '0') OR (o3[7] NE '0') THEN BEGIN
;Check for existence of TAV file in the input file directory or working directory
indir=FILE_DIRNAME(ifile[ndf],/MARK_DIRECTORY)
tav=FILE_SEARCH(indir+'tableattrvalue*.dat',/FULLY_QUALIFY_PATH)
IF tav[0] EQ '' THEN INFOTXT_OUTPUT_A,[12] $ ;No valid TAV file so format conversion cannot be done
ELSE BEGIN
;Convert data to a different format
validpath=1 & ftest=5 & dferr=''
IF demomode THEN ftestmax=6 ELSE ftestmax=7
WHILE (validpath EQ 1) AND (ftest LE ftestmax) DO BEGIN
IF o3[ftest] NE '0' THEN BEGIN
CATCH, path_error ;Return program generated error if IDLcr8HDF is not in the search path
;Error Handler
IF path_error NE 0 THEN BEGIN ;IDLcr8HDF is not in the IDL Search Path
validpath=0
INFOTXT_OUTPUT_A,[13]
;PRINT, 'Error index: ', path_error
;PRINT, 'Error message: ', !ERROR_STATE.MSG
CATCH, /CANCEL
ENDIF
IF validpath EQ 1 THEN BEGIN
;Try to call IDLcr8HDF
IF demomode THEN BEGIN
CASE 1 OF
ftest EQ 5: IDLCR8HDF,ga,sds,tav[0],indir,dferr
ftest EQ 6: IDLCR8HDF,ga,sds,tav[0],indir,dferr,/H5
ENDCASE
ENDIF ELSE BEGIN
CASE 1 OF
ftest EQ 5: IDLCR8HDF,ga,sds,tav[0],indir,dferr,/L
ftest EQ 6: IDLCR8HDF,ga,sds,tav[0],indir,dferr,/H5,/L
ftest EQ 7: IDLCR8HDF,ga,sds,tav[0],indir,dferr,/NC,/L
ENDCASE
ENDELSE
CATCH, /CANCEL
IF o3[4] NE '0' THEN BEGIN
;transfer contents of idlcr8hdf.log to idlcr8ascii.log
OPENR,fu,indir+'idlcr8hdf.log',/DELETE,/GET_LUN
dum='' READF,fu,dum & READF,fu,dum
WHILE NOT EOF(fu) DO BEGIN
READF,fu,dum & PRINTF,du,dum
ENDWHILE
FREE_LUN,fu
ENDIF ELSE FILE_DELETE,indir+'idlcr8hdf.log',/QUIET
ENDIF
ENDIF
ftest++
ENDWHILE
ENDELSE
ENDIF
FOR i=0,niso-1 DO IF gaiso[i] NE '' THEN ga[gih[i]]=gaiso[i]
IF o3[2] EQ 'M' THEN BEGIN
IF o3[3] EQ '' THEN BEGIN
lineno=lineno+1L
WIDGET_CONTROL,wtxt,set_value=ifile+' array and heap structure created!',/Append,Set_text_top_line=lineno
ENDIF
FOR i=dux[0],dux[1],dux[2] DO $
IF i EQ -1 THEN PRINT,' '+ifile[ndf]+' array and heap structure created!' $
ELSE PRINTF,i,' '+ifile[ndf]+' array and heap structure created!'
ENDIF
ENDELSE
ENDFOR
IF rerr EQ '' THEN $
IF nfile EQ 1 THEN reterr=ifile[0]+' successfully read' $
ELSE reterr='Files successfully read'
IF N_PARAMS() EQ 2 THEN ga=reterr
FOR i=dux[0],dux[1],dux[2] DO BEGIN
IF i EQ -1 THEN BEGIN
PRINT,'' & PRINT,'File read completed - Program Ended on '+SYSTIME(0)
ENDIF ELSE BEGIN
PRINTF,i,'' & PRINTF,i,'File read completed - Program Ended on '+SYSTIME(0)
ENDELSE
ENDFOR
IF dux[1] GT -1 THEN FREE_LUN,dux[1]
IF o3[3] EQ '' THEN BEGIN
WIDGET_CONTROL,b3,Sensitive=1,/Input_Focus
WIDGET_CONTROL,wtxt,set_value='',/Append,Set_text_top_line=lineno+2L
WIDGET_CONTROL,wtxt,set_value='HDF file read completed - hit <Finish> to close program',/Append
XMANAGER,'idlcr8ascii',base
ENDIF ELSE IF intype LT 0 THEN BEGIN ;Create Finish Dialog Box
res=DIALOG_MESSAGE('HDF file read completed!',/Information,Title='EVDC/AVDC IDLcr8ASCII')
ENDIF
END ;procedure IDLcr8ASCII