Skip to content

GitLab

Commit 1126e76a authored by Ian Boyd's avatar Ian Boyd
Browse files

Replace idlcr8hdf.pro

parent 5a6731a2
Hide whitespace changes
Inline Side-by-side
idlcr8hdf.pro
View file @ 1126e76a
;Main Program Version: idlcr8hdf.pro v4.0b55, 20200709
; Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Main Program Version: idlcr8hdf.pro v4.0b56, 20201020
; Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
;Sub-versions (refer to idlcr8hdf-v4.0_Readme.pdf for full history)
PRO idlcr8hdf_common
;Procedure to define the COMMON blocks for this program
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
;History:
; 20050909: Introduced to IDLCR8HDF - Version 1.1
......@@ -42,7 +42,7 @@ END ;Procedure idlcr8hdf_common
PRO intro_event, ev
;Procedure to define how Events from the Start-up Introduction Window are handled
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
;History:
; 20061012: Introduced to IDLCR8HDF - Version 2.0
......@@ -85,7 +85,7 @@ PRO intro, intype
;or invalid parameters, or is called by IDL Virtual Machine. The user has a choice of continuing with
;the program after selecting input options, or stopping the program.
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
;History:
; 20061012: Introduced to IDLCR8HDF - Version 2.0
......@@ -113,7 +113,7 @@ PRO intro, intype
COMMON WIDGET_WIN
nhdr=44 & errtxt=STRARR(nhdr)
vertxt=['idlcr8hdf-v4.0_Readme.pdf','v4.0b55 July 2020']
vertxt=['idlcr8hdf-v4.0_Readme.pdf','v4.0b56 October 2020']
errtxt[1]='Welcome to IDLcr8HDF. This program creates GEOMS compliant HDF4, HDF5 and netCDF files'
errtxt[2]='(also refer to '+vertxt[0]+').'
errtxt[4]='Inputs to the program (IDL Virtual Machine (VM) and IDL Licensed (LIC) Versions):'
......@@ -192,7 +192,7 @@ END ;Intro
PRO idlcr8hdf_event, ev
;Procedure to close the pop-up logging window after user selects 'Finish'.
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
;History:
; 20061012: Introduced to IDLCR8HDF - Version 2.0
......@@ -230,7 +230,7 @@ PRO stop_with_error, txt1, txt2, lu
;a Pop-up dialog window; the Pop-up logging window; the Output Logging window in the IDLDE;
;a log file (idlcr8hdf.log)
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
;History:
; 20050802: Original IDLCR8HDF Routine - Version 1.0
......@@ -273,7 +273,7 @@ COMMON WIDGET_WIN
IF lu NE -1L THEN FREE_LUN,lu
IF txt1 EQ '' THEN BEGIN ;<cancel> chosen on Intro box
res=DIALOG_MESSAGE('IDLcr8HDF Stopped!',/Information,Title='AVDC IDLcr8HDF')
res=DIALOG_MESSAGE('IDLcr8HDF Stopped!',/Information,Title='EVDC IDLcr8HDF')
ENDIF ELSE BEGIN
;If necessary free up memory by destroying the heap variables pointed at by its pointer arguments
IF N_ELEMENTS(ds) NE 0 THEN PTR_FREE,ds.data
......@@ -297,7 +297,7 @@ ENDIF ELSE BEGIN
;write error to DIALOG Box instead of Popup window
errtxt2=STRARR(4)
errtxt2[0]=STRMID(txt1,2) & errtxt2[1]=txt2 & errtxt2[3]='IDLcr8HDF Stopped!'
res=DIALOG_MESSAGE(errtxt2,/Error,Title='AVDC IDLcr8HDF Error')
res=DIALOG_MESSAGE(errtxt2,/Error,Title='EVDC IDLcr8HDF Error')
ENDIF ELSE IF rerr[0] EQ 'NA' THEN BEGIN ;write error to Popup window
IF o3[4] EQ '0' THEN endtxt='GEOMS file creation ' $
ELSE endtxt='GEOMS file testing '
......@@ -328,7 +328,7 @@ PRO infotxt_output, infotxt
;window, IDLDE output log window, and/or the log file. Code for this output was originally
;written directly in the affected procedures
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
;History:
; 20090311: Introduced to IDLCR8HDF - Version 3.06
......@@ -619,14 +619,14 @@ PRO var_units_test, vuvalue, rd, tab_type, var_si_conv, errstate
;Procedure to perform checks on the VAR_UNITS value in the metadata and, based on the VAR_UNITS input,
;calculate and return the VAR_SI_CONVERSION value.
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
; History:
; 20061012: Introduced to IDLCR8HDF - Version 2.0
; (Previously these checks were performed in the Extract_and_Test routine)
; 20080302: var_unit_arr and unit_pre_arr added which hold, respectively, the valid VAR_UNITS and
; corresponding VAR_SI_CONVERSION values, and the set of UNIT_PREFIXs (previously
; values from the AVDC TAV file have been used). This has been done so that the
; values from the EVDC/AVDC TAV file have been used). This has been done so that the
; routine can be stand-alone (i.e. not dependent on also having to read in a TAV file),
; and also because the original Envisat table.dat file does not contain the
; VAR_SI_CONVERSION values. The input parameters have been changed to reflect this: bu
......@@ -920,7 +920,7 @@ PRO read_tablefile, tablefile
;table.dat file used by Envisat or the GEOMS TAV file, with the HDF/netCDF file generated
;accordingly.
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
;History:
; 20050802: Original IDLCR8HDF Routine - Version 1.0
......@@ -1134,7 +1134,7 @@ PRO test_file_input,aname,fentry
;Procedure to test that a file name given as an entry to a free text attribute is valid, based on Envisat
;Metadata Guidelines (not currently used by AVDC). Note: No longer permitted under GEOMS guidelines.
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
; History:
; 20050802: Original IDLCR8HDF Routine - Version 1.0
......@@ -1189,7 +1189,7 @@ PRO geoms_rule_changes, code, in1, in2, in3, in4
;Procedure to check Metadata for old/redundant rules, labels and/or values and update/report
;as required
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
; History:
; 20110401: Introduced. Incorporates GEOMS rules changes from v3.0 to v4.0 - Version 4.01
......@@ -1235,6 +1235,9 @@ PRO geoms_rule_changes, code, in1, in2, in3, in4
; 20200709: Fix in option 5 when looking for obsolete variable names. Change from doing a
; STRPOS search to looking for an exact change for DATA_VARIABLES_01 names due to
; introduction of DRY.AIR.... names to the TAV - Version 4.0b55
; 20201020: Change infotxt status from 1 to 2 in option 5; Change to LIDAR.H2O data source
; if needing to update out-of-date H2O LIDAR DATA_SOURCE values in option 6
; - Version 4.0b56
;
; Inputs: code - Integer value identifying type of check to carry out
; in1 - First set of inputs required for checks (optional, dependent on code value)
......@@ -1312,7 +1315,7 @@ CASE 1 OF
in2[fai]=0 ;Metadata lines not wanted
test1=(STRMID(redundant[i],0,3) EQ 'VAR') OR (STRMID(redundant[i],0,3) EQ 'VIS')
IF test1 THEN att_type='Variable' ELSE att_type='Global'
infotxt='1 '+redundant[i]+' not a GEOMS '+att_type+' attribute|.'
infotxt='2 '+redundant[i]+' not a GEOMS '+att_type+' attribute|.'
infotxt=infotxt+' Removed from the metadata saved to the output file'
INFOTXT_OUTPUT, infotxt
ENDIF
......@@ -1684,9 +1687,9 @@ CASE 1 OF
old_ds=in3
;check special cases for LIDAR water vapor measurements
IF (spc_list[pri] EQ 'H2O.') AND (STRPOS(STRUPCASE(in3),'LIDAR.DIAL') NE -1) THEN $
in3='WATERVAPORDIAL' $
in3='H2O' $
ELSE IF (spc_list[pri] EQ 'H2O.') AND (STRPOS(STRUPCASE(in3),'LIDAR.BACKSCATTER') NE -1) THEN $
in3='WATERVAPORRAMAN' $
in3='H2O' $
ELSE in3=spc_list[pri]
IF STRPOS(in3,'.') NE -1 THEN in3=STRMID(in3,0,STRLEN(in3)-1)
in3=rep_list[oli]+in3+STRMID(old_ds,STRPOS(old_ds,'_'))
......@@ -1946,7 +1949,7 @@ PRO pre_defined_att_checks, ct, nav, vav, vnx, vdtv,verror
;the numeric variable attribute values are of the same data type as that given by the
;corresponding VAR_DATA_TYPE value (if input is via session memory)
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
; History:
; 20101120: Introduced - Version 4.0
......@@ -2081,7 +2084,7 @@ PRO read_metadata, metafile, inf
;2. If redundant attributes are detected they will be omitted from the Metadata written
; to the HDF file.
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
; History:
; 20050802: Original IDLCR8HDF Routine - Version 1.0
......@@ -2127,6 +2130,8 @@ PRO read_metadata, metafile, inf
; any tests - Version 4.0b53
; 20200311: Do not include '.' separator when doing checks for sub-value separators at the start or end of
; the attribute values e.g. allow DO_NAME=Boyd;Ian S. - Version 4.0b54
; 20201020: Generate message for invalid metadata attributes, previously removed them 'quietly'
; - Version 4.0b56
;
; Inputs: metafile - Either, the filename of the input file containing the Metadata or, if program
; input is via string array and Structure, a string array containing the Global
......@@ -2258,6 +2263,16 @@ IF nrline EQ 0 THEN BEGIN
STOP_WITH_ERROR,o3[3]+proname,errtxt[3],lu & RETURN
ENDIF
;Generate an error for invalid metadata attributes
bnri=WHERE(~test1,bnrline)
IF bnrline NE 0 THEN BEGIN
FOR i=0L,bnrline-1L DO BEGIN
infotxt='2 Invalid Metadata Attribute: '+mhhold[bnri[i]]+'|'
infotxt=infotxt+' removed from the metadata saved to the output file'
INFOTXT_OUTPUT,infotxt
ENDFOR
ENDIF
;Set holding arrays for metadata
mh=mh[nri] & mv_lng=mv_lng[nri] & mv_dbl=mv_dbl[nri]
lu=-1
......@@ -2745,7 +2760,7 @@ PRO extract_and_test, dentry, cpos, nel, ta1, ta2, taname, aname, ti
;Procedure called by Check_Metadata to extract the relevant parts of tab_arr and test against
;a corresponding Metadata value.
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
; History:
; 20050802: Original IDLCR8HDF Routine - Version 1.0
......@@ -2886,7 +2901,7 @@ PRO check_metadata
;Procedure to check the validity of Metadata values by checking them against the corresponding
;entries in the Table Attribute Values File
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
; History:
; 20050802: Original IDLCR8HDF Routine - Version 1.0
......@@ -3244,7 +3259,7 @@ END ;Procedure Check_Metadata
PRO extract_data, sds, inf, vc, dtest, ndl, infowrite
;Procedure to extract the set of data corresponding to a particular Variable Attribute
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
; History:
; 20050802: Introduced to IDLCR8HDF, originally part of the READ_DATA procedure - Version 1.0
......@@ -3468,7 +3483,7 @@ PRO set_up_structure, sds, inf
;VAR_FILL_VALUE metadata attributes and set up a structure which will be used to hold the
;data values.
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
; History:
; 20050802: Introduced to IDLCR8HDF, as Determine_Arr_Sizes. Data originally stored in 4
......@@ -3972,7 +3987,7 @@ PRO check_string_datatype, vc, dtest, ndl
;VAR_DATA_TYPE=STRING, including VAR_UNITS, VAR_VALID_MIN/MAX, and VAR_FILL_VALUE. The data
;values are returned in the string datatype.
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
; History:
; 20091203: Introduced to IDLCR8HDF routine - Version 3.08
......@@ -4069,7 +4084,7 @@ PRO check_min_max_fill, vc, dtest, ndl
;the valid minimum and maximum values (or the fill value). The VAR_VALID_MIN value is less than or
;equal to the VAR_VALID_MAX value. The data values are returned in the data type given by VAR_DATA_TYPE.
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
; History:
; 20050802: Original IDLCR8HDF routine - Version 1.0
......@@ -4370,7 +4385,7 @@ PRO read_data, sds, inf
;also adds a comment to VAR_NOTES if averaging kernel data is present, to indicate the
;proper array order (if this option is chosen on program start-up).
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
; History:
; 20050802: Original IDLCR8HDF routine - Version 1.0
......@@ -4414,6 +4429,8 @@ PRO read_data, sds, inf
; 20171121: Change check identifying FTIR templates with incorrect .BOUNDARIES ordering (FTIR
; changed to FTIR-0 to avoid conflict with other FTIR based templates); Change conditions
; for identifying invalid wind speed or direction values - Version 4.0b43
; 20201020: Add check for negative random uncertainty values (standard or relative only)
; - Version 4.0b56
;
; Inputs: sds - Either a structure containing the Variable Attributes and Data, or the input
; data file
......@@ -4792,6 +4809,52 @@ FOR vc=0,N_ELEMENTS(bchks)-1 DO BEGIN
ENDIF
ENDFOR
;Test that random uncertainty values are positive
posun=['UNCERTAINTY.RANDOM.STANDARD','UNCERTAINTY.RANDOM.STANDARD.RELATIVE']
n_p=N_ELEMENTS(posun)
vnx=vn
;vnx consists of only the last sub-name of vn
FOR i=0,nvn-1 DO $
IF STRPOS(vnx[i],'_') NE -1 THEN vnx[i]=STRMID(vnx[i],STRPOS(vnx[i],'_',/REVERSE_SEARCH)+1)
FOR i=0,n_p-1 DO BEGIN ;number of values to check
pi=WHERE(vnx EQ posun[i],pcnt)
IF pcnt NE 0 THEN BEGIN
FOR j=0,pcnt-1 DO BEGIN ;test for negative data values
sdata=(ndm[pi[j],8] EQ 6) ;test for string datatype
IF ~sdata THEN BEGIN
EXTRACT_DATA,sds,inf,pi[j],dtest,ndl,infowrite
IF STRLEN(rerr[0]) GT 2 THEN RETURN
CASE ndm[pi[j],8] OF ;identifies the VAR_DATA_TYPE
0:BEGIN
fv=BYTE(mv_lng[2,pi[j]]) & urd=BYTE(dtest) & zero=0
END
1:BEGIN
fv=FIX(mv_lng[2,pi[j]]) & urd=FIX(dtest) & zero=0
END
2:BEGIN
fv=LONG(mv_lng[2,pi[j]]) & urd=LONG(dtest) & zero=0L
END
3:BEGIN
fv=mv_lng[2,pi[j]] & urd=LONG64(dtest) & zero=0LL
END
4:BEGIN
fv=FLOAT(mv_dbl[2,pi[j]]) & urd=FLOAT(dtest) & zero=0.
END
5:BEGIN
fv=mv_dbl[2,pi[j]] & urd=DOUBLE(dtest) & zero=0.d
END
ENDCASE
ni=WHERE((urd LT zero) AND (urd NE fv),ncnt)
IF ncnt NE 0 THEN BEGIN
infotxt='3 '+vn[pi[j]]+' values cannot be negative'
INFOTXT_OUTPUT,infotxt
ENDIF
ENDIF
ENDFOR
ENDIF
ENDFOR
;Test 'grouped' datasets, i.e. datasets that must both be in the file if one of them is present
;Note that there might be more than one variable in a particular group e.g. LATITUDE and LATITUDE.INSTRUMENT
grouped=[['LATITUDE','LONGITUDE'],['WIND.DIRECTION','WIND.SPEED']]
......@@ -4901,7 +4964,7 @@ PRO find_hdf_filename, hdffilename
; created filename
;3. Create/change the FILE_GENERATION_DATE and ensure it is in ISO8601 format.
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
; History:
; 20050802: Original IDLCR8HDF routine - Version 1.0
......@@ -5161,7 +5224,7 @@ PRO makean, hdf_fn, obj_id, obj_type, label, value
;section (depending on its size). Currently set up to always write the contents
;of the file to HDF_SD as the text file size limit is also 4096 bytes.
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
; History:
; 20050802: Original IDLCR8HDF routine - Version 1.0
......@@ -5245,7 +5308,7 @@ PRO avdc_hdf5_write, hdffilename, natts, av, geoms_output
;and the HDF5 write option will not be available.
;##########################################################################
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
; History:
; 20061012: Introduced to IDLCR8HDF - Version 2.0
......@@ -5420,7 +5483,7 @@ END ;Procedure AVDC_HDF5_Write
PRO avdc_nc_write, hdffilename, natts, av, var_depend, var_size
;IDL subroutine to write AVDC-type global attributes and datasets to a netCDF file.
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
; History:
; 20111120: Introduced to IDLCR8HDF - Version 4.0
......@@ -5623,7 +5686,7 @@ PRO avdc_hdf_write, hdffilename, geoms_output
;file if requested, and calls the AVDC_HDF5_WRITE routine to create the HDF5 file if
;that option is chosen.
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
; History:
; 20050802: Originally split into two routines to separately write the Global
......@@ -5868,9 +5931,9 @@ PRO idlcr8hdf, ga, sds, tav, odir, reterr, H5=o1, AVK=o2, LOG=o4, POPUP=o5, QA=o
;
;Program documentation, idlcr8hdf-v4.0_Readme.pdf, available from http://avdc.gsfc.nasa.gov.
;
;Program sub-version 4.0b55 (20200709)
;Program sub-version 4.0b56 (20201020)
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;Written by Ian Boyd for the EVDC/AVDC - iboyd@astro.umass.edu
;
; History:
; 20050802: Original Release - Version 1.0
......@@ -6494,7 +6557,7 @@ IF o3[3] EQ '' THEN BEGIN
WIDGET_CONTROL,wtxt,set_value=endtxt+'completed - hit <Finish> to close program',/Append
XMANAGER,'idlcr8hdf',base
ENDIF ELSE IF intype LT 0 THEN BEGIN ;Create Finish Dialog Box
res=DIALOG_MESSAGE(endtxt+'completed successfully!',/Information,Title='AVDC IDLcr8HDF')
res=DIALOG_MESSAGE(endtxt+'completed successfully!',/Information,Title='EVDC IDLcr8HDF')
ENDIF
END ;Procedure IDLcr8HDF
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment