Skip to content

GitLab

Commit 398971a7 authored by Ian Boyd's avatar Ian Boyd
Browse files

Upload New File

parent b3501527
Hide whitespace changes
Inline Side-by-side
cds_convert.pro 0 → 100644
View file @ 398971a7
;---------------------------------------------------------------------------------------------------
;
; name: cds_convert.pro
; description: Wrapper for AVDC/EVDC CDS tool
;
; 2011-03-31, v1.0 Christian Retscher rename to cds_convert
; 2012-01-12, v2.0b14 Ian Boyd combine option to do multiple files; add GUI when no arguments
; are given
; 2012-06-14 v2.0b21 Ian Boyd align version with CDS_TOOLS. New CDS_TOOLS to provide SHADOZ
; conversion support
; 2012-07-03 v2.0b22 Ian Boyd align version with CDS_TOOLS. New CDS_TOOLS provides support for
; NDACC NASA/AMES format 2160
; 2012-07-18 v2.0b23 Ian Boyd align version with CDS_TOOLS. New CDS_TOOLS compares launch date
; in file with date in filename for NDACC ozonesonde files. Also modifies
; checks done on comments in NDACC files
; 2012-09-13 v2.0b24 Ian Boyd align version with CDS_TOOLS. New CDS_TOOLS accounts for
; modification to Sonde translation tables, which now include all
; AUXILIARY_PARAMETERS described in NDACCs discussion document. Also allows
; for 2 or more differences in Archive file label names (separated by '|')
; 2012-10-31 v2.0b25 Ian Boyd align version with CDS_TOOLS. New CDS_TOOLS accounts for numerous
; DATETIME issues in the WOUDC datasets. Version number also incremented to
; indicate changes in AmesToPointer.pro to account for v003 of the Lidar templates
; 2012-11-27 v2.0b26 Ian Boyd align version with CDS_TOOLS. New CDS_TOOLS includes support for
; NOAA-ESRL files and changes the SOURCE.PRODUCT entry to
; Archive.Source_Archive.File_Archive.File.Generation.Date - v2.0b26
; 2013-03-26 v2.0b27 Ian Boyd align version with CDS_TOOLS. Includes fixes to external Lidar
; conversion routines to account for new templates; Account for DATETIME issues
; in NDACC datasets similar to those applied to other datasets; Write NDACC files
; to a temporary file so that hidden characters affecting XCHGREAD.PRO are removed;
; Includes fixes to XCHGREAD.PRO to allow for fixes to the header information in
; many of the McMurdo Sonde files
; 2013-11-10 v2.0b28 Ian Boyd align version with CDS_TOOLS. Includes changes to NDACC_Lidar_HDF.pro
; and AmesToPointer.pro to carry out checks on Reunion files to determine whether
; measurements are made from St.Denis or Maido sites
; 2014-03-31 v2.0b29 Ian Boyd align version with CDS_TOOLS. Includes changes to idlcr8ascii.pro and
; idlcr8hdf.pro to fix bugs and AmesToPointer.pro to add Rio Galegos and newer
; Dumont and Ny-Alesund
; 2014-05-22 v2.0b30 Ian Boyd align version with CDS_TOOLS. Includes changes to idlcr8hdf.pro to
; acount for change in the *.BOUNDARIES dimension ordering
; 2014-08-11 v2.0b31 Ian Boyd changes to AmestoPointer.pro to account for change in PI at TMO and
; MLO, and updated idlcr8hdf.pro (v4.0b22)
; 2014-09-10 v2.0b32 Ian Boyd changes to idlcr8hdf.pro to account for obsolete DATA_SOURCE values
; when harmonizing measurements (v4.0b23)
; 2014-10-07 v2.0b33 Ian Boyd Fix in cds_tools to account for WOUDC measurements for Singapore
; where the GPH is given in feet instead of metres
; 2015-01-27 v2.0b34 Ian Boyd Changes to idlcr8hdf.pro and idlcr8ascii.pro to account for changes
; in handling of STRING datasets
; 2015-02-17 v2.0b35 Ian Boyd Changes to idlcr8hdf.pro and idlcr8ascii.pro to account for changes
; in handling of STRING datasets
; 2015-03-02 v2.0b36 Ian Boyd Fix bug in idlcr8hdf.pro
; 2015-04-09 v2.0b37 Ian Boyd Fix bugs in idlcr8hdf.pro and add Reunion Maido Lidar site
; information to AmestoPointer.pro
; 2015-08-11, v2.0b38 Ian Boyd Update due to bug fixes and improvements in idlcr8hdf
; 2015-09-26, v2.0b39 Ian Boyd Update due to bug fixes and improvements in idlcr8hdf
; 2015-10-21, v2.0b40 Ian Boyd Update due to bug fixes and improvements in idlcr8hdf
; 2015-11-04, v2.0b41 Ian Boyd Update due to fix in idlcr8hdf
; 2015-11-09, v2.0b42 Ian Boyd Update due to bug fixes and improvements in idlcr8hdf and idlcr8ascii
; 2015-11-16, v2.0b43 Ian Boyd Update due to fix in idlcr8hdf
; 2015-12-15, v2.0b44 Ian Boyd Update due to fix in idlcr8hdf
; 2016-02-13, v2.0b45 Ian Boyd Add checks for Wind Direction and speed if either of them are 0.0.
; 2016-10-11, v2.0b46 Ian Boyd Fixes to AmestoPointer.pro to account for new versions of the lidar
; templates; incorporate updates to idlcr8ascii and idlcr8hdf
; 2016-11-16, v2.0b47 Ian Boyd Update due to fix in idlcr8hdf
; 2016-11-30, v2.0b48 Ian Boyd Update due to bug fixes and improvements in idlcr8hdf and idlcr8ascii
; 2016-12-17, v2.0b49 Ian Boyd Update due to fixes in idlcr8ascii
; 2016-12-30, v2.0b50 Ian Boyd Update due to improvements and fixes to idlcr8hdf. ORIGINATOR attribute
; no longer part of the TAV file from 04R025
; 2017-03-31, v2.0b51 Ian Boyd Update due to improvemnets to idlcr8hdf
; 2017-11-21, v2.0b52 Ian Boyd Update due to fixes in idlcr8hdf
; 2018-06-22, v2.0b53 Ian Boyd Fix to cds_tools.pro to read SHADOZ station name from file when there
; is an extra comment in the entry (e.g. Hanoi, Vietnam (...)).
; Include updates made in idlcr8hdf and idlcr8ascii
; 2018-08-16, v2.0b54 Ian Boyd Fix for incorrect calculation of O3 number density when the insitu
; temperatures are in K. Previously assumed the temperatures were always
; in Celcius (affected NDACC NASA Ames only)
; 2018-09-01, v2.0b55 Ian Boyd Previous fix caused crash when checking non-NDACC measurements.
; Fixed in this version
;---------------------------------------------------------------------------------------------------
;---------------------------------------------------------------------------------------------------
PRO cds_convert_common
;Procedure to define the COMMON blocks for this program
; ----------
;Written by Ian Boyd for the AVDC/EVDC - iboyd@astro.umass.edu
;
;History:
; 20120112: Introduced to cds_convert - Version 2.0b14
;
;Input: Nil
;
;Output: Nil
;
;Called by: N/A
;
;Subroutines Called: None
COMMON WIDGET_WIN_CDS, o3
END ;Procedure idlcr8hdf_common
PRO intro_event, ev
;Procedure to define how Events from the Start-up Introduction Window are handled
; ----------
;Written by Ian Boyd for the AVDC/EVDC - iboyd@astro.umass.edu
;
;History:
; 20120112: Introduced to cds_convert - Version 2.0b14
;
;Input: ev - Selected widget event structure
;
;Output: o3 - Common string array defining the various options for program output
;
;Called by: XMANAGER in INTRO
;
;Subroutines Called: None
COMMON WIDGET_WIN_CDS
;The uservalue is retrieved from a widget when an event occurs
WIDGET_CONTROL,ev.id,GET_UVALUE=o3
WIDGET_CONTROL,ev.top,/DESTROY
END ;Intro_Event
PRO intro
;Procedure which creates an Introduction Window at start-up when cds_convert is called without parameters,
;or invalid parameters. The user has a choice of continuing with the program after selecting input options,
;or stopping the program.
; ----------
;Written by Ian Boyd for the AVDC/EVDC - iboyd@astro.umass.edu
;
;History:
; 20120112: Introduced to cds_convert - Version 2.0b14
; 20120614 onwards: Align version with cds_tools - Version 2.0b21 onwards
;
;Input: Nil
;
;Output: Nil
;
;Called by: cds_convert
;Subroutines Called: INTRO_EVENT (via XMANAGER)
nhdr=22 & errtxt=STRARR(nhdr)
vertxt=['cds_convert_Readme.pdf',' Release v2.0b55, September 2018']
errtxt[1]='Conversion Data Suite (CDS): cds_convert'
errtxt[2]=vertxt[1]
errtxt[3]=' Converts supported original data into GEOMS compliant HDF data.'
errtxt[4]=' For a list of supported data files, please contact a GEOMS compliant data center.'
errtxt[6]='Usage:'
errtxt[7]='Single file option (S):
errtxt[8]=' idl -rt=cds_convert.sav -args [GEOMSTEFILE] [TRANSFILE] [TAVFILE] [OUTPUTDIR] [DATAFORMAT] [ORIGFILE]'
errtxt[9]='Multiple file option (M) - requires output file created by cds_match.sav:
errtxt[10]=' idl -rt=cds_convert.sav -args [CDSMATCHINPUT] [TAVFILE] [OUTPUTDIR] [DATAFORMAT]'
errtxt[11]='Note: Windows syntax is idlrt [-vm] cds_convert.sav -args ...'
errtxt[13]='1 (S): GEOMSTEFILE is the path to the GEOMS template file'
errtxt[14]='1 (M): CDSMATCHINPUT is the path to the output of cds_match with a list of matching files'
errtxt[15]='2 (S): TRANSFILE is the path to the CDS compliant translation file'
errtxt[16]='3: TAVFILE is the path of the TAV file, you want to convert with'
errtxt[17]='4: OUTPUTDIR is the path of the output directory of the converted file'
errtxt[18]='5: DATAFORMAT: 0: HDF4; 1: HDF5; 2: netCDF'
errtxt[19]='6 (S): ORIGFILE is the path to the original file, you want to convert'
errtxt[21]='To continue, please choose from the options below (Note: HDF5 only available on IDL6.2 or greater).'
errtxt=' '+errtxt
;Set-up text display widget
base=WIDGET_BASE(Title='CDS_Convert '+vertxt[1],Tlb_Frame_Attr=1,/Column) ;,Tab_Mode=1)
wtxt=WIDGET_TEXT(base,xsize=100,ysize=nhdr+1,/Scroll)
base2=WIDGET_BASE(base,/Row)
b1=WIDGET_BUTTON(base2,value='HDF4 (Single)',uvalue='S',frame=3) ;,Tooltip=Tip)
IF FLOAT(!Version.Release) GE 6.2 THEN $
b2=WIDGET_BUTTON(base2,value='HDF5 (Single)',uvalue='h5S',frame=3) $;,Tooltip=Tip) $
ELSE b2=WIDGET_BUTTON(base2,value='HDF5 (Single)',Sensitive=0,frame=3)
b4=WIDGET_BUTTON(base2,value='netCDF (Single)',uvalue='ncS',frame=3)
b5=WIDGET_BUTTON(base2,value='HDF4 (Multiple)',uvalue='M',frame=3) ;,Tooltip=Tip)
IF FLOAT(!Version.Release) GE 6.2 THEN $
b6=WIDGET_BUTTON(base2,value='HDF5 (Multiple)',uvalue='h5M',frame=3) $;,Tooltip=Tip) $
ELSE b6=WIDGET_BUTTON(base2,value='HDF5 (Multiple)',Sensitive=0,frame=3)
b7=WIDGET_BUTTON(base2,value='netCDF (Multiple)',uvalue='ncM',frame=3)
b3=WIDGET_BUTTON(base2,value='Stop',uvalue='CANCEL',frame=3) ;,ToolTip=Tip)
WIDGET_CONTROL,base,/Realize
FOR i=0,N_ELEMENTS(errtxt)-1 DO $
WIDGET_CONTROL,wtxt,set_value=errtxt[i],/Append
XMANAGER,'intro',base
END ;Intro
function is_a_number,value
on_ioerror, ConversionError
n=DOUBLE(value)
return,1B
ConversionError:
return, 0B
END
;---------------------------------------------------------------------------------------------------
PRO cds_convert
COMMON WIDGET_WIN_CDS
args = command_line_args()
iNrArgs = size(args, /N_ELEMENTS)
CDS_error_code = 3
if ( iNrArgs ne 6 ) and ( iNrArgs ne 4 ) then begin
o3='CANCEL'
INTRO
if o3 eq 'CANCEL' then begin
status=CDS_error_code
endif else begin ;Open File and Directory Selection Windows
CD,CURRENT=path ;identify current working directory
if strmid(o3,strlen(o3)-1,1) eq 'S' then begin
iNrArgs=6
chFileGEOMSTE=DIALOG_PICKFILE(Filter=['GEOMS*.csv'], PATH=path, $
/MUST_EXIST,Title='Select GEOMS Template File')
if chFileGEOMSTE eq '' then stop, status=CDS_error_code
path=FILE_DIRNAME(chFileGEOMSTE,/MARK_DIRECTORY)
if strpos(STRUPCASE(chFileGEOMSTE),'LIDAR') ne -1 then begin
chFileTranslation=DIALOG_PICKFILE(/DIRECTORY, $
Title='Select Directory containing Lidar Translation File', PATH=path)
if chFileTranslation ne '' then path=chFileTranslation
endif else begin
chFileTranslation=DIALOG_PICKFILE(Filter=['*translation*.txt'], PATH=path, $
/MUST_EXIST,Title='Select Translation File')
if chFileTranslation ne '' then path=FILE_DIRNAME(chFileTranslation,/MARK_DIRECTORY)
endelse
if chFileTranslation eq '' then stop, status=CDS_error_code
path=FILE_DIRNAME(chFileTranslation,/MARK_DIRECTORY)
chFileOrig=DIALOG_PICKFILE(PATH=path,/MUST_EXIST,Title='Select Original File to be Converted')
if chFileOrig eq '' then stop, status=CDS_error_code
path=FILE_DIRNAME(chFileOrig,/MARK_DIRECTORY)
endif else begin
iNrArgs=4
chFilesList=DIALOG_PICKFILE(PATH=path, $
/MUST_EXIST,Title='Select output of cds_match with a list of Matching Files')
if chFilesList eq '' then stop, status=CDS_error_code
path=FILE_DIRNAME(chFilesList,/MARK_DIRECTORY)
endelse
;Select common files/directories
chFileTAV=DIALOG_PICKFILE(Filter=['tableattrvalue*.dat'], PATH=path, $
/MUST_EXIST,Title='Select GEOMS Table Attribute Values File')
if chFileTAV eq '' then stop, status=CDS_error_code
path=FILE_DIRNAME(chFileTAV,/MARK_DIRECTORY)
chDirOutput=DIALOG_PICKFILE(/DIRECTORY, $
Title='Select Output Directory of the Converted File', PATH=path)
if chDirOutput eq '' then stop, status=CDS_error_code
if strlen(o3) eq 3 then chDataFormat=strmid(o3,0,2) else chDataFormat=''
CDS_error_code = 0
endelse
endif else begin
CDS_error_code = 0
if iNrArgs eq 6 then begin ;Single File Option
chFileGEOMSTE = STRTRIM(args(0), 2)
chFileTranslation = STRTRIM(args(1), 2)
chFileTAV = STRTRIM(args(2), 2)
chDirOutput = STRTRIM(args(3), 2)
iDataFormat = STRTRIM(args(4), 2)
chFileOrig = STRTRIM(args(5), 2)
if ( file_test( chFileOrig ) eq 0 ) then begin
CDS_error_code = 3
print, 'ERROR: Original file not found.'
end
if ( file_test( chFileGEOMSTE ) eq 0 ) then begin
CDS_error_code = 3
print, 'ERROR: GEOMS template file not found.'
end
if ( file_test( chFileTranslation ) eq 0 ) then begin
CDS_error_code = 3
print, 'ERROR: CDS compliant translation file file not found.'
end
endif else begin ;Multiple file option
chFilesList = STRTRIM(args(0), 2)
chFileTAV = STRTRIM(args(1), 2)
chDirOutput = STRTRIM(args(2), 2)
iDataFormat = STRTRIM(args(3), 2)
if ( file_test( chFilesList ) eq 0 ) then begin
CDS_error_code = 3
print, 'ERROR: File with list of matching files not found.'
end
endelse
;Do checks on common arguments
if ( file_test( chFileTAV ) eq 0 ) then begin
CDS_error_code = 3
print, 'ERROR: TAV file not found.'
end
if ( file_test( chDirOutput ) eq 0 ) then begin
CDS_error_code = 3
print, 'ERROR: Output directory not found.'
end
if not ( is_a_number( iDataFormat ) ) then begin
CDS_error_code = 3
print, 'ERROR: Data Format option must be numeric [0|1|2].'
end else begin
if ( ( iDataFormat lt 0 ) or ( iDataFormat gt 2 ) ) then begin
CDS_error_code = 3
print, 'ERROR: Data Format option must have values: [0|1|2].'
endif else begin
case fix(iDataFormat) of
0: chDataFormat=''
1: chDataFormat='h5'
2: chDataFormat='nc'
endcase
endelse
end
endelse
if ( CDS_error_code eq 0 ) then begin
if ( iNrArgs eq 6 ) then begin ;Single File Input
; check if files are LIDAR, then use Thierry LeBlanc's routines instead of generic AVDC routines
chFileOrigBase = file_basename( chFileOrig )
chFileOrigBaseArr= STRSPLIT(chFileOrigBase, '.', ESCAPE='\', /EXTRACT)
iSize1 = 1L
endif else begin ;Multiple File Input
; read intut matches list
statusflag='' & dum='' & exit_code=3
flist=[''] & n_el=0L & lfflag=''
TEST_INPUT,chFilesList,chFileTAV,chDirOutput,flist,n_el,statusflag,lfflag
iSize1Dim = size(flist, /DIMENSIONS)
iSize1NrDim = size(flist, /N_DIMENSIONS)
if (iSize1NrDim eq 1) then begin
iSize1 = 1L
end else begin
iSize1 = iSize1Dim(1)
end
;print, iSize1NrDim, iSize1Dim
endelse
for iR1 = 0L, iSize1 - 1L do begin
if iNrArgs eq 4 then begin
chFileOrig = flist(0, iR1)
chFileGEOMSTE = flist(1, iR1)
chFileTranslation = flist(2, iR1)
endif
if (FILE_TEST(chFileOrig) eq 1) and (FILE_TEST(chFileGEOMSTE) eq 1) and $
(FILE_TEST(chFileTranslation) eq 1) then begin
print, chFileOrig
;print, chFileOrig+' '+chFileGEOMSTE+' '+chFileTranslation
; check if files are LIDAR, then use Thierry LeBlanc's routines instead of generic AVDC routines
chFileOrigBase = file_basename( chFileOrig )
chFileOrigBaseArr= STRSPLIT(chFileOrigBase, '.', ESCAPE='\', /EXTRACT)
;print,chFileOrigBaseArr
if (strlen(chFileOrigBaseArr(1)) eq 3) and (STRMID(strlowcase(chFileOrigBaseArr(1)), 2, 1) eq 'l' ) then begin
print, "CDS: LIDAR data processing"
CDS_error_code=cds_LIDAR( chFileOrig, chFileGEOMSTE, chFileTranslation, chFileTAV, $
chDirOutput, chDataFormat )
end else begin
chFilesList = STRARR(3)
chFilesList(0) = chFileOrig
chFilesList(1) = chFileGEOMSTE
chFilesList(2) = chFileTranslation
if ( chDataFormat eq '' ) then begin
CDS_error_code=cds_tools( chFilesList, chFileTAV, chDirOutput ) ;DATETIME=o1)
end else begin
CDS_error_code=cds_tools( chFilesList, chFileTAV, chDirOutput, chDataFormat )
end
end
end else begin
print, 'INFORMATION: One or more input files not valid.'
end
end
end
print, CDS_error_code
END
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment