idlcr8ascii.pro 114 KB
Newer Older
Ian Boyd's avatar
Ian Boyd committed
1
;Main Program Version: idlcr8ascii.pro v4.0b21, 20190514
Ian Boyd's avatar
Ian Boyd committed
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
;  Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;
;Sub-versions:
;  v3.01, 20081020 - If the HDF4 file is created using the HDF4.2r3 library, then extra information
;                    regarding the dimensions of the VAR_DEPEND values may be included in the file
;                    (which show up as extra datasets in the HDF_SD_FILEINFO call). A check for
;                    this is carried out and, if found, the information is excluded from the output.
;  v3.02, 20090311 - Improve HDF4.2r3 library checks by comparing the number of datasets recorded
;                    under DATA_VARIABLES, with the number returned by the HDF_SD_FILEINFO call;
;                    Change 'WARNING' to 'INFORMATION; Add procedure to handle 'INFORMATION' text
;                    output; Add 'INFORMATION' text if anything irregular found with the datasets;
;                    Change 'Data dimensions exceed 8' and 'Data type not allowable' ERROR messages
;                    to 'INFORMATION' messages. Make log file output to the same directory as the
;                    first HDF file to be read.
;  v3.03, 20091208 - Incorporate HDF_SD_ISCOORDVAR to differentiate between datasets and dimension
;                    variable names. Add check for allowable VAR_DATA_TYPE=STRING. Add all
;                    INFORMATION text messages to INFOTXT_OUTPUT_A procedure to avoid duplication
;                    of code that creates the messages; Add INFORMATION messages when reading HDF5
;                    files; Rename AVDC Button to FORMAT and replace keyword option /AVDC with
;                    /FORMAT. Improve checks on parameter and keyword inputs; Improve checks for
;                    non-standard (i.e. non-groundbased) HDF files.
;  v4.0b1, 20101122  - Adopt GEOMS metadata standard; Because FORMAT attribute(s) have been dropped
;                    can no longer use defined formatting values when outputing data to files;
;                    Default ASCII formatting adopted - scientific notation for REAL and DOUBLE,
;                    except for DATETIME and related values, which have formatting dependent on
;                    the data type; Numeric metadata values remain in their original data type
;                    when writing to session memory, meaning: variable attribute labels and values
;                    now written to separate structure variables sds.va_l and sds.va_v (previously
;                    sds.va), and structure has 2 dimensions (n_sds, n_atts) instead of 1
;                    (previously n_sds); The dataset is written to the initial n_atts index (i.e.
;                    sds[n_sds,0].data); Rename saved SDS datatype according to GEOMS rules
;                    (affects INT, LONG, and FLOAT); Add JDF_2_DATETIME and JULIAN_DATE routines
;                    to the program; Include coordinate variables if they and the non-coordinate
;                    variables add up to the number of saved DATA_VARIABLES. Add return error code
;                    option so that program returns to the calling program if an error is
;                    generated.
;  v4.0b2, 20110323  - If required reverse the order of VAR_SIZE and VAR_DEPEND values so that they
;                    match the order of the multi-dimensional dataset. Note that the HDF read programs
;                    automatically transpose the multi-dimensional datasets to conform to the IDL
;                    convention of fastest changing index being listed first (opposite for HDF).
;  v4.0b3, 20111014  - Add netCDF support - writes variable attributes to GEOMS standard attributes
;                    where possible, otherwise writes to VAR_NOTES
;  v4.0b4, 20111208  - Test for dimension ordering of the netCDF files, and reverse if required (same
;                    rules as for HDF). Create separate Dimension Ordering Routine
;  v4.0b5, 20120328  - Add options to convert HDF/netCDF files to other HDF/netCDF formats e.g. H4 to
;                    H5 etc. Requires IDLcr8HDF in the search path to do this.
;  v4.0b6, 20120426  - Bug fix when a netCDF file only contains datasets with single dimensions -
;                    VAR_SIZE information was not being extracted during the read process.
;  v4.0b7  20130114  - Add check for IDL being run in DEMO mode by using LMGR command. Replace PRINTF,-1
;                    statements (i.e. when dux[0] eq -1) with PRINT (o/w causes DEMO mode to stop). Also
;                    disable /LOG, /FORMAT and /DUMP keywords in DEMO mode.
;  v4.0b8  20140325  - Fix bug that meant VAR_SIZE and VAR_DEPEND values were not being automatically
;                    reversed if the array sizes were the same and VAR_DEPEND did not include a DATETIME
;                    value. Now defaults to the 'reverse' option in the TEST_DIM_ORDER routine if no
;                    other conditions are satisfied; Fix bug that caused crash when VAR_SIZE values were
;                    not of type string.
;  v4.0b9  20141110  - Fix bug that caused multi-dimensional array ordering to not be correctly identified
;                    if the first dataset to be checked in the file had the same number of elements in
;                    the array (e.g. was a set of Averaging Kernels).
;  v4.0b10 20150127  - Allow for VAR_FILL_VALUE values for string datasets to be written and saved as an
;                    (empty) string of the correct length. Ensure string datasets are all written and
;                    saved to the correct length (that of the longest string in the dataset).
;  v4.0b12 20151109  - Convert DATETIME (and related) max, min and fill values to LONG64 data type before
;                    determining the format for ASCII output to account for very large values.
;  v4.0b15 20160725  - Fix bug causing program to crash when the number of DATA_VARIABLES values is greater
;                    than the number of datasets in the file.
;  v4.0b19 20180218  - Fix bug that caused multi-dimensional array ordering to not be correctly identified
;                    if the datasets have the same VAR_SIZE i.e. rev_vd_vs stays as 2 through all the
;                    checks. Now rev_vd_vs changes to 1 (VAR_SIZE, VAR_DEPEND and dataset ordering is 
;                    reversed) if rev_vd_vs=2 after all the checks.
;  v4.0b20 20181116  - Fix bug that caused multi-dimensional array ordering to not be correctly written if
;                    not consistent through the file i.e. rev_vd_vs eq 3. Now rev_vd_vs changes to 1 
;                    (VAR_SIZE, VAR_DEPEND and dataset ordering is reversed), but information message 
;                    advising of the issue is still reported.
Ian Boyd's avatar
Ian Boyd committed
76
77
78
;  v4.0b21 20190514  - Fix bug that occurred when GEOMS variable attribute information is missing from an 
;                    HDF4 file but written to the heap structure based on the contents of the file. Previously
;                    the variable attribute label was not written correctly.
Ian Boyd's avatar
Ian Boyd committed
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197

PRO idlcr8ascii_common
;Procedure to define the data COMMON block WIDGET_WIN_A, containing common variables
;associated with the Graphical User Interface
; ----------
;  Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;
;  History:
;    20061004: Introduced to IDLCR8ASCII - Version 2.0
;    20080302: 'cwidg' variable removed, and 'dux' array added to identify requested
;              output options - Version 3.0
;    20101122: Add rerr - Version 4.0b1
;
;
;  Input: Nil
;
;  Output: Nil
;
;  Called by: N/A
;
;  Subroutines Called: None

COMMON WIDGET_WIN_A,wtxt,lineno,base,o3,b3,dux,rerr

END ; Procedure idlcr8ascii_common



PRO intro_a_event, ev
;Procedure to define how Events from the Start-up Introduction Window are handled
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;
;  History:
;    20061004: Introduced to IDLCR8ASCII - Version 2.0
;    20080302: Code added to define the Pop-up Window and Log Output events added to
;              the Introduction Window - Version 3.0
;
;  Input: ev - Selected widget event structure
;
;  Output: o3 - Common string array defining the various options for program output
;
;  Called by: XMANAGER in INTRO_A
;
;  Subroutines Called: None

COMMON WIDGET_WIN_A

;The uservalue is retrieved from a widget when an event occurs
WIDGET_CONTROL,ev.id,GET_UVALUE=uv
IF (uv EQ 'C') OR (uv EQ 'P') OR (uv EQ 'idlcr8ascii.log') OR $
   (uv EQ 'H4') OR (uv EQ 'H5') OR (uv EQ 'NC') THEN BEGIN
  CASE 1 OF
    uv EQ 'C': BEGIN
                 uvi=1 & eadd=12L
               END
    uv EQ 'P': uvi=3
    uv EQ 'H4': BEGIN
                  uvi=5 & eadd=6L
                END
    uv EQ 'H5': BEGIN
                  uvi=6 & eadd=5L
                END
    uv EQ 'NC': BEGIN
                  uvi=7 & eadd=4L
                END
    ELSE: uvi=4
  ENDCASE
  IF o3[uvi] EQ uv THEN o3[uvi]='0' ELSE o3[uvi]=uv
  IF (uv EQ 'C') OR (uv EQ 'H4') OR (uv EQ 'H5') OR (uv EQ 'NC') THEN BEGIN
    IF (o3[1] EQ '0') AND (o3[5] EQ '0') AND (o3[6] EQ '0') AND (o3[7] EQ '0') THEN $
      WIDGET_CONTROL,ev.id+eadd,Sensitive=0 $
    ELSE BEGIN
      WIDGET_CONTROL,ev.id+eadd,Sensitive=1
      IF uv EQ 'C' THEN BEGIN
        WIDGET_CONTROL,ev.id+1,/Set_Button
        o3[3]='P'
      ENDIF
    ENDELSE
  ENDIF
ENDIF ELSE IF (uv EQ 'F') OR (uv EQ 'D') THEN o3[0]=uv $ ;Assign button event to a variable name
ELSE IF uv EQ 'Cont' THEN o3[0]='0' ;changes o3[0] from -1 to 0
IF (uv NE 'C') AND (uv NE 'P') AND (uv NE 'idlcr8ascii.log') AND (uv NE 'H4') AND $
   (uv NE 'H5') AND (uv NE 'NC') THEN WIDGET_CONTROL,ev.top,/DESTROY

END ;Intro_Event handler



PRO intro_a, intype
;Procedure which creates an Introduction Window at start-up when IDLCR8ASCII is called without parameters,
;or invalid parameters, or is called by IDL Virtual Machine. The user has a choice of continuing with
;the program after selecting input options, or stopping the program.
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;
;  History:
;    20061004: Introduced to IDLCR8ASCII - Version 2.0
;    20080302: Swapped the order of the option windows, so that the options indicated by the check
;              boxes appear above the option boxes that close the Introduction Window and continue
;              with the program; Added options to allow the user to send input/output log information
;              to a Pop-Up Window and/or a Log File - Version 3.0
;    20091208: Change AVDC button to FORMAT and keyword option from /AVDC to /FORMAT - Version 3.03
;    20101122: Add vertxt array. Modify Introduction text to indicate that program accepts 'GEOMS
;              Compliant' files, and also to show the new structure format - Version 4.0b1
;
;  Input: intype - Integer set to -2 or -3: -2 indicates normal state; -3 indicates that input
;                  parameters or keywords at the IDLCR8ASCII call are invalid.
;
;  Output: Nil
;
;  Called by: IDLCR8ASCII
;
;  Subroutines Called: INTRO_A_EVENT (via XMANAGER)

COMMON WIDGET_WIN_A

;procedure which provides an introduction message before starting the program.
nhdr=46 & errtxt=STRARR(nhdr)
Ian Boyd's avatar
Ian Boyd committed
198
vertxt=['idlcr8ascii-v4.0_Readme.pdf','v4.0b21 May 2019']
Ian Boyd's avatar
Ian Boyd committed
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
errtxt[1]='Welcome to IDLcr8ASCII.  This program reads GEOMS compliant HDF4, HDF5 and netCDF files and'
errtxt[2]='saves contents to either session memory, an output window (summary only) or to ASCII or formatted'
errtxt[3]='files (also refer to '+vertxt[0]+').'
errtxt[5]='Inputs to the program:'
errtxt[6]='  HDF/netCDF FILE(s) - GEOMS compliant HDF4, HDF5 or netCDF files.  Input can be by DIALOG_BOX'
errtxt[7]='    (IDL VM or full version of IDL), or by command line (full version of IDL only).'
errtxt[9]='Choice of output is made by selecting options in this DIALOG_BOX (IDL VM or full version'
errtxt[10]='  of IDL), or by ''Keywords'' at the command line input (full version only), as follows:'
errtxt[11]='  /F or /FORMAT - generates two output files (with .META and .DATA extensions).  The resulting'
errtxt[12]='    files can be used as inputs to GEOMS compliant write programs (IDLcr8HDF etc), OR'
errtxt[13]='  /D or /DUMP - generates two output files (with .META and .DATA extensions).  Data values'
errtxt[14]='    will be shown as indicated in the array format defined by VAR_SIZE, AND/OR'
errtxt[15]='  /C or /CATALOG - sends variable index, variable name, data format, and dimension size'
errtxt[16]='    information to an output window and/or log file, AND/OR'
errtxt[17]='  /P or /POPUP - sends log input/output information to a Pop-up Dialog Box, AND/OR'
errtxt[18]='  /L or /LOG - sends log input/output information to a file named idlcr8ascii.log'
errtxt[19]='  /H4 - outputs contents of the input file as an HDF4 file'
errtxt[20]='  /H5 - outputs contents of the input file as an HDF5 file'
errtxt[21]='  /NC - outputs contents of the input file as a netCDF file'
errtxt[23]='Example of command line input:  idlcr8ascii,DFSpec[,/F][,/D][,/C][,/P][,/L][,/H4][,/H5][,/NC]'
errtxt[24]='  where ''DFSpec'' can either be a string array containing filenames or a scalar string containing a'
errtxt[25]='  filespec or a single file name.'
errtxt[26]='If running the full version of IDL, and input is a single HDF file, output can also be saved'
errtxt[27]='  to session memory.  This can be done by calling idlcr8ascii with the following command line:'
errtxt[28]='  idlcr8ascii,DFFile,GA,SDS[,/F][,/D][,/C][,/P][,/L][,/H4][,/H5][,/NC].'
errtxt[29]='DFFile can either be the name of an input file or '''' (in which case a DIALOG_BOX will open'
errtxt[30]='  and prompt for the input filename).  ''GA'' is a returned string array containing the Global'
errtxt[31]='  Attributes, and ''SDS'' is a returned structure using pointers containing the Variable'
errtxt[32]='  Attribute Labels (sds.va_l), Variable Attribute Values (sds.va_v), and the Data (sds.data).
errtxt[34]='Contacts -'
errtxt[35]='  Ian Boyd, University of Massachusetts (iboyd@astro.umass.edu)'
errtxt[36]='  Department of Astronomy, 619 Lederle GRC, University of Massachusetts'
errtxt[37]='  710 North Pleasant St, Amherst, MA 01002, USA'
errtxt[39]='  Ghassan Taha, AVDC Project Coordinator (ghassan.taha@nasa.gov)'
errtxt[40]='  NASA Goddard Space Flight Center, Code 613.3'
errtxt[41]='  Greenbelt, MD 20771, USA'
errtxt[43]='AVDC Website: Tools and documentation available from http://avdc.gsfc.nasa.gov/Overview/index.html'
errtxt[45]='To continue, please choose from the options below.'
errtxt='        '+errtxt

;Set-up text display widget
demomode=LMGR(/DEMO) & optsens=1
IF intype EQ -3 THEN xtxt=' - Command Line Input Error' ELSE xtxt=''
base=WIDGET_BASE(Title='idlcr8ascii '+vertxt[1]+xtxt,Tlb_Frame_Attr=1,/Column) ;,Tab_Mode=1)
wtxt=WIDGET_TEXT(base,xsize=88,ysize=25,/Scroll)

tip='Left Mouse Click or Tab to entry and hit <Spacebar>'
base2=WIDGET_BASE(base,/Nonexclusive)
cattxt='Catalog - Generate a summary of the contents of the input file(s) and send to an output window or file'
poptext='Open a Pop-Up Window to display log input/output (default if ''Catalog'' option is chosen)'
logtext='Append log input/output to the file ''idlcr8ascii.log'' '
IF demomode THEN BEGIN
  logtext=logtext+'(No log or netCDF file output permitted in IDL DEMO Mode)' & optsens=0
ENDIF ELSE logtext=logtext+'(will create the file IF it doesn''t exist)'
h4text='Convert input file(s) to HDF4'
h5text='Convert input file(s) to HDF5'
nctext='Convert input file(s) to netCDF'
fctext1='For the following options, any existing files will be overwritten, IDLcr8HDF must be in the IDL Search Path,'
fctext2='and a valid Table Attribute Values (TAV) file must be in the same directory as the Input file(s).'
b1=WIDGET_BUTTON(base2,value=cattxt,uvalue='C',frame=3)
b2=WIDGET_BUTTON(base2,value=poptext,uvalue='P',frame=3)
b3=WIDGET_BUTTON(base2,value=logtext,uvalue='idlcr8ascii.log',sensitive=optsens,frame=3)

base3=WIDGET_LABEL(base)
WIDGET_CONTROL,base3,set_value=fctext1
base4=WIDGET_LABEL(base)
WIDGET_CONTROL,base4,set_value=fctext2

base5=WIDGET_BASE(base,/Nonexclusive,ROW=1)
b4=WIDGET_BUTTON(base5,value=h4text,uvalue='H4',frame=3)
b5=WIDGET_BUTTON(base5,value=h5text,uvalue='H5',frame=3)
b6=WIDGET_BUTTON(base5,value=nctext,uvalue='NC',sensitive=optsens,frame=3)

base6=WIDGET_BASE(base,/Row)
b7=WIDGET_BUTTON(base6,value='ASCII Formatted',uvalue='F',sensitive=optsens,frame=3) ;,Tooltip=Tip)
b8=WIDGET_BUTTON(base6,value='ASCII Dump',uvalue='D',sensitive=optsens,frame=3) ;,Tooltip=Tip)
b9=WIDGET_BUTTON(base6,value='Continue',uvalue='Cont',frame=3,Sensitive=0) ;,Tooltip=Tip)
b10=WIDGET_BUTTON(base6,value='Stop',uvalue='S',frame=3) ;,ToolTip=Tip)
WIDGET_CONTROL,base,/Realize
WIDGET_CONTROL,b1,/Input_Focus
FOR i=0,N_ELEMENTS(errtxt)-1 do $
  WIDGET_CONTROL,wtxt,set_value=errtxt[i],/Append
XMANAGER,'intro_a',base

END ;Intro_A



PRO idlcr8ascii_event, ev
;Procedure to close the pop-up logging window after user selects 'Finish'.
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;
;  History:
;    20061004: Introduced to IDLCR8ASCII - Version 2.0
;
;  Input: Selected widget event structure
;
;  Output: Nil
;
;  Called by: XMANAGER in IDLCR8ASCII and STOP_WITH_ERROR_A
;
;  Subroutines Called: None

WIDGET_CONTROL,ev.top,/DESTROY
RETALL & HEAP_GC

END ;Proc IDLcr8ASCII_Event



PRO stop_with_error_a, txt1, txt2, lu
;Procedure called when an error in the program is detected. An error message is displayed
;and the program stopped and reset. If necessary, open files are closed. The error message
;is displayed in one or more of the following: a Pop-up dialog window; the Pop-up logging
;window; the Output Logging window in the IDLDE.
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;
;  History:
;    20050729: Original IDLCR8ASCII Routine - Version 1.0
;    20061004: Set-up so that the error output is displayed in the output window dependent on the
;              method that IDLCR8ASCII is called. If txt1 is preceeded by 'D_' or is null, the
;              error output is to a Pop-up Dialog window. Otherwise output will be to a logging
;              window. Renamed from STOP_WITH_ERROR to STOP_WITH_ERROR_A to avoid a name conflict
;              with the equivalent procedure in IDLCR8HDF. Common variable definition WIDGET_WIN_A
;              added - Version 2.0
;    20080302: Added code which sends the error message to the IDLDE output window and/or an external
;              file (as determined by the dux array values) - Version 3.0
;    20101122: Allow routine to return to the calling program, rather than stop the application, if
;              a 'reterr' argument is included in the call to idlcr8ascii - Version 4.0b1

;
;  Inputs: txt1 - the initial text line of the error message. Generally contains the name of the routine
;                 where the error was detected.
;          txt2 - the second text line which generally describes the error state.
;          lu - Where applicable, the file unit that needs to be closed at the termination of the program,
;               otherwise set to -1.
;
;  Output: Nil
;
;  Called by: The routine in which the error was detected. The following routines call STOP_WITH_ERROR:
;             READ_HDF_SDS; IDLCR8ASCII
;
;  Subroutines Called: IDLCR8ASCII_EVENT (via XMANAGER)

COMMON WIDGET_WIN_A

IF lu NE -1L THEN FREE_LUN,lu

IF txt1 EQ '' THEN BEGIN ;<cancel> chosen on Intro box
  res=DIALOG_MESSAGE('IDLcr8ASCII Stopped!',/Information,Title='AVDC IDLcr8ASCII')
ENDIF ELSE BEGIN
  IF STRMID(txt1,0,2) EQ 'D_' THEN txtx=STRMID(txt1,2) ELSE txtx=txt1
  FOR i=dux[0],dux[1],dux[2] DO BEGIN
    IF i EQ -1 THEN BEGIN
      PRINT,'  ERROR in '+txtx & PRINT,'  '+txt2
      PRINT,'' & PRINT,'IDLcr8ASCII stopped - Program Ended on '+SYSTIME(0)
    ENDIF ELSE BEGIN
      PRINTF,i,'  ERROR in '+txtx & PRINTF,i,'  '+txt2
      PRINTF,i,'' & PRINTF,i,'IDLcr8ASCII stopped - Program Ended on '+SYSTIME(0)
    ENDELSE
  ENDFOR
  IF dux[1] GT -1 THEN FREE_LUN,dux[1]
  IF (STRMID(txt1,0,2) EQ 'D_') AND (rerr EQ 'NA') THEN BEGIN
    ;write error to DIALOG Box instead of Popup window
    errtxt2=STRARR(4)
    errtxt2[0]=STRMID(txt1,2) & errtxt2[1]=txt2 & errtxt2[3]='IDLcr8ASCII Stopped!'
    res=DIALOG_MESSAGE(errtxt2,/Error,Title='AVDC IDLcr8ASCII Error')
  ENDIF ELSE IF rerr EQ 'NA' THEN BEGIN ;write error to Popup window
    lineno=lineno+4L
    WIDGET_CONTROL,wtxt,set_value='    ERROR in '+txt1,/Append
    WIDGET_CONTROL,wtxt,set_value='    '+txt2,/Append
    WIDGET_CONTROL,wtxt,set_value='',/Append,Set_text_top_line=lineno
    WIDGET_CONTROL,wtxt,set_value='HDF file read stopped - hit <Finish> to close program',/Append
    WIDGET_CONTROL,b3,Sensitive=1,/Input_Focus
    XMANAGER,'stop_with_error_a',base,Event_Handler='idlcr8ascii_event'
  ENDIF
ENDELSE
HEAP_GC
IF rerr EQ 'NA' THEN RETALL ELSE rerr='Unable to read HDF or NC file - '+txtx+txt2

END ;Procedure Stop_With_Error_A



PRO infotxt_output_a, in0, in1
;Procedure called to report information relevant to the reading of the HDF file.
;This information can be reported to a Pop-up logging window, IDLDE output log window, and/or
;the log file. Code for this output was originally written directly in the affected procedures
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;
;  History:
;    20090311: Introduced to IDLCR8ASCII - Version 3.02
;    20091208: Added all the INFORMATION text messages to the routine, to avoid duplication
;              in the HDF4 and HDF5 read sections of READ_HDF_SDS - Version 3.03
;    20160213: Update information message when in0[0] EQ 13 - Version 4.0b13
;    20161130: Added in0[0] messages 18 and 19 to cover non-GEOMS metadata attributes issues
;              - Version 4.0b16
;
;  Inputs: in0 - Integer array containing input used to make correct text output
;          in1 - String array containing input used to make correct text output
;
;  Output: Nil
;
;  Called by: The routine in which the request for INFORMATION text was made.
;             The following routines call INFOTXT_OUTPUT_A: READ_HDF_SDS; OUTPUT_HDF_DATA;
;             IDLCR8ASCII
;
;  Subroutines Called: None

COMMON WIDGET_WIN_A

CASE 1 OF
  in0[0] EQ 0: BEGIN
      infotxt=STRARR(3)
      IF LONG(in1[0]) EQ 0L THEN BEGIN
        infotxt[0]='  INFORMATION: Global Attribute DATA_VARIABLES not found or has no values.'
        infotxt[1]='    Dataset listing order matches the order that the dataset is read in the HDF file.'
      ENDIF ELSE BEGIN
        infotxt[0]='  INFORMATION: Number of Global Attribute DATA_VARIABLES values ('+in1[0]+')'
        infotxt[1]='    does not match the number of datasets saved to the HDF file ('+in1[1]+')'
      ENDELSE
      infotxt[2]='    Number of datasets determined from '+in1[2]+' call'
    END
  in0[0] EQ 1: infotxt='  INFORMATION: '+in1[0]+' has '+in1[1]+' data dimensions'
  in0[0] EQ 2: infotxt='  INFORMATION: '+in1[0]+' dataset is not one of the allowable Data Types - '+in1[1]
  in0[0] EQ 3: BEGIN
      infotxt=STRARR(4)
      IF (STRPOS(STRUPCASE(in1[1]),STRUPCASE(in1[0])) NE -1) AND (in0[1] EQ 4) AND (in0[2] EQ 0) THEN $
        infotxt[0]='  INFORMATION: Dataset Name is truncated (File created with HDF4.2r1 library or earlier):' $
      ELSE infotxt[0]='  INFORMATION: Dataset Name does not match VAR_NAME value:'
      infotxt[1]='    SDS_NAME: '+in1[0]
      infotxt[2]='    VAR_NAME: '+in1[1]
      IF in0[2] EQ 0 THEN infotxt[3]='    Output uses VAR_NAME from Metadata' $
      ELSE infotxt[3]='    Output uses Dataset Name from the HDF file'
    END
  in0[0] EQ 4: BEGIN
      infotxt=STRARR(2)
      IF in0[1] EQ 0 THEN BEGIN
        infotxt[0]='  INFORMATION: Metadata label VAR_NAME not found for dataset:'
        infotxt[1]='    SDS_NAME: '+in1[0]
      ENDIF ELSE IF (in0[2] EQ 0) AND (in0[3] NE 0L) THEN BEGIN
        infotxt[0]='  INFORMATION: VAR_NAME value does not match any DATA_VARIABLES values:'
        infotxt[1]='    VAR_NAME: '+in1[1]
      ENDIF
    END
  in0[0] EQ 5: BEGIN
      infotxt=STRARR(2)
      infotxt[0]='  INFORMATION: No Variable Attributes returned after call to '+in1[1]
      infotxt[1]='    SDS_NAME: '+in1[0]
    END
  in0[0] EQ 6: BEGIN
      infotxt='  INFORMATION: Metadata Dataset listing order may not match the order of'
      infotxt=infotxt+' the Global Attribute DATA_VARIABLES Values'
    END
  in0[0] EQ 7: BEGIN
      infotxt='  INFORMATION: Can read one HDF or netCDF file into session memory.'
      infotxt=infotxt+' Only first file in the array will be read'
    END
  in0[0] EQ 8: BEGIN
      infotxt='  INFORMATION: '+in1[0]+' is a coordinate variable'
    END
  in0[0] EQ 9: BEGIN
      infotxt=STRARR(3)
      infotxt[0]='  INFORMATION: Calibrated data in dataset '+in1[0]
      infotxt[1]='    has been converted back to its original form using the formula:'
      infotxt[2]='    Orig_Data = Scale_Factor * (Cal_Data - Offset)'
    END
  in0[0] EQ 10: BEGIN
      infotxt=STRARR(2)
      infotxt[0]='  INFORMATION: VAR_DATA_TYPE='+in1[1]+' for Calibrated data in dataset '+in1[0]
      infotxt[1]='    has been changed to the original datatype of '+in1[2]
    END
  in0[0] EQ 11: BEGIN
      infotxt=STRARR(2)
      CASE 1 OF
        in0[1] EQ 20: BEGIN
            infotxt[0]='  INFORMATION: Order of multi-dimensional datasets changed to match the programming'
            infotxt[0]=infotxt[0]+' language convention:
            infotxt[1]='    IDL/Fortran - fastest changing dimension first (datasets transposed when'
            infotxt[1]=infotxt[1]+' reading/writing to the HDF file)'
          END
        in0[1] MOD 10 EQ 0: BEGIN
            infotxt[0]='  INFORMATION: Listing order of multi-dimensional VAR_DEPEND and VAR_SIZE values'
            infotxt[1]='    in the HDF file changed to match corresponding dataset dimension ordering'
          END
        ELSE: BEGIN
            infotxt[0]='  INFORMATION: Unable to determine the correct listing order of multi-dimensional'
            infotxt[1]='    VAR_SIZE and VAR_DEPEND values in the HDF file'
          END
      ENDCASE
    END
  in0[0] EQ 12: BEGIN
      infotxt=STRARR(2)
      infotxt[0]='  INFORMATION: Table Attribute Values file must be in the same directory as the input file'
      infotxt[1]='    if wanting to convert from one Data Format to another'
    END
  in0[0] EQ 13: BEGIN
      infotxt=STRARR(2)
      infotxt[0]='  INFORMATION: IDLcr8HDF called but it is not in the IDL Search Path or it encountered an'
      infotxt[1]='    unexpected error. If the latter please contact Ian Boyd at iboyd@astro.umass.edu'
    END
  in0[0] EQ 14: BEGIN
      infotxt=STRARR(2)
      infotxt[0]='  INFORMATION: /POPUP keyword cannot be used together with the ''reterr'' argument.'
      infotxt[1]='    The request for a POPUP window has been ignored'
    END
  in0[0] EQ 15: BEGIN
      infotxt=STRARR(2)
      infotxt[0]='  INFORMATION: Argument ''reterr'' must be of type string.' ;a returnable variable of type string.'
      infotxt[1]='    idlcr8ascii will stop normally if an error is encountered'
    END
  in0[0] EQ 16: BEGIN
      infotxt=STRARR(2)
      infotxt[0]='  INFORMATION: /LOG, /FORMAT, and /DUMP keywords cannot be used in IDL DEMO mode and will be ignored.'
      infotxt[1]='    To generate ASCII files please use the free IDL Virtual Machine or a licenced version of IDL'
    END
  in0[0] EQ 17: BEGIN
      infotxt=STRARR(2)
      infotxt[0]='  INFORMATION: NetCDF file create feature is disabled in IDL DEMO mode.'
      infotxt[1]='    To generate files please use the free IDL Virtual Machine or a licenced version of IDL'
    END
  in0[0] EQ 18: BEGIN
      infotxt='  INFORMATION: Value for Attribute Label '+in1[0]+' is an invalid Data Type'
    END
  in0[0] EQ 19: BEGIN
      infotxt='  INFORMATION: '+in1[0]+' is not a valid GEOMS Metadata Attribute'
    END
ENDCASE

dm=SIZE(infotxt,/N_Elements)
IF o3[3] EQ '' THEN lineno=lineno+dm

FOR n=0,dm-1 DO BEGIN
  IF o3[3] EQ '' THEN BEGIN
    IF n EQ dm THEN WIDGET_CONTROL,wtxt,set_value='',/Append $
    ELSE WIDGET_CONTROL,wtxt,set_value=infotxt[n],/Append,Set_text_top_line=lineno
  ENDIF
  FOR m=dux[0],dux[1],dux[2] DO $
    IF m EQ -1 THEN PRINT,infotxt[n] ELSE PRINTF,m,infotxt[n] ;write out to log(s)
ENDFOR

END ;Procedure InfoTxt_Output_A



FUNCTION jdf_2_datetime, jdf, MJD2000=mjd2000, SHORTISO8601=iso, LONGISO8601=isoms
;Computes the UT date/time from JDF/MJD2000.
; ----------
;   Bojan R. Bojkov
;   bojan.r.bojkov@nasa.gov
;   03/04/2004
;   03/11/2004 bug fix
;   05/27/2005 add ShortISO8601 and LongISO8601 switches,
;              and change seconds value to decimal seconds (Ian Boyd)
;
; References ----------
;   Explan. Supp. Astron. Almanac, p.604 (1992); through E. Celarier
;
; Caveats ----------
;   Valid for all YYYY >= -4712 (i.e. for all JD .ge. 0)
;   The true Gregorian calendar was only adopted on 15 October 1582,
;   so any dates before this are "virtual" Gregorian dates.
;
; Input ----------
;   jdf     : double precision value
;   mjd2000 : flag if input is MJD2000
;   iso     : flag if output is in ISO8601 (YYYYMMDDThhmmssZ)
;   isoms   : flag if output is in ISO8601ms (YYYYMMDDThhmmss.sssZ)
;
; Output ---------
;   floating point array [YYYY, MM, DD, hh, mn, ss.sss],
;   or ISO8601 string format
; External subroutines ---------
;   NONE

jdf=DOUBLE(jdf)

j0=2451544.5D
IF KEYWORD_SET(mjd2000) THEN jdhold=jdf+j0 ELSE jdhold=jdf
jdi=LONG(jdhold)
df=jdhold-jdi

;Determine hh, mm, ss, ms
hh=(df+0.5)*24.D
q=WHERE(hh GE 24.D)
IF q[0] NE -1 THEN BEGIN
  hh[q]=hh[q]-24.D
  jdi[q]=jdi[q]+1
ENDIF
t1=hh
hh=LONG(t1)
t2=(t1-hh)*60.D
mn=LONG(t2)
t3=(t2-mn)*60.D
ss=t3
;ss=LONG(t3)
;ms=LONG((t3-ss)*1.d3)

;Determine YYYY, MM, DD
t1=jdi+68569L
t2=(4*t1)/146097L
t1=t1-(146097L*t2+3L)/4L
t3=(4000L*(t1+1L))/1461001L
t1=t1-(1461L*t3)/4L + 31L
t4=(80L*t1)/2447L

dd=t1-(2447L*t4)/80L
t1=t4/11L
mm=t4+2L-12L*t1
yyyy=100L*(t2-49L)+t3+t1

dt=TRANSPOSE([[yyyy],[mm],[dd],[hh],[mn],[ss]])
IF (KEYWORD_SET(iso)) OR (KEYWORD_SET(isoms)) THEN BEGIN
  dts=STRARR(6)
  IF (KEYWORD_SET(iso)) AND (ss-FIX(ss) GE 0.5) THEN BEGIN
    ;recalculate to get correct seconds value
    jdhold=jdf+0.000008D ;add ~0.7 secs
    IF KEYWORD_SET(mjd2000) THEN jdhold=jdhold+j0
    CALDAT,jdhold,mm,dd,yyyy,hh,mn,ss
    dt=TRANSPOSE([[yyyy],[mm],[dd],[hh],[mn],[ss]])
  ENDIF
  dt=LONG(dt)
  FOR i=1,5 DO $
    IF dt[i] LT 10L THEN dts[i]='0'+STRTRIM(dt[i],2) ELSE dts[i]=STRTRIM(dt[i],2)
  IF KEYWORD_SET(isoms) THEN BEGIN
    ssd=STRING(format='(f6.3)',ss)
    IF FLOAT(ssd) LT 10.0 THEN dts[5]='0'+STRTRIM(ssd,2) ELSE dts[5]=STRTRIM(ssd,2)
  ENDIF
  RETURN,STRTRIM(dt[0],2)+dts[1]+dts[2]+'T'+dts[3]+dts[4]+dts[5]+'Z'
ENDIF ELSE RETURN, dt

END ;Function jdf_2_datetime



FUNCTION julian_date, date_ut, ISO8601=iso, MJD2000=mjd2000
;Computes the Julian date (jd) or date in MJD2000 format in double precision.
; ----------
;   Bojan R. Bojkov
;   bojan.r.bojkov@nasa.gov
;   06/03/2001
;   09/24/2001: Code cleanup                        - Function Version 1.0
;   10/16/2001: Mode corrections                    - Function Version 1.01
;   03/07/2002: Comment cleanup and variable change - Function Version 2.0
;   10/08/2002: Added jdf option                    - Function Version 2.1
;   07/21/2003: Converted to a function             - Function Version 3.0
;   05/25/2005: Added ISO8601 and MJD2000 keywords.
;               Input can either be a string or a numeric array
;               (see below for input details). Returns -99999.d
;               if the Input is invalid - Ian Boyd
;
; References ----------
;   Hughes, D.W., Yallop, B.D. and Hohenkerk, C.Y., "The Equation of Time",
;     Mon. Not. R. astr. Soc., 238, pp 1529-1535. 1989.
;
;   Results verified using:
;     Meeus, J, "Astronomical Algorithms", William-Bell, Inc., Richmond VA,
;     p62, 1991.
;
; Caveats ----------
;   NONE
;
; Input ----------
;   date_ut: Numeric array (UT: YYYY, MM, DD, [hh, mm, ss]), or if ISO8601
;            keyword set, a string containing datetime as YYYYMMDDThhmmssZ
;
; Output ---------
;   jd:      julian day, or if MJD2000 keyword set,
;            a double precision day in MJD2000 format
;
; External subroutines ---------
;   NONE

ON_IOERROR,TypeConversionError
valid=0 ;Type conversion check
inputerror=0 ;Check that input is as expected
;Check input format
IF KEYWORD_SET(iso) THEN BEGIN ;input is YYYYMMDDThhmmssZ
  IF STRLEN(date_ut) EQ 16 THEN BEGIN
    ;test for numeric values (will return type conversion error value if not a number)
    FOR i=0,7 DO check=FIX(STRMID(date_ut,i,1))
    FOR i=9,14 DO check=FIX(STRMID(date_ut,i,1))
    dt=INTARR(6)
    dt[0]=FIX(STRMID(date_ut,0,4)) & dt[1]=FIX(STRMID(date_ut,4,2))
    dt[2]=FIX(STRMID(date_ut,6,2)) & dt[3]=FIX(STRMID(date_ut,9,2))
    dt[4]=FIX(STRMID(date_ut,11,2)) & dt[5]=FIX(STRMID(date_ut,13,2))
    date_ut=dt & achk=[0,6] ;to add hhmmss component
  ENDIF ELSE inputerror=1
ENDIF ELSE BEGIN
  ;check that input contains at least YMD (and at most YMDhms) information and is either
  ;an integer, long, float or double array
  achk=SIZE(date_ut)
  IF (achk[1] LT 3) OR (achk[1] GT 6) OR (achk[2] LT 2) OR (achk[2] GT 5) THEN inputerror=1
ENDELSE

IF inputerror EQ 0 THEN BEGIN ;can perform JD calculation
  IF date_ut[1] GT 2 THEN BEGIN
    y=DOUBLE(date_ut[0])
    m=DOUBLE(date_ut[1]-3)
    d=DOUBLE(date_ut[2])
  ENDIF ELSE BEGIN
    y=DOUBLE(date_ut[0]-1)
    m=DOUBLE(date_ut[1]+9)
    d=DOUBLE(date_ut[2])
  ENDELSE

  ;Compute Julian date:
  j=LONG(365.25D0*(y+4712.D0))+LONG(30.6D0*m+0.5D0)+59.D0+d-0.5D0

  ;Check for Julian or Gregorian calendar:
  IF j LT 2299159.5D0 THEN jd=j $ ;If Julian calendar.
  ELSE BEGIN ;If Gregorian calendar.
    gn=38.D0-LONG(3.D0*LONG(49.D0+y/100.D0)/4.D0)
    jd=j+gn
  ENDELSE

  ;add hhmmss values if present
  CASE 1 OF
    achk[1] EQ 4: jd=jd+DOUBLE(date_ut[3])/24.D ;hh only
    achk[1] EQ 5: jd=jd+(DOUBLE(date_ut[3])*3600.D + DOUBLE(date_ut[4])*60.D)/86400.D ;hhmm only
    achk[1] EQ 6: jd=jd+(DOUBLE(date_ut[3])*3600.D + DOUBLE(date_ut[4])*60.D + $
                         DOUBLE(date_ut[5]))/86400.D ;hhmmss
    ELSE:
  ENDCASE

  valid=1 ;Type conversion OK

  ;Set to MJD2000 if required
  IF KEYWORD_SET(mjd2000) THEN jd=jd-2451544.5D
ENDIF

TypeConversionError:
IF valid EQ 0 THEN RETURN,-99999.D ELSE RETURN, jd

END ;Function Julian_Date



PRO test_dim_order, va_name, va_value, va_type, sds_dim, rev_vd_vs
;Procedure to test the dimension ordering of any multi-dimensional datasets in the input DF file, and
;return correct ordering code (dimension ordering for idlcr8ascii uses IDL/Fortran convention)
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;
;  History:
;    20111208: Introduced - Version 4.0b4
;    20140325: Fix bug that meant VAR_SIZE and VAR_DEPEND values were not being automatically
;              reversed if the array sizes were the same and VAR_DEPEND did not include a DATETIME
;              value. Now defaults to the 'reverse' option if no other conditions are satisfied;
;              Fix bug that caused crash when VAR_SIZE values were not of type string - Version 4.0b8
;    20141110: Fix bug that caused multi-dimensional array ordering to not be correctly identified
;              if the first dataset to be checked in the file had the same number of elements in
;              the array (e.g. was a set of Averaging Kernels) - Version 4.0b9

;
;  Inputs: va_name - An abbreviated version of the Variable Name; either 'VD' (VAR_DEPEND) or 'VS'
;                    (VAR_SIZE)
;          va_value - The corresponding Variable Value
;          sds_dim - Dimension information for the DF Dataset being tested
;
;  Outputs: rev_vd_vs - Scalar to indicate whether VAR_DEPEND and VAR_SIZE values (and data, in the case
;                       of netCDF measurements) need to be reversed
;
;  Called by: Read_HDF_SDS
;

IF va_type NE 'STRING' THEN BEGIN ;VAR_SIZE contains numeric values instead of in the form of a string
  vavhold='' & n_vav=N_ELEMENTS(va_value)
  FOR k=0,n_vav-1 DO BEGIN
    IF k EQ n_vav-1 THEN vtxt='' ELSE vtxt=';'
    vavhold=vavhold+STRTRIM(va_value[k],2)+vtxt
  ENDFOR
ENDIF ELSE vavhold=va_value
vs_v=STRCOMPRESS(STRSPLIT(vavhold,';, ',/EXTRACT,COUNT=rcnt),/REMOVE_ALL)

IF va_name EQ 'VD' THEN BEGIN ;VAR_DEPEND tests
  dti=WHERE(STRUPCASE(vs_v) EQ 'DATETIME',dticnt)
  ;Note: if DATETIME has VAR_SIZE=1 then the ordering is dependent on the order of the VAR_SIZE values only
  IF (rcnt GT 1) AND (dticnt NE 0) THEN BEGIN ;multi-dimensions including DATETIME
    test1=(dti[0] EQ rcnt-1) AND (sds_dim[0] GT 1)
    ;DATETIME is the last value and present only once and has VAR_SIZE GT 1
    test2=(dti[0] EQ 0) AND (dticnt EQ 1) AND (sds_dim[0] GT 1)
    ;DATETIME is the first value and present only once and has VAR_SIZE GT 1
    IF (test1) AND (rev_vd_vs EQ 2) THEN rev_vd_vs=10 $
    ELSE IF (test2) AND (rev_vd_vs EQ 2) THEN rev_vd_vs=11 $
    ELSE IF ((test1) AND (rev_vd_vs MOD 10 EQ 1)) OR ((test2) AND (rev_vd_vs EQ 10)) THEN $
      rev_vd_vs=3 $ ;inconsistent rules regarding dimension ordering in file
    ELSE IF (test2) AND (rev_vd_vs EQ 0) THEN rev_vd_vs=20
    ;Dimension ordering is slowest changing first so change dataset ordering as well as
    ;VAR_DEPEND and VAR_SIZE ordering
  ENDIF
ENDIF ELSE BEGIN ;VAR_SIZE tests
  IF rcnt GT 1 THEN BEGIN ;multi-dimensions
    ;check all values are numeric
    numchk=STRJOIN(vs_v,/SINGLE) & valok=1
    FOR k=0,STRLEN(numchk)-1 DO BEGIN
      bchar=BYTE(STRMID(numchk,k,1))
      IF (bchar LT 48) OR (bchar GT 57) THEN valok=0 ;i.e. non-numeric character
    ENDFOR
    IF valok EQ 1 THEN BEGIN ;all values are numeric so do checks
      vs_vl=LONG(vs_v)
      test1=ARRAY_EQUAL(vs_vl,sds_dim)
      test2=ARRAY_EQUAL(REVERSE(vs_vl),sds_dim) ;test in case values are the same e.g. 40;40
      ;If arrays are the same size then the default will be to reverse the VS and VD values
      ;unless a different dimension ordering has already been identified
      IF (test1 EQ 1) AND (test2 EQ 0) AND (rev_vd_vs EQ 2) THEN rev_vd_vs=0 $
      ELSE IF (test1 EQ 0) AND (test2 EQ 1) AND (rev_vd_vs EQ 2) THEN rev_vd_vs=1 $
      ELSE IF ((test1 EQ 1) AND (test2 EQ 0) AND (rev_vd_vs MOD 10 EQ 1)) OR $
              ((test1 EQ 0) AND (test2 EQ 1) AND (rev_vd_vs MOD 10 EQ 0)) THEN $
        rev_vd_vs=3 $ ;inconsistent VAR_SIZE and sds_dim agreement from dataset to dataset
      ELSE IF (test1 EQ 1) AND (test2 EQ 1) AND (rev_vd_vs EQ 2) THEN $
        rev_vd_vs=2 $ ;first set of measurements tested have same dimensions so do not do anything
      ELSE IF rev_vd_vs EQ 2 THEN rev_vd_vs=1 ;default if no other criteria are found as this
      ;will then automatically reverse the VD and VS values
    ENDIF
  ENDIF
ENDELSE

END ;Procedure Test_Dim_Order



PRO read_hdf_sds, ifile, ga, sds, catinfo
;Procedure to read the contents of a GEOMS standard HDF4 or HDF5 compatible file
;into session memory
; ----------
;Written by Ian Boyd for the AVDC - iboyd@astro.umass.edu
;
;  History:
;    20050729: Original IDLCR8ASCII Routine - Version 1.0
;    20050912: Removed Common variable definition CATALOGINFO and made the variable a parameter passed
;              to the procedure; Checks that the attribute variable names read by HDF_SD_GETINFO, match
;              those recorded in DATA_VARIABLES in the Global Attributes. If not use the variable name
;              listed in DATA_VARIABLES - Version 1.1
;    20061004: Common variable definition WIDGET_WIN_A added for Error calls; Add option to read HDF5
;              files (needs IDL5.6 or newer); Add check that the HDF4 file has valid information on the
;              number of global and variable attributes; Checks added in the event that the HDF4 or
;              HDF5 file has been created by the NCSA utility programs H5toH4 or H4toH5 from an original
;              AVDC/Envisat/NDACC HDF4 or HDF5 file; Ensures variable attributes and data are listed in
;              the same order as that given under DATA_VARIABLES - Version 2.0
;    20081020 - If the HDF4 file is created using the HDF4.2r3 library, then extra information
;               regarding the dimensions of the VAR_DEPEND values may be included in the file (which
;               show up as extra datasets in the HDF_SD_FILEINFO call). A check for this is carried out
;               and, if found, the information is excluded from the output - Version 3.01
;    20090311: Improve HDF4.2r3 library checks by comparing the number of datasets recorded under
;              DATA_VARIABLES, with the number returned by the HDF_SD_FILEINFO call; Change 'WARNING' to
;              'INFORMATION and make calls to procedure INFOTXT_OUTPUT_A to avoid duplication. Add
;              'INFORMATION' text if anything irregular found with the datasets; Change 'Data dimensions
;               exceed 8' and 'Data type not allowable' ERROR messages to 'INFORMATION' messages
;               - Version 3.02
;    20091208: Incorporate HDF_SD_ISCOORDVAR to differentiate between datasets and dimension variable
;              names. Add check for allowable VAR_DATA_TYPE=STRING. Add all INFORMATION text messages to
;              INFOTXT_OUTPUT_A procedure to avoid duplication of code that creates the messages;
;              Add INFORMATION messages when reading HDF5 files; Improve checks for non-standard (i.e.
;              non-groundbased) HDF input file - Version 3.03
;    20101122: Numeric metadata values remain in their original data type when transferring to session
;              memory, meaning: variable attribute labels and values now written to separate structure
;              variables sds.va_l and sds.va_v (previously sds.va), and structure has 2 dimensions
;              (n_sds, n_atts) instead of 1 (previously n_sds); The dataset is written to the initial
;              n_atts index (i.e. sds[n_sds,0].data); Rename saved SDS datatype according to GEOMS rules
;              (affects INT, LONG, and FLOAT); Compare HDF dimensions with VAR_SIZE values and transpose
;              VAR_SIZE and VAR_DEPEND values if required - Version 4.0b1
;    20111014: Add netCDF read capability - writes variable attributes to standard GEOMS variable
;              attribute labels where possible, otherwise appends information to VAR_NOTES - Version 4.0b3
;    20120426: Bug fix when a netCDF file only contains datasets with single dimensions, then VAR_SIZE
;              information was not being extracted during the read process - Version 4.0b6
;    20140325: Fix bug that caused crash when VAR_SIZE values were not of type string - Version 4.0b8
;    20150127: Allow for VAR_FILL_VALUE values for string datasets to be written and saved as an
;              (empty) string of the correct length. Ensure string datasets are all written and
;              saved to the correct length (that of the longest string in the dataset) - Version 4.0b10
;    20150217: Do not remove whitespace of any variable attribute values that are written as strings
;              - Version 4.0b11
;    20160614: Some HDF4 string datasets may not have 2 dimensions, so do not need to remove the first
;              dimension (= maximum number of characters) - Version 4.0b14
;    20160725: Fix bug causing program to crash when the number of DATA_VARIABLES values is greater
;              than the number of datasets in the file - Version 4.0b15
;    20161130: Assign SDS_name to catinfo for attributes when the VAR_NAME is not present; rewrite
;              netCDF section to align it with HDF5 section and to account for dataset ordering that
;              does not match that in DATA_VARIABLES - Version 4.0b16
;    20161213: Fix bug when reading in netCDF files to stop all dataset variable attribute values to
;              be converted to string format (now matches HDF4 and HDF5 checks). Fix bug that caused
;              the program to crash when VAR_DEPEND=CONSTANT in netCDF files (sds_ndim=0) - Version 4.0b17
;    20161218: Fix bug when reading netCDF attribute values that are of type byte rather than string -
;              Version 4.0b18
;    20180218  Fix bug that caused multi-dimensional array ordering to not be correctly identified
;              if the datasets have the same VAR_SIZE. Now rev_vd_vs changes to 1 (VAR_SIZE and
;              VAR_DEPEND ordering is reversed) if rev_vd_vs=2 after all the checks - Version 4.0b19
;    20181116  Fix bug that caused multi-dimensional array ordering to not be correctly written if
;              not consistent through the file i.e. rev_vd_vs eq 3. Now rev_vd_vs changes to 1
;              (VAR_SIZE, VAR_DEPEND and dataset ordering is reversed), but information message
;              advising of the issue is still reported - Version 4.0b20
Ian Boyd's avatar
Ian Boyd committed
893
894
895
;    20190514  Fix bug that occurred when GEOMS variable attribute information is missing from an
;              HDF4 file but written to the heap structure based on the contents of the file. Previously
;              the variable attribute label was not written correctly - Version 4.0b21
Ian Boyd's avatar
Ian Boyd committed
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
;
;  Inputs: ifile - a string containing the name of the input file to be read in.
;          catinfo - a string array identifying the type of input file ('H4','H5','NC')
;
;  Outputs: ga - a string array containing the global attribute labels and values extracted from the HDF
;                file.
;           sds - a structure using pointers, of size [n_sds,n_atts], containing the variable attribute
;                 labels and values (sds[n,m].va_l and sds[n,m].va_v) and the data (sds[n,0].data)
;                 extracted from the HDF file.
;           catinfo - a string array of size [n_sds,4] (where n_sds is the number of datasets in the HDF
;                     file), containing information on the variable name, data type, data dimension, and
;                     number of attributes.
;
;  Called by: IDLCR8ASCII
;
;  Subroutines Called: STOP_WITH_ERROR_A (if error state detected)
;                      INFOTXT_OUTPUT_A (if program can make a change)
;    Possible Conditions for STOP_WITH_ERROR call (plus [line number] where called):
;      1. No global attributes or datasets present in the root group of an HDF5 file.
;      2. The number of global attributes or datasets is zero or not able to be determined
;         in an HDF4 file.
;
;    Information Conditions (when the program is able to make changes):
;      1. DATA_VARIABLES global attribute not found or has no values.
;      2. Number of datasets given under DATA_VARIABLES is not equal to the number saved to the
;         file.
;      3. The number of dimensions in a dataset exceeds 8.
;      4. The data type of a dataset is not BYTE, SHORT, INTEGER, LONG, REAL, DOUBLE, or STRING.
;      5. Dataset name is truncated or does not match VAR_NAME value.
;      6. Metadata label VAR_NAME not found for the dataset.
;      7. VAR_NAME does not match any DATA_VARIABLES values.
;      8. No Variable Attributes returned after HDF call.
;      9. Metadata Dataset listing order may not match the order of the Global Attribute
;         DATA_VARIABLES Values.
;     10. Variable is written to the HDF file as a coordinate variable.

COMMON WIDGET_WIN_A

;Note any calibrated (scaled) HDF4 dataset is corrected using the formula specified in
;UG_print42r3.pdf pg. 3-107: orig = cal * (cal_val - offset)
ncsa_cal=['scale_factor','scale_factor_err','add_offset','add_offset_err','calibrated_nt']

;Initialize scalar to indicate whether VAR_DEPEND and VAR_SIZE values need to be reversed
rev_vd_vs=2 ;1 = reverse attribute values; 0 = no reverse (and will generate message);
;2 = no action required; 3 = rev_vd_vs changed between 0 and 1 during checking, therefore error
;Note if rev_vd_vs is 2 or 3 after all the checks then it will be changed to 1 so that dimensions
;are swapped by default

;List of Standard GEOMS Variable Attributes
attr_arr_data=['VAR_NAME','VAR_DESCRIPTION','VAR_NOTES','VAR_SIZE','VAR_DEPEND',$
               'VAR_DATA_TYPE','VAR_UNITS','VAR_SI_CONVERSION','VAR_VALID_MIN',$
               'VAR_VALID_MAX','VAR_FILL_VALUE']
n_aad=N_ELEMENTS(attr_arr_data)

;Possible error messages for this procedure
proname='Read_HDF_SDS procedure: '
errtxt=STRARR(2)
errtxt[0]=' found in the Root Group of the HDF5 File.'
errtxt[1]=' is zero or not able to be determined (check for corrupted file).'
lu=-1

;Allowable VAR_DATA_TYPEs
avdt=['BYTE','SHORT','INTEGER','LONG','REAL','DOUBLE','STRING']
slabel=['VAR_UNITS','VAR_SI_CONVERSION','VAR_VALID_MIN','VAR_VALID_MAX','VAR_FILL_VALUE'] ;save values as is if dataset is of type STRING

;If necessary, free up memory by destroying the heap variables pointed at by its pointer arguments
;from previous calls to this program
IF N_ELEMENTS(sds) NE 0 THEN PTR_FREE,sds.va_l,sds.va_v,sds.data

;Define the HDF SDS storage structure
sds_set={va_l: PTR_NEW(), $ ;Variable Attribute Labels
         va_v: PTR_NEW(), $ ;Variable Attribute Values
         data: PTR_NEW()}   ;SD data array

IF catinfo[0,0] EQ 'H4' THEN BEGIN

  ;The HDF_SD_START function opens an HDF file and initializes the SD interface.
  sd_id=HDF_SD_START(ifile,/READ)

  ;Determine the number of SDS (n_sds) and global attributes (n_ga) found
  ;in the current file.
  HDF_SD_FILEINFO,sd_id,n_sds,n_ga
  IF n_sds EQ 0L THEN ntxt='Number of SDS datasets' $
  ELSE IF n_ga EQ 0L THEN ntxt='Number of Global Attributes' $
  ELSE ntxt=''
  IF ntxt NE '' THEN BEGIN
    STOP_WITH_ERROR_A,o3[3]+proname,ntxt+errtxt[1],lu & RETURN
  ENDIF
  ga=STRARR(n_ga) ;set the Global Attribute dimensions
  vn=[''] ;initialize vn array (to hold Dataset names)
  n_sds_hold=n_sds & n_vn=0L & do_coordvar=0

  ;Read the file's Global Attributes and determine number of DATA_VARIABLES
  FOR i=0L,n_ga-1L DO BEGIN
    ;The HDF_SD_ATTRINFO procedure reads or retrieves info about an SD attribute.
    HDF_SD_ATTRINFO,sd_id,i,NAME=ga_name,DATA=ga_data
    ga_name=STRTRIM(ga_name,2) & ga_data=STRTRIM(ga_data,2)
    ;Assign global attributes to ga
    ga[i]=ga_name+'='+ga_data
    ;read list of DATA_VARIABLES into array
    IF STRUPCASE(ga_name) EQ 'DATA_VARIABLES' THEN vn=STRSPLIT(ga_data,' ;',/Extract,COUNT=n_vn)
  ENDFOR

  IF n_sds_hold NE n_vn THEN BEGIN
    ;Do check for sds being a Dimension attribute
For faster browsing, not all history is shown. View entire blame