Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1856959

Summary: NCL error while trying to read GRIB2 files
Product: [Fedora] Fedora EPEL Reporter: Jim E <edgar1196>
Component: gdalAssignee: markusN <neteler>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: epel8CC: alex, devrim, jmlich83, mmahut, oliver, orion, pavel.lisy, pertusus, praiskup, scitech-bugs, volker27
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-08-04 03:37:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jim E 2020-07-14 19:08:23 UTC
Description of problem:

An error occurs while trying to read the contents of a GRIB2-formatted file, either with ncl (via script) or ncl_filedump commands. This behavior doesn't happen with GRIB, netcdf or HDF5 formatted files.

Version-Release number of selected component (if applicable):

ncl-6.6.2-10.el8

How reproducible:


Steps to Reproduce:
 
ncl_filedump -c rap.t00z.awp200f00.grib2

Actual results:

Copyright (C) 1995-2019 - All Rights Reserved
 University Corporation for Atmospheric Research
 NCAR Command Language Version 6.6.2
 The use of this software is governed by a License Agreement.
 See http://www.ncl.ucar.edu/ for more details.
g2_getfld: Request for field number must be positive.
warning:Invalid data field request.

Variable: f
(0) File Missing Value : -1
warning:getfiledimsizes: undefined file variable
Segmentation fault (core dumped)

Expected results:

 Copyright (C) 1995-2019 - All Rights Reserved
  University Corporation for Atmospheric Research
  NCAR Command Language Version 6.6.2
  The use of this software is governed by a License Agreement.
  See http://www.ncl.ucar.edu/ for more details.
   
 Variable: f
 Type: file
 filename: rap.t00z.awp200f00
 path:     rap.t00z.awp200f00.grib2
 file global attributes:
  dimensions:
        ygrid_0 = 94
        xgrid_0 = 108
         lv_ISBL0 = 39
         lv_HTGL1 = 2
         lv_SPDL2 = 6
         lv_HTGL3 = 2
         lv_ISBL4 = 37
         lv_SPDL5 = 3
         lv_HTGL6 = 2
      variables:
 ....


Additional info:

1. Data source:   ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/rap/prod/

2. Looking at the core dump file:
]$ coredumpctl info

           PID: 10046 (ncl)
           UID: 1000 ()
           GID: 1000 ()
          Signal: 11 (SEGV)
          Timestamp: Tue 2020-07-14 19:46:45 BST (14min ago)
          Command Line: ncl -n filename="rap.t00z.awp200f00.grib2" nvars=0 coords=1 ftype="GRIB" singleElemDims=(/"none"/) timePeriodSuffix=True /tmp/tmp10036.ncl
           Executable: /usr/bin/ncl
...
          Message: Process 10046 (ncl) of user 1000 dumped core.
                
                Stack trace of thread 10046:
                #0  0x00000000005ce174 _NclIGetVarDims (ncl)
                #1  0x00000000005878c9 CallINTRINSIC_FUNC_CALL (ncl)
                #2  0x000000000059261d _NclExecute (ncl)
                #3  0x000000000046f4da yyparse (ncl)
                #4  0x00000000004657f0 NclDriver (ncl)
                #5  0x00007f262c3386a3 __libc_start_main (libc.so.6)
                #6  0x0000000000461bce _start (ncl)

Comment 1 Orion Poplawski 2020-07-18 03:48:12 UTC
valgrind points to the issue:

==15801== Command: ncl -n filename="rap.t00z.awp200f00.grib2" nvars=0 coords=0 ftype="GRIB" singleElemDims=(/"none"/) timePeriodSuffix=True /tmp/tmp15791.ncl
==15801== 
 Copyright (C) 1995-2019 - All Rights Reserved
 University Corporation for Atmospheric Research
 NCAR Command Language Version 6.6.2
 The use of this software is governed by a License Agreement.
 See http://www.ncl.ucar.edu/ for more details.
==15801== Use of uninitialised value of size 8
==15801==    at 0x6D4708B: g2_getfld (in /usr/lib64/libgdal.so.26.0.4)
==15801==    by 0x56E4C5: ??? (in /usr/bin/ncl)
==15801==    by 0x4F9582: _NclFileCreate (in /usr/bin/ncl)
==15801==    by 0x4C25B9: _NclOpenFile (in /usr/bin/ncl)
==15801==    by 0x5B092C: _NclIAddFile (in /usr/bin/ncl)
==15801==    by 0x5878C8: CallINTRINSIC_FUNC_CALL (in /usr/bin/ncl)
==15801==    by 0x59261C: _NclExecute (in /usr/bin/ncl)
==15801==    by 0x46F4D9: yyparse (in /usr/bin/ncl)
==15801==    by 0x4657EF: NclDriver (in /usr/bin/ncl)
==15801==    by 0xC9796A2: (below main) (in /usr/lib64/libc-2.28.so)
==15801==
==15801==
==15801== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==15801==  Bad permissions for mapped region at address 0x56E0AE
==15801==    at 0x6D4708B: g2_getfld (in /usr/lib64/libgdal.so.26.0.4)
==15801==    by 0x56E4C5: ??? (in /usr/bin/ncl)
==15801==    by 0x4F9582: _NclFileCreate (in /usr/bin/ncl)
==15801==    by 0x4C25B9: _NclOpenFile (in /usr/bin/ncl)
==15801==    by 0x5B092C: _NclIAddFile (in /usr/bin/ncl)
==15801==    by 0x5878C8: CallINTRINSIC_FUNC_CALL (in /usr/bin/ncl)
==15801==    by 0x59261C: _NclExecute (in /usr/bin/ncl)
==15801==    by 0x46F4D9: yyparse (in /usr/bin/ncl)
==15801==    by 0x4657EF: NclDriver (in /usr/bin/ncl)
==15801==    by 0xC9796A2: (below main) (in /usr/lib64/libc-2.28.so)

the issue is that gdal modified its internal copy of the g2clib to change g2_getfld() to use 6 arguments and they inserted that argument in the middle.  So ncl is trying to call the standard 5 arg g2_getfld() and ending up with gdal's 6 arg version which leads to chaos.

Comment 2 Orion Poplawski 2020-07-18 04:49:10 UTC
So, I can work around this in ncl by being sure to link g2clib before gdal.  But gdal really should get fixed to prevent other users from having this issue.

Comment 3 Fedora Update System 2020-07-18 14:18:10 UTC
FEDORA-2020-3d7cbc5979 has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2020-3d7cbc5979

Comment 4 Jim E 2020-07-18 20:37:29 UTC
Indeed, ncl seems to be working alright except for this. Looking forward to the new update. Cheers.

Comment 5 Fedora Update System 2020-07-19 01:27:35 UTC
FEDORA-2020-3d7cbc5979 has been pushed to the Fedora 32 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-3d7cbc5979`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-3d7cbc5979

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 6 Fedora Update System 2020-07-19 02:00:15 UTC
FEDORA-EPEL-2020-b7ff36d3e7 has been pushed to the Fedora EPEL 8 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-b7ff36d3e7

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 7 Fedora Admin user for bugzilla script actions 2020-07-28 14:55:49 UTC
This package has changed maintainer in the Fedora.
Reassigning to the new maintainer of this component.

Comment 8 Fedora Update System 2020-07-29 14:38:15 UTC
FEDORA-2020-3d7cbc5979 has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 9 Fedora Update System 2020-07-29 21:16:42 UTC
FEDORA-2020-3d7cbc5979 has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 10 Fedora Update System 2020-07-30 13:07:45 UTC
FEDORA-2020-3d7cbc5979 has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 11 Fedora Update System 2020-07-30 17:50:43 UTC
FEDORA-2020-3d7cbc5979 has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 12 Fedora Update System 2020-08-04 01:22:09 UTC
FEDORA-EPEL-2020-b7ff36d3e7 has been pushed to the Fedora EPEL 8 stable repository.
If problem still persists, please make note of it in this bug report.