Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1838435 - python-pydicom fails to build with numpy 1.19: TypeError: data type 'uint15' not understood
Summary: python-pydicom fails to build with numpy 1.19: TypeError: data type 'uint15' ...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: python-pydicom
Version: 33
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Alessio
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: F33FTBFS
TreeView+ depends on / blocked
 
Reported: 2020-05-21 07:11 UTC by Miro Hrončok
Modified: 2021-02-24 22:40 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-02-24 22:40:26 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github pydicom pydicom issues 1119 0 None closed test_handler_util fails with numpy 1.19 2021-01-18 13:18:33 UTC

Description Miro Hrončok 2020-05-21 07:11:57 UTC
python-pydicom fails to build with Python 3.9.0b1.

_____________ TestNumpy_PaletteColor.test_invalid_bit_depth_raises _____________

self = <pydicom.tests.test_handler_util.TestNumpy_PaletteColor object at 0x7f0c4fe0ae50>

    def test_invalid_bit_depth_raises(self):
        """Test that an invalid bit depth raises an exception."""
        ds = dcmread(PAL_08_256_0_16_1F)
        ds.RedPaletteColorLookupTableDescriptor[2] = 15
        msg = (
            r'data type "uint15" not understood'
        )
        with pytest.raises(TypeError, match=msg):
>           apply_color_lut(ds.pixel_array, ds)

pydicom/tests/test_handler_util.py:921: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

arr = array([[244, 244, 244, ..., 244, 244, 244],
       [244, 244, 244, ..., 244, 244, 244],
       [244, 244, 244, ..., 24...,   0,   0,   0],
       [  0,   0,   0, ...,   0,   0,   0],
       [  0,   0,   0, ...,   0,   0,   0]], dtype=uint8)
ds = (0008, 0005) Specific Character Set              CS: 'ISO_IR 100'
(0008, 0008) Image Type                          CS:...ion LUT Shape              CS: 'IDENTITY'
(7fe0, 0010) Pixel Data                          OW: Array of 480000 elements
palette = None

    def apply_color_lut(arr, ds=None, palette=None):
        """Apply a color palette lookup table to `arr`.
    
        .. versionadded:: 1.4
    
        If (0028,1201-1203) *Palette Color Lookup Table Data* are missing
        then (0028,1221-1223) *Segmented Palette Color Lookup Table Data* must be
        present and vice versa. The presence of (0028,1204) *Alpha Palette Color
        Lookup Table Data* or (0028,1224) *Alpha Segmented Palette Color Lookup
        Table Data* is optional.
    
        Use of this function with the :dcm:`Enhanced Palette Color Lookup Table
        Module<part03/sect_C.7.6.23.html>` or :dcm:`Supplemental Palette Color LUT
        Module<part03/sect_C.7.6.19.html>` is not currently supported.
    
        Parameters
        ----------
        arr : numpy.ndarray
            The pixel data to apply the color palette to.
        ds : dataset.Dataset, optional
            Required if `palette` is not supplied. A
            :class:`~pydicom.dataset.Dataset` containing a suitable
            :dcm:`Image Pixel<part03/sect_C.7.6.3.html>` or
            :dcm:`Palette Color Lookup Table<part03/sect_C.7.9.html>` Module.
        palette : str or uid.UID, optional
            Required if `ds` is not supplied. The name of one of the
            :dcm:`well-known<part06/chapter_B.html>` color palettes defined by the
            DICOM Standard. One of: ``'HOT_IRON'``, ``'PET'``,
            ``'HOT_METAL_BLUE'``, ``'PET_20_STEP'``, ``'SPRING'``, ``'SUMMER'``,
            ``'FALL'``, ``'WINTER'`` or the corresponding well-known (0008,0018)
            *SOP Instance UID*.
    
        Returns
        -------
        numpy.ndarray
            The RGB or RGBA pixel data as an array of ``np.uint8`` or ``np.uint16``
            values, depending on the 3rd value of (0028,1201) *Red Palette Color
            Lookup Table Descriptor*.
    
        References
        ----------
    
        * :dcm:`Image Pixel Module<part03/sect_C.7.6.3.html>`
        * :dcm:`Supplemental Palette Color LUT Module<part03/sect_C.7.6.19.html>`
        * :dcm:`Enhanced Palette Color LUT Module<part03/sect_C.7.6.23.html>`
        * :dcm:`Palette Colour LUT Module<part03/sect_C.7.9.html>`
        * :dcm:`Supplemental Palette Color LUTs
          <part03/sect_C.8.16.2.html#sect_C.8.16.2.1.1.1>`
        """
        # Note: input value (IV) is the stored pixel value in `arr`
        # LUTs[IV] -> [R, G, B] values at the IV pixel location in `arr`
        if not ds and not palette:
            raise ValueError("Either 'ds' or 'palette' is required")
    
        if palette:
            # Well-known palettes are all 8-bits per entry
            datasets = {
                '1.2.840.10008.1.5.1': 'hotiron.dcm',
                '1.2.840.10008.1.5.2': 'pet.dcm',
                '1.2.840.10008.1.5.3': 'hotmetalblue.dcm',
                '1.2.840.10008.1.5.4': 'pet20step.dcm',
                '1.2.840.10008.1.5.5': 'spring.dcm',
                '1.2.840.10008.1.5.6': 'summer.dcm',
                '1.2.840.10008.1.5.7': 'fall.dcm',
                '1.2.840.10008.1.5.8': 'winter.dcm',
            }
            if not UID(palette).is_valid:
                try:
                    uids = {
                        'HOT_IRON': '1.2.840.10008.1.5.1',
                        'PET': '1.2.840.10008.1.5.2',
                        'HOT_METAL_BLUE': '1.2.840.10008.1.5.3',
                        'PET_20_STEP': '1.2.840.10008.1.5.4',
                        'SPRING': '1.2.840.10008.1.5.5',
                        'SUMMER': '1.2.840.10008.1.5.6',
                        'FALL': '1.2.840.10008.1.5.8',
                        'WINTER': '1.2.840.10008.1.5.7',
                    }
                    palette = uids[palette]
                except KeyError:
                    raise ValueError("Unknown palette '{}'".format(palette))
    
            try:
                from pydicom import dcmread
                fname = datasets[palette]
                ds = dcmread(get_palette_files(fname)[0])
            except KeyError:
                raise ValueError("Unknown palette '{}'".format(palette))
    
        # C.8.16.2.1.1.1: Supplemental Palette Color LUT
        # TODO: Requires greyscale visualisation pipeline
        if getattr(ds, 'PixelPresentation', None) in ['MIXED', 'COLOR']:
            raise ValueError(
                "Use of this function with the Supplemental Palette Color Lookup "
                "Table Module is not currently supported"
            )
    
        # All channels are supposed to be identical
        lut_desc = ds.RedPaletteColorLookupTableDescriptor
        # A value of 0 = 2^16 entries
        nr_entries = lut_desc[0] or 2**16
    
        # May be negative if Pixel Representation is 1
        first_map = lut_desc[1]
        # Actual bit depth may be larger (8 bit entries in 16 bits allocated)
        nominal_depth = lut_desc[2]
>       dtype = np.dtype('uint{:.0f}'.format(nominal_depth))
E       TypeError: data type 'uint15' not understood

pydicom/pixel_data_handlers/util.py:125: TypeError

During handling of the above exception, another exception occurred:

self = <pydicom.tests.test_handler_util.TestNumpy_PaletteColor object at 0x7f0c4fe0ae50>

    def test_invalid_bit_depth_raises(self):
        """Test that an invalid bit depth raises an exception."""
        ds = dcmread(PAL_08_256_0_16_1F)
        ds.RedPaletteColorLookupTableDescriptor[2] = 15
        msg = (
            r'data type "uint15" not understood'
        )
        with pytest.raises(TypeError, match=msg):
>           apply_color_lut(ds.pixel_array, ds)
E           AssertionError: Pattern 'data type "uint15" not understood' not found in "data type 'uint15' not understood"


This seem related to the new version of numpy (1.19.0 rc1) that has been built form dis git in the Python 3.9 copr.


For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.9/fedora-rawhide-x86_64/01397586-python-pydicom/

For all our attempts to build python-pydicom with Python 3.9, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/package/python-pydicom/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.9:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/

Let us know here if you have any questions.

Python 3.9 will be included in Fedora 33. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.9.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Miro Hrončok 2020-05-21 11:59:26 UTC
numpy 1.19 was temporarily reverted, but I'm keeping this open for future reference

Comment 2 Alessio 2020-06-03 12:20:37 UTC
Reported upstream. Is the report correct?
https://github.com/pydicom/pydicom/issues/1119


I added this conditional in the spec file:

%if "%{python3_version}" == "3.9"
   %{__python3} -m pytest -k "not test_color_3d and not test_handler_util"
%else
   %{__python3} -m pytest -k "not test_color_3d"
%endif

Is it ok?

Comment 3 Miro Hrončok 2020-06-03 12:31:57 UTC
> Reported upstream. Is the report correct?

The report says "test_handler_util fails with Python 3.9", while it should really say "test_handler_util fails with numpy 1.19"

> Is it ok?

Technically, the problem is numpy related, not python version related. So you might as well change it to %if 0%{?fedora} > 32. But what you did will do (and hopefully the issue will be fixed before Python 3.10).

Comment 4 Ankur Sinha (FranciscoD) 2020-06-03 12:38:32 UTC
+1

Could you also attach the build.log and root.log files from the failed build to the upstream issue. That way upstream knows exactly what versions of packages the error was observed with.

If you're working on this Alessio, should we assign it to you? :)

Comment 5 Alessio 2020-06-03 12:47:41 UTC
(In reply to Ankur Sinha (FranciscoD) from comment #4)
> +1
> 
> Could you also attach the build.log and root.log files from the failed build
> to the upstream issue. That way upstream knows exactly what versions of
> packages the error was observed with.

Will do.

> 
> If you're working on this Alessio, should we assign it to you? :)

If you like. Btw I will ping you for each doubt :-)

Comment 6 Ankur Sinha (FranciscoD) 2020-06-03 14:07:20 UTC
Sure, always happy to help! If you're unsure, feel free to use PRs so we can go over the changes before pushing to the actual SCM.

Comment 7 Ben Cotton 2020-08-11 13:33:33 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 33 development cycle.
Changing version to 33.


Note You need to log in before you can comment on or make changes to this bug.