Dwifslpreproc Unhandled Python exception error

1

Hi!

I’m extremely new to MRtrix and I’m trying to work my way through some basic tractography preprocessing steps. However, I ran into some unhandled python exceptions when trying to use dwifslpreproc that I couldn’t solve myself. Do anyone now why these occured (and how they can be resolved)?
OS: Ubuntu 18

Output:

karin@bionic-schnauzer:~/mri_testData/DICOM$ dwifslpreproc test_dwi.mif DWI_out.mif -rpe_none -pe_dir ap -readout_time 0.55
dwifslpreproc:
dwifslpreproc: Note that this script makes use of commands / algorithms that have relevant articles for citation; INCLUDING FROM EXTERNAL SOFTWARE PACKAGES. Please consult the help page (-help option) for more information.
dwifslpreproc:

dwifslpreproc: [ERROR] Unhandled Python exception:
dwifslpreproc: [ERROR] UnicodeDecodeError: ‘utf-8’ codec can’t decode byte 0xe4 in position 630: invalid continuation byte
dwifslpreproc: [ERROR] Traceback:
dwifslpreproc: [ERROR] /home/karin/anaconda3/envs/old_mne/bin/dwifslpreproc:92 (in execute())
dwifslpreproc: [ERROR] image.check_3d_nonunity(path.from_user(app.ARGS.input, False))
dwifslpreproc: [ERROR] /home/karin/anaconda3/envs/old_mne/lib/mrtrix3/image.py:134 (in check_3d_nonunity())
dwifslpreproc: [ERROR] image_in = Header(image_in)
dwifslpreproc: [ERROR] /home/karin/anaconda3/envs/old_mne/lib/mrtrix3/image.py:46 (in init())
dwifslpreproc: [ERROR] data = json.loads(json_file.read().decode(‘utf-8’, errors=‘replace’))
dwifslpreproc: [ERROR] /home/karin/anaconda3/envs/old_mne/lib/python3.6/codecs.py:321 (in decode())
dwifslpreproc: [ERROR] (result, consumed) = self._buffer_decode(data, self.errors, final)

Thank you!

2

Welcome Karin!

It’s a fairly obtuse error, so definitely not easy to interpret if you’re not accustomed to such things.

What seems to be the problem is that within the header contents of file test_dwi.mif exists some non-text content that Python refuses to convert to text. It’s probably come from a DICOM header where e.g. a subject name has an accented character or something like that; we’ve made a number of modifications to try to make the software robust to such, but it seems you’ve come across a case we’ve not yet encountered.

What would help is if you could run:
mrinfo test_dwi.mif -json_all test_dwi.json
, and then upload file “test_dwi.json”. Don’t copy and paste the contents of the text file here, as that may obscure the origin of the fault; I think it should be possible to do a direct upload here just by dragging & dropping the file into the text box while writing a reply.

Thanks
Rob

3

Ouch. My guess is the text fields in the DICOM were written in some European encoding (most likely
ISO-8859-1), where the additional non-ASCII characters are not compatible with UTF-8. We don’t perform conversions to UTF-8 within our DICOM import unfortunately – I’m not even sure how to determine the text encoding used within the DICOM headers, we’d have to research this to figure it out.

One option might be to modify our handling in the python scripts to also try reading with ISO-8859-1 encoding in case of failure with UTF-8…?

@karin_westin, maybe you could try modifying the problematic line in dwifslpreproc (line 92) to replace the 'utf-8' with 'iso-8859-1'…? It’s a long shot, but it might allow you to at least run the command for this one dataset…

4

Hello! I received the same error for two (out of 40) participants.
I ran the mrinfo command and got the txt file but I cannot uploaded it so I have to copy paste the content (Sorry):

{
    "datatype": "Float32LE",
    "format": "MRtrix",
    "intensity_offset": 0.0,
    "intensity_scale": 1.0,
    "keyval": {
        "EchoTime": 0.074,
        "FlipAngle": 90,
        "InversionTime": 0,
        "PhaseEncodingDirection": "j-",
        "PixelBandwidth": 2876.87,
        "RepetitionTime": 5,
        "TotalReadoutTime": 0.0173,
        "command_history": [
            "mrconvert DWI/ DWI.mif  (version=3.0.2)",
            "mrconvert DWI.mif DWI.mif -import_pe_table /media/u0136350/EleD/DWI/dwi_manual_pe_scheme1.txt -force  (version=3.0.2)",
            "dwidenoise DWI.mif DWI_deno7.mif -noise noise7.mif -extent '7,7,7'  (version=3.0.2)",
            "mrdegibbs DWI_deno7.mif DWI_deno7_unr.mif -axes '0,1'  (version=3.0.2)"
        ],
        "comments": [
            "GEBRUERS^BRUNO EUG�NE (87441853) [MR] DKI_101d_2800_56sl_1p",
            "study: RAD smr neuro 120 [ ORIGINAL PRIMARY M_SE M SE ]",
            "DOB: 08/04/1947",
            "DOS: 15/02/2023 17:04:30"
        ],
        "dw_scheme": [
            [
                -0.70710676908493,
                -0.707106828689575,
                -4.8264121588204e-09,
                0.5
            ],
            [
                -1.7742398661369e-09,
                -0.70710676908493,
                0.70710676908493,
                0.5
            ],
            [
                0.692424893379211,
                0.658772885799408,
                -0.294220954179764,
                700.0
            ],
            [
                -0.06734599173069,
                -0.977834939956665,
                -0.198250979185104,
                2800.0
            ],
            [
                0.777793049812317,
                -0.500094056129456,
                -0.380715042352676,
                1200.0
            ],
            [
                -0.838028311729431,
                0.211096078157425,
                -0.503137171268463,
                2800.0
            ],
            [
                0.438642024993896,
                0.071385994553566,
                0.895822107791901,
                1200.0
            ],
            [
                0.55715411901474,
                0.283150047063828,
                -0.780644178390503,
                2800.0
            ],
            [
                0.185803934931755,
                0.347333878278732,
                0.91914963722229,
                2800.0
            ],
            [
                -0.648208022117615,
                -0.745291948318481,
                0.156096994876862,
                1200.0
            ],
            [
                -0.28165191411972,
                -0.3818618953228,
                -0.880257725715637,
                700.0
            ],
            [
                -0.731457650661469,
                0.459943771362305,
                0.503409743309021,
                2800.0
            ],
            [
                -0.0377360060811043,
                0.726978063583374,
                -0.685622990131378,
                2800.0
            ],
            [
                -0.110358953475952,
                0.816668629646301,
                -0.56645679473877,
                1200.0
            ],
            [
                0.932438731193542,
                0.349825888872147,
                0.0904429703950882,
                2800.0
            ],
            [
                -0.908841073513031,
                0.405422031879425,
                -0.0981880128383636,
                700.0
            ],
            [
                -0.0185019951313734,
                -0.778653800487518,
                -0.627180874347687,
                1200.0
            ],
            [
                0.476650029420853,
                -0.876188039779663,
                0.0714090019464493,
                2800.0
            ],
            [
                -0.567680954933167,
                -0.793585956096649,
                0.218996986746788,
                2800.0
            ],
            [
                -0.968418896198273,
                -0.0553349927067757,
                -0.243110969662666,
                1200.0
            ],
            [
                0.307196140289307,
                -0.258246123790741,
                0.915936350822449,
                2800.0
            ],
            [
                -0.407881945371628,
                0.91156280040741,
                -0.0518229827284813,
                700.0
            ],
            [
                -0.578205049037933,
                -0.674435138702393,
                -0.459147095680237,
                2800.0
            ],
            [
                -0.46405029296875,
                -0.268962174654007,
                0.843988537788391,
                1200.0
            ],
            [
                0.917925953865051,
                -0.15218198299408,
                -0.366404980421066,
                2800.0
            ],
            [
                0.573760867118835,
                -0.670215845108032,
                0.470753908157349,
                1200.0
            ],
            [
                3.06384295711837e-09,
                -7.3222858887334e-09,
                1.0,
                0.5
            ],
            [
                -0.138969004154205,
                0.580892026424408,
                0.802029967308044,
                2800.0
            ],
            [
                -0.4564468562603,
                -0.363870888948441,
                0.811944723129272,
                700.0
            ],
            [
                0.670235872268677,
                -0.556068956851959,
                0.49149888753891,
                2800.0
            ],
            [
                0.0109130050987005,
                0.980054914951324,
                0.198426991701126,
                1200.0
            ],
            [
                -0.229185044765472,
                -0.198216035962105,
                0.952987194061279,
                2800.0
            ],
            [
                0.739451885223389,
                0.112038969993591,
                0.663820862770081,
                2800.0
            ],
            [
                0.345881074666977,
                -0.172756060957909,
                -0.922237277030945,
                1200.0
            ],
            [
                0.752856016159058,
                0.00861100386828184,
                0.658128917217255,
                700.0
            ],
            [
                0.232782110571861,
                0.960188329219818,
                -0.154437050223351,
                2800.0
            ],
            [
                -0.776854574680328,
                -0.555194675922394,
                -0.297078847885132,
                1200.0
            ],
            [
                0.434972196817398,
                -0.704557359218597,
                -0.560712218284607,
                2800.0
            ],
            [
                -0.808813035488129,
                -0.24099200963974,
                0.536418080329895,
                2800.0
            ],
            [
                0.777745306491852,
                0.354129105806351,
                -0.519331157207489,
                1200.0
            ],
            [
                0.9959916472435,
                -0.0894459709525108,
                0.000263003894360736,
                2800.0
            ],
            [
                -0.0609230026602745,
                0.822552144527435,
                0.565417051315308,
                700.0
            ],
            [
                -0.402919262647629,
                0.810258507728577,
                -0.425602227449417,
                2800.0
            ],
            [
                -0.670716047286987,
                0.741608083248138,
                -0.0125480005517602,
                1200.0
            ],
            [
                0.175303012132645,
                -0.25144299864769,
                -0.951864123344421,
                2800.0
            ],
            [
                -0.382165968418121,
                0.415637969970703,
                -0.825344920158386,
                1200.0
            ],
            [
                0.275056064128876,
                0.849255204200745,
                0.450677067041397,
                2800.0
            ],
            [
                0.871492207050323,
                -0.00360000878572464,
                -0.490396112203598,
                700.0
            ],
            [
                -0.306895017623901,
                -0.811028063297272,
                0.498045027256012,
                2800.0
            ],
            [
                0.0692400187253952,
                0.400898188352585,
                0.913502335548401,
                1200.0
            ],
            [
                -0.878428161144257,
                0.452622085809708,
                0.153288021683693,
                2800.0
            ],
            [
                1.0,
                -1.97052330008773e-09,
                -4.2707404279696e-10,
                0.5
            ],
            [
                0.0577129982411861,
                0.810342073440552,
                -0.583108067512512,
                700.0
            ],
            [
                -0.532166957855225,
                -0.424084901809692,
                -0.732768952846527,
                2800.0
            ],
            [
                -0.965787053108215,
                0.0833859965205193,
                0.245565012097359,
                1200.0
            ],
            [
                0.816806018352509,
                0.525012016296387,
                -0.239144995808601,
                2800.0
            ],
            [
                -0.222533941268921,
                -0.948774755001068,
                0.224287942051888,
                1200.0
            ],
            [
                -0.310588032007217,
                0.911943137645721,
                0.268132030963898,
                2800.0
            ],
            [
                -0.0192640051245689,
                0.443916112184525,
                -0.895861268043518,
                2800.0
            ],
            [
                0.753445625305176,
                0.221924901008606,
                0.61892569065094,
                1200.0
            ],
            [
                0.619103610515594,
                0.66631954908371,
                0.415606737136841,
                700.0
            ],
            [
                -0.466966271400452,
                0.05771604180336,
                0.882389605045319,
                2800.0
            ],
            [
                -0.742919385433197,
                -0.659212291240692,
                -0.116232059895992,
                2800.0
            ],
            [
                -0.417462229728699,
                0.560606360435486,
                0.715154409408569,
                1200.0
            ],
            [
                -0.73222291469574,
                0.665329933166504,
                -0.145552963018417,
                2800.0
            ],
            [
                -0.155493006110191,
                0.241642981767654,
                -0.95782595872879,
                700.0
            ],
            [
                -0.0780149698257446,
                -0.526199817657471,
                0.846774697303772,
                1200.0
            ],
            [
                -0.606890916824341,
                0.261404991149902,
                -0.750567018985748,
                2800.0
            ],
            [
                -0.944238364696503,
                -0.0728780180215836,
                -0.321096122264862,
                2800.0
            ],
            [
                0.91057300567627,
                -0.388727009296417,
                0.140528008341789,
                1200.0
            ],
            [
                0.0508680120110512,
                -0.913985252380371,
                0.40254607796669,
                2800.0
            ],
            [
                0.653514206409454,
                -0.60517817735672,
                -0.454619169235229,
                700.0
            ],
            [
                0.0798719674348831,
                -0.83412367105484,
                -0.545763790607452,
                2800.0
            ],
            [
                0.432559847831726,
                0.858580708503723,
                0.27519291639328,
                1200.0
            ],
            [
                0.610581040382385,
                0.579107046127319,
                -0.540209054946899,
                2800.0
            ],
            [
                0.404462963342667,
                -0.843509912490845,
                -0.353412955999374,
                1200.0
            ],
            [
                5.89649085114274e-09,
                1.0,
                -3.51392626107838e-09,
                0.5
            ],
            [
                0.450241982936859,
                0.086355984210968,
                0.88872092962265,
                2800.0
            ],
            [
                -0.226623013615608,
                -0.973895072937012,
                -0.0130550051108003,
                700.0
            ],
            [
                0.738206088542938,
                -0.103628024458885,
                -0.666568040847778,
                2800.0
            ],
            [
                0.425969868898392,
                0.698339879512787,
                -0.575213968753815,
                1200.0
            ],
            [
                0.643366813659668,
                -0.726657748222351,
                -0.240930929780006,
                2800.0
            ],
            [
                0.453680068254471,
                0.881508111953735,
                0.130835011601448,
                2800.0
            ],
            [
                -0.764744877815247,
                0.251014977693558,
                -0.593427956104279,
                1200.0
            ],
            [
                0.545484304428101,
                -0.594059348106384,
                0.591219365596771,
                700.0
            ],
            [
                0.374248862266541,
                -0.582357883453369,
                0.721662700176239,
                2800.0
            ],
            [
                -0.738896429538727,
                0.118197068572044,
                0.663371384143829,
                1200.0
            ],
            [
                -0.0748299956321716,
                0.0043369960039854,
                -0.997186839580536,
                2800.0
            ],
            [
                0.803580105304718,
                0.402195066213608,
                0.438746094703674,
                2800.0
            ],
            [
                -0.447165161371231,
                -0.577664256095886,
                -0.682896316051483,
                1200.0
            ],
            [
                -0.956851601600647,
                -0.184132933616638,
                0.224788904190063,
                2800.0
            ],
            [
                -0.967645108699799,
                -0.232725024223328,
                -0.0974780172109604,
                700.0
            ],
            [
                0.484682768583298,
                -0.409450829029083,
                -0.772937715053558,
                2800.0
            ],
            [
                -0.0234179981052876,
                0.0984359830617905,
                -0.994867861270905,
                1200.0
            ],
            [
                -0.30902686715126,
                -0.558042883872986,
                0.770123720169067,
                2800.0
            ],
            [
                0.925538063049316,
                0.365415036678314,
                -0.0992530062794685,
                1200.0
            ],
            [
                -0.229649037122726,
                -0.643482089042664,
                -0.730200052261353,
                2800.0
            ],
            [
                -0.34744992852211,
                0.261399954557419,
                0.900526821613312,
                700.0
            ],
            [
                -0.149285972118378,
                0.985596776008606,
                -0.0794519856572151,
                2800.0
            ],
            [
                0.281391084194183,
                -0.94612729549408,
                0.160194054245949,
                1200.0
            ],
            [
                0.90826028585434,
                -0.365002155303955,
                0.204540058970451,
                2800.0
            ],
            [
                -3.06384295711837e-09,
                7.3222858887334e-09,
                -1.0,
                0.5
            ]
        ],
        "mrtrix_version": "3.0.2"
    },
    "name": "DWI_deno7_unr.mif",
    "size": [
        96,
        96,
        56,
        102
    ],
    "spacing": [
        2.5,
        2.5,
        2.5,
        null
    ],
    "strides": [
        -1,
        -2,
        3,
        4
    ],
    "transform": [
        [
            0.992954417263766,
            -0.0860745485197903,
            0.0814413777706694,
            -108.323726889064
        ],
        [
            0.0959563985195514,
            0.987323074870284,
            -0.126433835101266,
            -102.343584235856
        ],
        [
            -0.0695262167785143,
            0.133357856633202,
            0.988626212139047,
            -73.5561112314834
        ],
        [
            0.0,
            0.0,
            0.0,
            1.0
        ]
    ]
}

Thanks!

5

As an update, I just checked the name of the two participants and both of them contain accents. How could I change this/fix it?

Thank you!

6

I ran into the same problem, caused by “ä” in the header. In fact, in geraldines data one can see that there is a problem with the encoding of the (probably) è in the name.

I’m reviving this old thread to provide a quick solution on how to edit the header fields, probably too late for geraldine, but maybe helpful for other people in the future.

Assuming your file is called dwi.mif (replace accordingly) and the problem is in the comments header field:

mrinfo dwi.mif -property comments >> comments.txt

You’ll now see a file comments.txt with the content of the comments header field. Find and remove all offending characters, and replace with basic ASCII characters (see wikipedia )

Now, Insert the edited header information into the file:

mrconvert dwi.mif -set_property comments "$(cat comments.txt)" dwi_fixed.mif

dwi_fixed.mif should now work with dwifslpreproc.