Average_response error: Input files do not contain the same number of b-values

Hey all,

I’m trying to run average_response on a number of white matter response functions text files. Here is my command: average_response /DIR/wm_avg/*.txt /DIR/wm_group.txt -force -debug
When I run it though I get the error message: Error in file /DIR/wm_group.txt: Input files do not contain
the same number of b-values
Initially I was trying it with 13 subjects to pilot the pipeline but upon running into this error I reduced it to two to see if that would help but I’m still getting the same error. As a reference, here is the content of the wm_rf.txt files

301.txt contains:

24394.47219637542 0 0 0 0 0
19486.48326934965 -2364.482064558061 231.928722947274 19.1686149492679 35.31871269379779 8.877404144922922
12011.40569480979 -4728.281741359913 1099.832650873847 -169.6599138862255 11.03630098120721 15.82008702188409
8227.014276358388 -4455.099320099695 1848.15164166984 -474.9620748224591 95.46426018647716 -31.87337134764449
6411.417642042889 -3671.204422615961 2052.293682999709 -854.4040644118663 272.0434564133208 -57.96439047986732
5253.624064573517 -2918.068339736795 1928.981763487655 -1056.009857972033 429.4356575789573 -122.0034440111275

660.txt contains:

21762.08489372593 0 0 0 0 0
17711.46522888893 -2150.96577102586 217.3336345564332 195.7515279000222 10.73993578241922 -58.12449907440799
11067.36253053835 -4271.394086499094 957.0532318616922 -142.5638988169535 55.80151041799036 -7.572190794065395
7760.578164027334 -4062.350790678182 1605.345729429341 -382.7646304451376 67.50613372913038 -18.118542672746
6065.9544102743 -3351.391274665364 1763.314241837077 -661.6432345226233 149.680489212285 -19.33240016847093
4990.704801759557 -2683.412515951854 1677.769276284188 -845.6849682949255 319.9622512025565 -88.30889749826103

As far as I can tell, they both contain 6 lines with 6 values each so I’m not sure why the command thinks they contain different numbers of bvals.

average_response is implemented as a python script, and looking at the current code, it looks like the number of shells is determined simply by the number of lines in the file, with no attempt to ignore empty lines. All it would take for that test to fail is an additional empty line at the end of the file. You can use wc -l 660.txt to check the number of lines in each response file. If they don’t match (which I assume is the issue), open the one with too many lines in a text editor, remove any superfluous empty lines, save and try again. Hopefully that’ll fix it…

They actually have the same number of lines. Any other suggestions?23%20PM

Upon further investigation, all of the files have always had the same number of lines:

The script itself is relatively simple, so it would not be too difficult to go in and see how the script itself is reading the data (as opposed to how some other unix command reads / displays the contents). E.g. After line 19:

    lines = f.readlines()

, you could insert:

    print (len(lines))
    print (lines)

; that would tell you how many lines the script thinks that each file contains, and the contents of those lines. Seeing where the discrepancy is between how data from the two files is read would likely give a strong hint as to where the discrepancy is in the underlying data.

It wouldn’t surprise me if one of the files contains an empty line at the end whereas the other doesn’t, or one of the files doesn’t have a newline character at the end of the last data line whereas the other does, and this difference is being obscured currently because the wc command is based on different code and is accounting for such discrepancy intelligently whereas average_response is not.

When I added the line print(len(lines)) it spits out the below message. First of all, I’m feeding it 13 subjects so I’m not sure where its getting the idea that there are 15 subjects. Secondly, it spits out 14 different lines, the last of which is 0, which leads me to believe that the problem is that the last “phantom” subject the command is trying to deal with is the problem. How do I remedy this when the error is not caused by one of the .txt files I feed it but rather by some mysterious phantom which I cannot identify?

OK, it could be that you have hidden files in that folder? What does ls -a /DIR/wm_avg/ report? Alternatively, what’s the output if you put the word echo at the very beginning of your command?