abcd-hcp 1.3 and 1.4 do not produce all relevant folders and files for rsfc data

[johannesjuliusm]

What happened?

When using versions 1.4 and 1.3 of the abcd-hcp pipeline, we encounter a perplexing issue where certain datasets undergo preprocessing without generating the expected DCAN_BOLD_Proc_4.0.0 folder. This occurs inconsistently across datasets despite them being thoroughly checked and organized in BIDS structure. Surprisingly, the pipeline completes without any reported errors or warnings. However, for a subset of the data, essential files such as ptseries and movement .mat files are not produced along with the designated folder. Upon examination of the raw data, no discernible differences are found between the successfully processed and problematic datasets. Adding to the complexity, the subset of data that processes successfully with version 1.3 may fail with version 1.4, and vice versa. Notably, the datasets that do process successfully remain consistent for subsequent pipeline runs, indicating a lack of randomness. This issue appears to be linked partially to the pipeline version and possibly some underlying characteristics of the data that we have yet to uncover.

What command did you use?

We use the singularity image

What version of the abcd-hcp pipeline are you using?

.0.1.3 and .0.1.4

Directory Structure

No response

Relevant log output

No response

Add any additional information or context about the problem here.

No response

[madisoth]

Hello and apologies for the late response-- to your issue, can you share the logs directory (sub-<subject>/ses<session>/logs/) for a subject that ran as 0.1.3 but had missing files with 0.1.4, or vice versa, or just the logs from any pipeline stage that show a discrepancy between 0.1.3 and 0.1.4? Thanks!