NA-MIC Project Weeks

Back to Projects List

Whitematteranalysis To Python3

Key Investigators

• Scott Lill (Univestiy of Sydney, Harvard Medical School)
• Steve Pieper (Harvard Medical School)
• Sandy Wells (Harvard Medical School)
• Yang Song (University of New South Wales)
• Tom Cai (University of Sydney)
• Fan Zhang (Harvard Medical School)
• Lauren O’Donnell (Harvard Medical School)

Project Description

Whitematteranalysis is a python pipeline for performing fiber processing and clustering. The project aims to upgrade this package, porting it to python3, adding a comprehensive suite of regression tests and porting it into 3D Slicer. This will make the pipeline easier to use, maintain and extend, all of which are beneficial for applicable dMRI workflows.

Objective

1. Objective A. Port the codebase to python3, adding regression tests to ensure consistency.
2. Objective B. Bundle the pipeline into 3D Slicer, adding a qt based GUI as an alternative to the CLI interface
3. Objective C. Add regression tests for existing and added functionality.

Approach and Plan

1. Test and iterate the python version conversion
2. Explore options for running python externally to 3D slicer to allow the porting of required dependancies.
3. Document functionality to ensure coverage for the testing suite

Progress

1. Transited codebase into python3 syntactically with automated upgrading
2. Manual review of codebase to fix error in the automatic review Sanity testing of entire pipeline - running through with both python 2 and 3 for consistency within a certain tolerance, due to stochastic functionality.
3. Fixing errors caused by library changes, revealed by sanity testing.
4. Testing with controlled stochasticity via a python testing environment. Changes also applied to python2, to allow comparisons.
5. Ensuring portability in serialisation format. Settled on providing backwards compatibility of files serialised under python2 by making the python3 code able to read python2 or python3 objects, but only support writing python3. This will shift serialised objects towards python3 standards and facilitate eventually dropping python2 support.
6. Wrote a script for bundling whitematteranalysis (WMA) in its own python environment, seperate from the system python or Slicer python. This is necessary for porting WMA to Slicer, as it uses dependancies unsupported by Slicer’s python and the system python not under the control of Slicer and therefore unreliable. The next step of writing a GUI extension in Slicer was not completed.

Illustrations

Github links of the updated repos:

• Python3 branch: https://github.com/ScLill/whitematteranalysis/tree/2to3
• Testing branch：https://github.com/ScLill/whitematteranalysis/tree/2test

Clustering done under python3 View 1 View 2 View 3

Next Steps

Integrate into the master branch of the WMA repo.

Background and References

Reference Documentation for original project - https://github.com/SlicerDMRI/whitematteranalysis/blob/master/doc/subject-specific-tractography-parcellation.md