Loading Volumes and Creating Segments
Get started with creating an extension, loading volumes, and creating segments using an embedded SegmentEditor widget.
May 19, 2024
10 min Read
By: Abhilaksh Singh Reen
In this Blog Post, we'll be creating an Extension for 3D Slicer which is a free and open source software used for analysis of medical images. A Slicer Extension allows us to easily add custom functionality by building on top of the APIs provided by Slicer. Today, we'll be making an Extension that allows us to load a volume and create segments using a SegmentEditor widget embedded into the extension.
The GitHub Repo for this Project is available here.
In these posts, we're not doing anything fancy - they were written as tutorials that can help developers better understand how to work with 3D Slicer Extensions.
Getting Started
Enable Developer Mode
Head to Edit > Application Settings > Developer and check the box next to Enable developer mode.
Developer mode is not necessary for creating an extension but it will make our lives easier by providing some easy ways we can Reload, Test, and Edit our modules.
The Extension Wizard
Slicer offers a module called Extension Wizard that Create, Load, and Edit extensions. We can find this module in Modules > Developer Tools > Extension Wizard
Hit the Create Extension button and give your extension a name and a destination. We'll leave the type to default. Move to the next screen. Now, optionally, you can give the extension a Description and credit the Contributors.
An Extension in Slicer is just a collection of modules, so let's add a module to our Extension. Hit the Add Module to Extension button. You can give your module a name. For this tutorial, we'll be creating a scripted type module that allows us to write our logic in Python,
If you now go to the destination folder where the Extension's files are stored, you will see that a new folder with the name of your module has been created in it. The destination folder's directory structure should now look something like the following.
│
│ CMakeLists.txt
│ TutorialExtension.png
│
└───TutorialModule
│ CMakeLists.txt
│ TutorialModule.py
│
├───Resources
│ ├───Icons
│ │ TutorialModule.png
│ │
│ └───UI
│ TutorialModule.ui
│
└───Testing
│ CMakeLists.txt
│
└───Python
CMakeLists.txtIf we go back to Slicer and open up the Modules dropdown, in the Examples category, we should see a new module called TutorialModule (you will have to look for the category and name that you specified). Clicking on this module will open it up and you'll see the default module GUI with three dropdowns: Inputs, Outputs, and Advanced.
To avoid having to open the Modules dropdown every time we want to use our module, we can add it to Favorites.
Go to Edit > Application Settings > Modules
Find your module in the Modules section and drag it into the Favorite Modules section. You should see the module's logo appear in the Menu Bar.
Developing our Module
If we take a look at the TutorialModule folder at TutorialExtension/TutorialModule, we'll a file called TutorialModule.py that contains the following four classes:
1) TutorialModule: defines the module description and acknowledgments.
1) TutorialModuleWidget: to contain UI-related logic.
1) TutorialModuleLogic: to contain the computation done by the module.
1) TutorialModuleTest: to contain test cases for our module.
If you wish, you can separate the Widget, Logic, and Test classes into their own files. Right now, we'll just leave things the way they are.
Inside this folder, we'll also find another folder called Resources. Resources/Icons contains an image that is the icon of our module and Resources/UI contains a .ui file that we can edit with Qt Designer to manipulate the GUI of the module.
The End Goal
Let's start with the User Interface: we'll create an input where the user can enter the path to a volume (.nii or .nii.gz). The user can then click the Load Volume button to load that volume into Slicer. Finally, we'll have an embedded qMRMLSegmentEditorWidget that we can use to draw segments on the slices of our volume.
The UI
Let's start off by designing the UI for our extension. Open up the TutorialModule/Resources/UI/TutorialModule.ui file in Qt Designer - you can also do this by clicking the Edit UI button on Slicer under the Reload & Test section when your extension is loaded.
First, delete everything from the GUI. Now, we can build our own UI.
1) First, drag and drop a qLineEdit and give it the name volumePathTextBox.
2) Then, below it, drop a qPushButton and name it loadVolumeButton. Change the text property of this object to say Load Volume.
Save the file from Qt Designer. The entire TutorialModule.ui file should now look something like the following:
<?xml version="1.0" encoding="UTF-8"?>
<ui version="4.0">
<class>TutorialModule</class>
<widget class="qMRMLWidget" name="TutorialModule">
<property name="geometry">
<rect>
<x>0</x>
<y>0</y>
<width>316</width>
<height>338</height>
</rect>
</property>
<layout class="QVBoxLayout" name="verticalLayout">
<item>
<widget class="QLineEdit" name="volumePathTextBox"/>
</item>
<item>
<widget class="QPushButton" name="loadVolumeButton">
<property name="text">
<string>Load Volume</string>
</property>
</widget>
</item>
<item>
<spacer name="verticalSpacer">
<property name="orientation">
<enum>Qt::Vertical</enum>
</property>
<property name="sizeHint" stdset="0">
<size>
<width>20</width>
<height>40</height>
</size>
</property>
</spacer>
</item>
</layout>
</widget>
<customwidgets>
<customwidget>
<class>qMRMLWidget</class>
<extends>QWidget</extends>
<header>qMRMLWidget.h</header>
<container>1</container>
</customwidget>
</customwidgets>
<resources/>
<connections/>
</ui>Go back to Slicer and hit Reload, you should see that the UI of the extension gets updated. For now, we can ignore the errors in the console.
Let's also add our embedded qMRMLSegmentEditorWidget. Unfortunately, we can't really do this in Qt Designer. Open up the TutorialModule.ui file in a code editor and add the following lines after the ending item tag of the Load Volume button.
<item>
<widget class="qMRMLSegmentEditorWidget" name="embeddedSegmentEditorWidget">
<property name="autoShowSourceVolumeNode">
<bool>true</bool>
</property>
<property name="maximumNumberOfUndoStates">
<number>10</number>
</property>
</widget>
</item>Now, if we head back to Slicer and hit Reload & Test again, we see that a Segment Editor widget has been created right below the Load Volume button.
That's all we'll be doing with the UI file today. Here is the entire TutorialModule/Resources/UI/TutorialModule.ui file:
<?xml version="1.0" encoding="UTF-8"?>
<ui version="4.0">
<class>TutorialModule</class>
<widget class="qMRMLWidget" name="TutorialModule">
<property name="geometry">
<rect>
<x>0</x>
<y>0</y>
<width>316</width>
<height>338</height>
</rect>
</property>
<layout class="QVBoxLayout" name="verticalLayout">
<item>
<widget class="QLineEdit" name="volumePathTextBox"/>
</item>
<item>
<widget class="QPushButton" name="loadVolumeButton">
<property name="text">
<string>Load Volume</string>
</property>
</widget>
</item>
<item>
<widget class="qMRMLSegmentEditorWidget" name="embeddedSegmentEditorWidget">
<property name="autoShowSourceVolumeNode">
<bool>true</bool>
</property>
<property name="maximumNumberOfUndoStates">
<number>10</number>
</property>
</widget>
</item>
<item>
<spacer name="verticalSpacer">
<property name="orientation">
<enum>Qt::Vertical</enum>
</property>
<property name="sizeHint" stdset="0">
<size>
<width>20</width>
<height>40</height>
</size>
</property>
</spacer>
</item>
</layout>
</widget>
<customwidgets>
<customwidget>
<class>qMRMLWidget</class>
<extends>QWidget</extends>
<header>qMRMLWidget.h</header>
<container>1</container>
</customwidget>
</customwidgets>
<resources/>
<connections/>
</ui>The Widget
The errors we can see in Slicer are arising from the TutorialModuleWidget class because we deleted a bunch of UI resources from our UI file but the TutorialModuleWidget class still expects to find these resources. If we quickly search for ".connect", we see that the class is expecting resources like self.ui.inputSelector, which we have deleted from our UI file.
So, let's remove the occurrences of these UI elements until we get rid of all the errors.
In the TutorialModuleWidget.setup function, we can delete the following lines:
# These connections ensure that whenever user changes some settings on the GUI, that is saved in the MRML scene
# (in the selected parameter node).
self.ui.inputSelector.connect("currentNodeChanged(vtkMRMLNode*)", self.updateParameterNodeFromGUI)
self.ui.outputSelector.connect("currentNodeChanged(vtkMRMLNode*)", self.updateParameterNodeFromGUI)
self.ui.imageThresholdSliderWidget.connect("valueChanged(double)", self.updateParameterNodeFromGUI)
self.ui.invertOutputCheckBox.connect("toggled(bool)", self.updateParameterNodeFromGUI)
self.ui.invertedOutputSelector.connect("currentNodeChanged(vtkMRMLNode*)", self.updateParameterNodeFromGUI)
# Buttons
self.ui.applyButton.connect('clicked(bool)', self.onApplyButton)There are more references to these elements in the TutorialModuleWidget.updateGUIFromParameterNode function, let's get rid of these as well:
# Update node selectors and sliders
# self.ui.inputSelector.setCurrentNode(self._parameterNode.GetNodeReference("InputVolume"))
# self.ui.outputSelectorThere.setCurrentNode(self._parameterNode.GetNodeReference("OutputVolume"))
# self.ui.invertedOutputSelector.setCurrentNode(self._parameterNode.GetNodeReference("OutputVolumeInverse"))
# self.ui.imageThresholdSliderWidget.value = float(self._parameterNode.GetParameter("Threshold"))
# self.ui.invertOutputCheckBox.checked = (self._parameterNode.GetParameter("Invert") == "true")
# Update buttons states and tooltips
# if self._parameterNode.GetNodeReference("InputVolume") and self._parameterNode.GetNodeReference("OutputVolume"):
# self.ui.applyButton.toolTip = "Compute output volume"
# self.ui.applyButton.enabled = True
# else:
# self.ui.applyButton.toolTip = "Select input and output volume nodes"
# self.ui.applyButton.enabled = FalseWe can also delete the onApplyButton function of TutorialWidget since that is not being used anywhere.
Now, if we hit Reload again, we should see that there are no errors.
Great, so let's set up on-click handlers for the Load Volume button. For now, we'll just make it so that it prints the text from the text box to the console.
Inside the TutorialWidget class, we make a function that will be called when our button is clicked.
def onLoadVolumeButtonClick(self):
print(self.ui.volumePathTextBox.text)Inside the setup function, before self.initializeParameterNode(), we can add the following line to connect the button to the click handler.
self.ui.loadVolumeButton.connect('clicked(bool)', self.onLoadVolumeButtonClick)Head back to Slicer and hit Reload. Now, if we type something into the text box and click the button, we should get the same string printed in the console.
Now, let's actually load the volume. Inside the setup function, you will find a line self.logic = TutorialModuleLogic(). The TutorialModuleLogic class should contain the computations performed by our module. So, that is the place we will use to load our volume. Since we already have an instance of TutorialModuleLogic stored in self.logic, we can update the onLoadVolumeButtonClick function to the following:
def onLoadVolumeButtonClick(self):
volumePath = self.ui.volumePathTextBox.text
self.logic.loadVolume(volumePath)The Logic
Inside the TutorialModuleLogic class, we can get rid of the process function and create a new function called loadVolume.
def loadVolume(self, volumePath):
if not os.path.exists(volumePath):
slicer.util.errorDisplay("The given volume path does not exist.")
return
if not os.path.isfile(volumePath) or \
(not volumePath.endswith(".nii") and not volumePath.endswith(".nii.gz")):
slicer.util.errorDisplay("The given path is not a Nifti image.")
return
if volumePath == self.loadedVolumePath:
slicer.util.messageBox("This volume is already loaded.")
return
self.volumeNode = slicer.util.loadVolume(volumePath)
return self.volumeNodeWe should add default values for self.loadedVolumePath and self.volumeNode. Inside the __init__ function, add these lines at the end.
self.loadedVolumePath = None
self.volumeNode = NoneOnce again, head back to Slicer and hit Reload. If you enter the path to a .nii.gz file in the text box and hit the Load Volume button, the volume will be loaded into Slicer and you should be able to see the Axial, Coronal, and Sagittal slices.
But our segment editor widget is still disabled. Let's fix that.
Segmentation Node and the Segment Editor
In order for the Segment Editor widget to be activated, we need to give it three things:
1) A SourceVolume node: we have this already being returned by the loadVolume function.
2) A Segmentation Node
3) A SegmentEditor Node
Inside the TutorialModuleLogic class, create a new function that will allow us to get the SegmentEditor Node:
def getSegmentEditorNode(self):
segmentEditorSingletonTag = "SegmentEditor"
segmentEditorNode = slicer.mrmlScene.GetSingletonNode(segmentEditorSingletonTag, "vtkMRMLSegmentEditorNode")
if segmentEditorNode is None:
segmentEditorNode = slicer.mrmlScene.CreateNodeByClass("vtkMRMLSegmentEditorNode")
segmentEditorNode.UnRegister(None)
segmentEditorNode.SetSingletonTag(segmentEditorSingletonTag)
segmentEditorNode = slicer.mrmlScene.AddNode(segmentEditorNode)
return segmentEditorNodeCreate another function to get the Segmentation Node
def getSegmentationNode(self):
if self.segmentationNode is not None:
return
self.segmentationNode = slicer.mrmlScene.AddNewNodeByClass("vtkMRMLSegmentationNode")
return self.segmentationNodeYou can see that we have made references to the variables self.loadedVolumePath, self.volumeNode, self.segmentationNode, and self.segmentEditorNode before defining them. This will give us errors. So, in the __init__ functions, let's define and initialize these variables to None.
self.loadedVolumePath = None
self.volumeNode = None
self.segmentationNode = None
self.segmentEditorNode = NoneHere is the entire TutorialModuleLogic class:
class TutorialModuleLogic(ScriptedLoadableModuleLogic):
"""This class should implement all the actual
computation done by your module. The interface
should be such that other python code can import
this class and make use of the functionality without
requiring an instance of the Widget.
Uses ScriptedLoadableModuleLogic base class, available at:
https://github.com/Slicer/Slicer/blob/main/Base/Python/slicer/ScriptedLoadableModule.py
"""
def __init__(self):
"""
Called when the logic class is instantiated. Can be used for initializing member variables.
"""
ScriptedLoadableModuleLogic.__init__(self)
self.loadedVolumePath = None
self.volumeNode = None
self.segmentationNode = None
self.segmentEditorNode = None
def setDefaultParameters(self, parameterNode):
"""
Initialize parameter node with default settings.
"""
if not parameterNode.GetParameter("Threshold"):
parameterNode.SetParameter("Threshold", "100.0")
if not parameterNode.GetParameter("Invert"):
parameterNode.SetParameter("Invert", "false")
def getSegmentEditorNode(self):
segmentEditorSingletonTag = "SegmentEditor"
segmentEditorNode = slicer.mrmlScene.GetSingletonNode(segmentEditorSingletonTag, "vtkMRMLSegmentEditorNode")
if segmentEditorNode is None:
segmentEditorNode = slicer.mrmlScene.CreateNodeByClass("vtkMRMLSegmentEditorNode")
segmentEditorNode.UnRegister(None)
segmentEditorNode.SetSingletonTag(segmentEditorSingletonTag)
self.segmentEditorNode = slicer.mrmlScene.AddNode(segmentEditorNode)
return self.segmentEditorNode
def getSegmentationNode(self):
if self.segmentationNode is not None:
return
self.segmentationNode = slicer.mrmlScene.AddNewNodeByClass("vtkMRMLSegmentationNode")
return self.segmentationNode
def loadVolume(self, volumePath):
if not os.path.exists(volumePath):
slicer.util.errorDisplay("The given volume path does not exist.")
return
if not os.path.isfile(volumePath) or \
(not volumePath.endswith(".nii") and not volumePath.endswith(".nii.gz")):
slicer.util.errorDisplay("The given path is not a Nifti image.")
return
if volumePath == self.loadedVolumePath:
slicer.util.messageBox("This volume is already loaded.")
return
self.volumeNode = slicer.util.loadVolume(volumePath)
return self.volumeNodeThe Widget (Again)
In order to set the SourceVolume, SegmentEditor, and Segmentation nodes of the SegmentEditorWidget, we have to visit the TutorialModuleWidget class once again.
Inside the onLoadVolumeButtonClick function, we'll add the logic to set these nodes.
def onLoadVolumeButtonClick(self):
volumePath = self.ui.volumePathTextBox.text
volumeNode = self.logic.loadVolume(volumePath)
if volumeNode is None:
return
segmentationNode = self.logic.getSegmentationNode()
segmentEditorNode = self.logic.getSegmentEditorNode()
self.ui.embeddedSegmentEditorWidget.setMRMLScene(slicer.mrmlScene)
self.ui.embeddedSegmentEditorWidget.setSegmentationNodeSelectorVisible(False)
self.ui.embeddedSegmentEditorWidget.setSourceVolumeNodeSelectorVisible(False)
self.ui.embeddedSegmentEditorWidget.setMRMLSegmentEditorNode(segmentEditorNode)
self.ui.embeddedSegmentEditorWidget.setSegmentationNode(segmentationNode)
self.ui.embeddedSegmentEditorWidget.setSourceVolumeNode(volumeNode)Here's the entire TutorialModuleWidget class, we have just made changes to the setup function and created the onLoadVolumeButtonClick function.
class TutorialModuleWidget(ScriptedLoadableModuleWidget, VTKObservationMixin):
"""Uses ScriptedLoadableModuleWidget base class, available at:
https://github.com/Slicer/Slicer/blob/main/Base/Python/slicer/ScriptedLoadableModule.py
"""
def __init__(self, parent=None):
"""
Called when the user opens the module the first time and the widget is initialized.
"""
ScriptedLoadableModuleWidget.__init__(self, parent)
VTKObservationMixin.__init__(self) # needed for parameter node observation
self.logic = None
self._parameterNode = None
self._updatingGUIFromParameterNode = False
def setup(self):
"""
Called when the user opens the module the first time and the widget is initialized.
"""
ScriptedLoadableModuleWidget.setup(self)
# Load widget from .ui file (created by Qt Designer).
# Additional widgets can be instantiated manually and added to self.layout.
uiWidget = slicer.util.loadUI(self.resourcePath('UI/TutorialModule.ui'))
self.layout.addWidget(uiWidget)
self.ui = slicer.util.childWidgetVariables(uiWidget)
# Set scene in MRML widgets. Make sure that in Qt designer the top-level qMRMLWidget's
# "mrmlSceneChanged(vtkMRMLScene*)" signal in is connected to each MRML widget's.
# "setMRMLScene(vtkMRMLScene*)" slot.
uiWidget.setMRMLScene(slicer.mrmlScene)
# Create logic class. Logic implements all computations that should be possible to run
# in batch mode, without a graphical user interface.
self.logic = TutorialModuleLogic()
# Connections
# These connections ensure that we update parameter node when scene is closed
self.addObserver(slicer.mrmlScene, slicer.mrmlScene.StartCloseEvent, self.onSceneStartClose)
self.addObserver(slicer.mrmlScene, slicer.mrmlScene.EndCloseEvent, self.onSceneEndClose)
# Buttons
self.ui.loadVolumeButton.connect('clicked(bool)', self.onLoadVolumeButtonClick)
# Make sure parameter node is initialized (needed for module reload)
self.initializeParameterNode()
def cleanup(self):
"""
Called when the application closes and the module widget is destroyed.
"""
self.removeObservers()
def enter(self):
"""
Called each time the user opens this module.
"""
# Make sure parameter node exists and observed
self.initializeParameterNode()
def exit(self):
"""
Called each time the user opens a different module.
"""
# Do not react to parameter node changes (GUI wlil be updated when the user enters into the module)
self.removeObserver(self._parameterNode, vtk.vtkCommand.ModifiedEvent, self.updateGUIFromParameterNode)
def onSceneStartClose(self, caller, event):
"""
Called just before the scene is closed.
"""
# Parameter node will be reset, do not use it anymore
self.setParameterNode(None)
def onSceneEndClose(self, caller, event):
"""
Called just after the scene is closed.
"""
# If this module is shown while the scene is closed then recreate a new parameter node immediately
if self.parent.isEntered:
self.initializeParameterNode()
def initializeParameterNode(self):
"""
Ensure parameter node exists and observed.
"""
# Parameter node stores all user choices in parameter values, node selections, etc.
# so that when the scene is saved and reloaded, these settings are restored.
self.setParameterNode(self.logic.getParameterNode())
# Select default input nodes if nothing is selected yet to save a few clicks for the user
if not self._parameterNode.GetNodeReference("InputVolume"):
firstVolumeNode = slicer.mrmlScene.GetFirstNodeByClass("vtkMRMLScalarVolumeNode")
if firstVolumeNode:
self._parameterNode.SetNodeReferenceID("InputVolume", firstVolumeNode.GetID())
def setParameterNode(self, inputParameterNode):
"""
Set and observe parameter node.
Observation is needed because when the parameter node is changed then the GUI must be updated immediately.
"""
if inputParameterNode:
self.logic.setDefaultParameters(inputParameterNode)
# Unobserve previously selected parameter node and add an observer to the newly selected.
# Changes of parameter node are observed so that whenever parameters are changed by a script or any other module
# those are reflected immediately in the GUI.
if self._parameterNode is not None and self.hasObserver(self._parameterNode, vtk.vtkCommand.ModifiedEvent, self.updateGUIFromParameterNode):
self.removeObserver(self._parameterNode, vtk.vtkCommand.ModifiedEvent, self.updateGUIFromParameterNode)
self._parameterNode = inputParameterNode
if self._parameterNode is not None:
self.addObserver(self._parameterNode, vtk.vtkCommand.ModifiedEvent, self.updateGUIFromParameterNode)
# Initial GUI update
self.updateGUIFromParameterNode()
def updateGUIFromParameterNode(self, caller=None, event=None):
"""
This method is called whenever parameter node is changed.
The module GUI is updated to show the current state of the parameter node.
"""
if self._parameterNode is None or self._updatingGUIFromParameterNode:
return
# Make sure GUI changes do not call updateParameterNodeFromGUI (it could cause infinite loop)
self._updatingGUIFromParameterNode = True
# All the GUI updates are done
self._updatingGUIFromParameterNode = False
def updateParameterNodeFromGUI(self, caller=None, event=None):
"""
This method is called when the user makes any change in the GUI.
The changes are saved into the parameter node (so that they are restored when the scene is saved and loaded).
"""
if self._parameterNode is None or self._updatingGUIFromParameterNode:
return
wasModified = self._parameterNode.StartModify() # Modify all properties in a single batch
self._parameterNode.SetNodeReferenceID("InputVolume", self.ui.inputSelector.currentNodeID)
self._parameterNode.SetNodeReferenceID("OutputVolume", self.ui.outputSelector.currentNodeID)
self._parameterNode.SetParameter("Threshold", str(self.ui.imageThresholdSliderWidget.value))
self._parameterNode.SetParameter("Invert", "true" if self.ui.invertOutputCheckBox.checked else "false")
self._parameterNode.SetNodeReferenceID("OutputVolumeInverse", self.ui.invertedOutputSelector.currentNodeID)
self._parameterNode.EndModify(wasModified)
def onLoadVolumeButtonClick(self):
volumePath = self.ui.volumePathTextBox.text
volumeNode = self.logic.loadVolume(volumePath)
if volumeNode is None:
return
segmentationNode = self.logic.getSegmentationNode()
segmentEditorNode = self.logic.getSegmentEditorNode()
self.ui.embeddedSegmentEditorWidget.setMRMLScene(slicer.mrmlScene)
self.ui.embeddedSegmentEditorWidget.setSegmentationNodeSelectorVisible(False)
self.ui.embeddedSegmentEditorWidget.setSourceVolumeNodeSelectorVisible(False)
self.ui.embeddedSegmentEditorWidget.setMRMLSegmentEditorNode(segmentEditorNode)
self.ui.embeddedSegmentEditorWidget.setSegmentationNode(segmentationNode)
self.ui.embeddedSegmentEditorWidget.setSourceVolumeNode(volumeNode)Conclusion
Congratulations! You have just created your very own extension for 3D Slicer that you can use to segment medical images.
But, we haven't really done any computations on the segments. That is exactly what we do in Part 2 which is a short read that serves to explain how segment manipulation works.
Once, you are familiar with the segment-updating process, you can check out Part 3 in which we use a model called TotalSegmentator to segment an entire volume with a single click.
See you next time :)
