FW4SPL, a framework for applications based on medical imaging.



Flavien Bridault

RMLL 2015 - Beauvais, Wednesday, 8th 2015

IRCAD (Strasbourg, France)

  • IRCAD is a research institute against digestive system cancers.
  • It is known worldwide especially because it is a training center, where 4000 surgeons/year attend courses on mini-invasive and laparoscopic surgery.
  • There are also two research and development teams. One about Robotics, and an another one about Computer Vision. FW4SPL is the framework developed by this team.

IHU Strasbourg

  • IHU is located next to IRCAD.
  • It is a research center about mini-invasive surgery guided by image.
  • IHU also contributes to FW4SPL.

Presentation purpose

FW4SPL meaning

  • Also a disclaimer

IRCAD context

  • At IRCAD, we make prototypes/PoC of software based on medical imaging
  • Our data is the patient
  • Historically we have worked on different steps of the image process
Viewer/Segmentation
Planning

Thanks to the 3D segmentation, we can help the surgeon to analyse a patient pathology. We worked on tools that help to plan the surgery. This activity has led to create a start-up called Visible Patient.

Simulation

We also worked on medical simulators in laparoscopy. Laparoscopy is a way to perform a surgery with only small incisions in the abdomen. Surgeons interact with mechanical instruments and watch the organs through camera. Simulators help them to train without a . This activity has led to create a start-up called Digital Trainers.

Augmented reality

Today we are focused on augmented reality. In laparoscopy, the surgeon has a very tight field of view. We superimpose informations on the video to help them during the surgery, for instance here, the location of the tumor on a liver.

IRCAD R&D team

  • Around 20 people

Why a framework ?

  • software/prototype - Windows, OSX, Linux, Android, IOs
  • reuse source code
  • we don't want people to work on different library versions, with different options,...
  • we need to integrate the different applications/fragmented code

FW4SPL characteristics

  • To match all these goals...
  • XML, not common to build applications

Important dates

  • Sofa: biomechanical engine
  • Altran: proof that FW4SPL could be used outside IRCAD
  • Visible Patient was created in 2013 to commercialize the planning applications,...

Outline

What is the Object/Service approach ?

Classic object-oriented approach

Limits of this approach

Solution

  • Too many functions, if team continue to add functions
  • Many dependencies required (itk,vtk,qt,dcmtk,...) even if you just need to crop an image
  • Everyone work on the same file

Service

  • to this end, we use to concept of service
  • we can observe that for each process, we always have to repeat the same execution pattern, life cycle, configure, then initialize, update and stop
  • update can be repeated
  • transitions, on ne peut pas passer de configure à stop
  • We need to store data into it

Service approach example

  • Common interface 4 methods
  • sub-classes for each type
  • one service for each functionality

DcmtkReaderSrv

ItkCropOperatorSrv

VtkQtVisuSrv

  • configure() : verify if the screen support this size
  • start() : initialize Qt frame and vtk pipeline and show the frame (image is not shown if image buffer is null )
  • update() : check if the buffer has be changed, if true, refresh the vtk pipeline to show negato
  • stop() : destroy vtk pipeline and uninitialize Qt frame.

Application description in XML

XML configuration file

<object type="::fwData::Image">

    <service uid="myFrame" impl="DefaultFrame" type="IFrame" >
        <gui>
            <frame>
                <minSize width="800" height="600" />
            </frame>
        </gui>
        <registry>
            <view uid="myVisu" />
        </registry>
    </service>

    <service uid="myVisu" impl="vtkSimpleNegatoRenderer" type="IRender" />

    <service uid="myReader" impl="VtkImageReader" type="IReader" >
        <filename path="./TutoData/patient1.vtk"/>
    </service>

    <start uid="myFrame" />
    <start uid="myVisu"/>
    <start uid="myReader"/>

    <update uid="myReader"/>    <!-- Read the image on filesystem -->
    <update uid="myVisu"/>      <!-- Refresh the visu -->

</object>

Problem

And if we read a new image later ?

Outline

Communication













void DcmtkReaderSrv::update()
{
    // Load an image using dcmtk
    Dcmtk::Image img;
    ...

    Image* img = this->getObject<Image>();

    // Convert dcmtk image data in our format
    img->createImage(img, size);

    // Emit the signal "modified"
    Signal* sig = img->signal("modified");
    sig->asyncEmit();
}
<object uid="imageUID" type="::fwData::Image">

    ...

    <service uid="myVisu" impl="vtkSimpleNegatoRenderer" type="IRender" />

    <service uid="myReader" impl="VtkImageReader" type="IReader" >
        <filename path="./TutoData/patient1.vtk"/>
    </service>

    <connect>
        <signal>imageUID/modified</signal>
        <slot>myVisu/update</slot>
    </connect>

    <start uid="myFrame" />
    <start uid="myVisu"/>
    <start uid="myReader"/>

</object>

Outline

Component in FW4SPL

Component based approach

Benefits

Examples

  • Code split
  • Reuse code in another application, without recompiling your program, even no link of your application against a library
  • Easier support EXAMPLE correction of bug

Content of a Bundle

  • When a Bundle is compiled
  • Xml description file ( plugin.xml ) to describe the content of the dynamic library

Extract of plugin.xml (ioITK)

<plugin id="ioITK" class="ioITK::Plugin">
    <library name="ioITK" />

    <requirement id="io" />
    <requirement id="gui" />

    <extension implements="::fwServices::registry::ServiceFactory">
        <type>::io::IReader</type>
        <service>::ioITK::InrImageReaderService</service>
        <object>::fwData::Image</object>
        <desc>Inrimage Reader (ITK/Ircad)</desc>
    </extension>

    <extension implements="::fwServices::registry::ServiceFactory">
        <type>::io::IWriter</type>
        <service>::ioITK::InrImageWriterService</service>
        <object>::fwData::Image</object>
        <desc>Inrimage Writer (ITK/Ircad)</desc>
    </extension>

    <extension implements="::fwServices::registry::ServiceFactory">
        <type>::io::IWriter</type>
        <service>::ioITK::JpgImageWriterService</service>
        <object>::fwData::Image</object>
        <desc>Jpeg Writer (ITK)</desc>
    </extension>
    ...
</plugin>
  • This shows how to register services in the factory
  • This helps to load bundles dynamically
  • Don't talk about extension points

Bundles in application

profile.xml

<profile name="TestApp" version="0.1.0">

        <activate id="dataReg" version="0-1" />

        <activate id="gui" version="0-1" />
        <activate id="guiQt" version="0-1" />

        <activate id="io" version="0-1" />
        <activate id="ioVTK" version="0-1" />

        <activate id="media" version="0-1" />

        <activate id="visu" version="0-1" />
        <activate id="visuVTK" version="0-1" />
        <activate id="visuVTKQt" version="0-1" />

        <activate id="TestApp" />
        <activate id="appXml" version="0-1" >
            <param id="config" value="TestAppBase" />
            <param id="parameters" value="TestAppBase" />
        </activate>

        <start id="visuVTK" />
        <start id="visuVTKQt" />
        <start id="guiQt" />
        <start id="appXml" />

</profile>

Example : I/O Bundles

  • Switch GUI !!!

Outline

Design of a new application

Discussion

Services and components

Cons

Pros

Outline

Online documentation

Downloading FW4SPL

  • googlecode may still be in the search engine results

Which version to use ?

Current stable version : 0.10.1

Current development version : 0.10.2

hg qclone https://bitbucket.org/fw4splorg/fw4spl-patches

Repositories

Type

Sources

Dependencies

Main

fw4spl

fw4spl-deps

Extended

fw4spl-ext

fw4spl-ext-deps

AR

fw4spl-ar

fw4spl-ar-deps

Third-part libraries

  • dependencies = external libraries (examples)
  • before building fw4spl, you must build dependencies
  • extended : work in progress

Main repository (fw4spl)

Extended repository (fw4spl-ext)

Augmented reality repository (fw4spl-ar)

Debian Integration

Stay tuned !!!

Coming in September 2015

Thank you !

fw4spl at gmail.com

fbridault at ircad.fr

johan.moreau at ircad.fr





Presentation made with Hovercraft

SpaceForward
Left, Down, Page DownNext slide
Right, Up, Page UpPrevious slide
POpen presenter console
HToggle this help