Image Fusion Profile Todd Kantchev Siemens Molecular Imaging

Image Fusion Profile Todd Kantchev, Siemens Molecular Imaging Jerold Wallis, Mallinckrodt Institute of Radiology Kevin O’Donnell, Toshiba America Medical Systems IHE Workshop – June 2006 1

Image Registration and Fusion • Translate • Rotate Rigid Registration • Scale • Warp- optional (Deformable Registration) • Fuse IHE Workshop – June 2006 2

Image Fusion- General Use Case CT MR Tech acquires two or more series of images, possibly separated in time. Tech reconstructs images into volumes, performs registration, and stores results data. Radiologist reads study or Oncologist completes RT planning on RT Planning System. Referring Physician or Researcher reviews results. IHE Workshop – June 2006 3

Fusion- The Integration Problem Results not readily transferable between systems Ø Proprietary or inconsistent implementations Ø Lack of guidance Results presented on different systems may not appear the same. Systems must correctly handle issues such as Ø Identifying and retrieving the right datasets Ø Matching data between single-slice and multi-slice datasets Ø Performing spatial translations Ø Rendering the fused display IHE Workshop – June 2006 4

Fusion- The Integration Solution A collection of DICOM objects exists to address the problem. Their use is constrained and clarified to avoid misinterpretation and promote compatibility. IHE Workshop – June 2006 5

Essential Clinical Features: MDs will be Ecstatic MPR Fusion Side-by-side Ø Registered IHE Workshop – June 2006 6

Value Proposition Interoperable DICOM encoding for registered/fused data Consistent and reliable acquisition, storage and presentation of fused data across vendors Allows separation of workflow steps: Acquisition on “A”; Fusion on “B”; Display on “C” Wider distribution of results Separable registration for re-doing if necessary Can easily request basic interoperable fusion features. E. g. blended display, MPR, side by side display, etc. Supports PET-CT (primary initial interest) viewing on general radiology PACS displays Modality independent Supports general needs in radiation oncology pre-planning IHE Workshop – June 2006 7

Image Fusion Technical Basis Spatial Registration Object Encodes transformation to align images in a common registered space Identifies the registered space References the images to be aligned Not required for already-registered PET-CT data from a hybrid scanner IHE Workshop – June 2006 8

Image Fusion Technical Basis Blending Presentation Object References one set of underlying images (e. g. CT) References one set of superimposed images (e. g. PET) References (optionally) a Registration Object to align both image sets before blending. Encodes the window level for each set Encodes the blending factor (0 to 100%) of the overlay transparency IHE Workshop – June 2006 9

Blending Presentation Overlay Transparency 80% 50% 100% 50% IHE Workshop – June 2006 20% 50% 10

Use Case- The IT Solution Modality CT Modality MR Image Archive Evidence Creator Image Display Store CT Store MR Retrieve CT, MR Register MR to CT Fuse CT and MR Store SRO Store BSPS Retrieve CT, MR, SRO, BSPS Apply T to MR Fuse CT and MR IHE Workshop – June 2006 11

Image Fusion Actors and Transactions Acquisition Modality Evidence Creator [RAD 8] Modality Images Stored ↓ [RAD 57] Blending Presentation States Stored ↓ [RAD 56] Spatial Registrations Stored ↓ [RAD 10] Storage Commitment ↓ ↓ [RAD 18] Creator Images Stored ↓ [RAD 57] Blending Presentation States Stored ↓ [RAD 56] Spatial Registrations Stored ↓ [RAD 10] Storage Commitment Image Manager Image Archive Image Display IHE Workshop – June 2006 ↓ [RAD 14] Query Images ↓ [RAD 16] Retrieve Images ↓ [RAD 15] Query Presentation States ↓ [RAD 17] Retrieve Presentation States ↓ [RAD 58] Retrieve Spatial Registrations 12

Fusion- The Creating Process Load CT and PET images CT PET Align images (automatically or manually) and fuse with desired blending factor Store the alignment, including Ø Registration object Ø Blending State (defining the blending Blending State Registration Object IHE Workshop – June 2006 factor, window levels and referencing all other objects) Storage can be on PACS or disk. Images do not need to be stored if already stored. 13

Fusion- The Displaying Process Blending State Registration Object CT PET IHE Workshop – June 2006 Blending state can be searched and retrieved from PACS Ø Can be used to retrieve the image data and the alignment automatically. User selects a Blending state. Ø Automatically load the referenced CT and PET images and the Registration object. Alignment can be applied, PET resampled to CT and displayed fused automatically. Display is required to provide Ø Scrolling through volume Ø Adjustment of fusion blending and window levels Ø MPR Ø Side-by-side display 14

IHE Workshop – June 2006 15