ComputerAssisted Surgery Medical Robotics Medical Image Processing LECTURE

Computer-Assisted Surgery Medical Robotics Medical Image Processing LECTURE 1 1. What‘s in a surgery 2. Technical tools in CS 3. CAS systems CAS, Spring 2001 © L. Joskowicz

PAST: Cut, then see CAS, Spring 2001 © L. Joskowicz 3

PRESENT: See, then cut CAS, Spring 2001 © L. Joskowicz 4

FUTURE: Combine, see, minimally cut CAS, Spring 2001 © L. Joskowicz 5

How do surgeries proceed? • Diagnosis – based on physical exams, images, lab tests • Preoperative planning – determine the surgical approach – elaborate intraoperative plan (path, tools, implants) • Surgery – prepare patient and assess condition – acquire intraoperative images, adapt and execute plan • Postoperative follow-up – exams, lab tests, images to be corroborated CAS, Spring 2001 © L. Joskowicz 6

Treatment procedures • Invasive – neurosurgery: tumor removal – hear surgery: clogged arteries, transplants – orthopaedic surgery: spine, hip replacement, knee, fractures – gall bladder removal, prostate, various cancers • Non-invasive – radiation therapy – kidney stone pulverization CAS, Spring 2001 © L. Joskowicz 7

Medical imaging modalities • Preoperative – Film X-rays, Digital X-rays, Ultrasound, Angiography, Doppler, …. – Computed Tomography (CT), Magnetic Resonance (MR), Nuclear Medicine (PET, SPECT, …) • Intraoperative – X-ray fluoroscopy, ultrasound – video images (laparoscopy, arthorscopy) – Open MR CAS, Spring 2001 © L. Joskowicz 8

Medical imaging modalities: X-rays Film or Digital X-ray CAS, Spring 2001 © L. Joskowicz X-ray Fluoroscopy 9

Medical imaging modalities: continuous X-ray angiography CAS, Spring 2001 © L. Joskowicz 10

Medical imaging modalities: Ultrasound CAS, Spring 2001 © L. Joskowicz 11

Medical imaging modalities: CT Series of parallel slices 2 mm apart Single slice CAS, Spring 2001 © L. Joskowicz 12

Medical imaging modalities: MRI Good imaging of soft tissue CAS, Spring 2001 © L. Joskowicz 13

Medical imaging modalities: Nuclear medicine (PET, SPECT, NMR) Functional imaging: colors indicate electrical activity CAS, Spring 2001 © L. Joskowicz 14

Medical imaging modalities: video TV quality image from small camera (laparoscope or endoscope) CAS, Spring 2001 © L. Joskowicz 15

Surgical approaches • Open surgery – area of interest directly exposed by cutting – direct sight and touch of anatomy by surgeon – direct access but causes additional damage • Closed surgery not always feasible – indirect access to anatomical area of interest – no direct visual sight or tactile feel – catheterization, biopsies – intraoperative imaging is often required – require more skills: lengthier, more difficult • Diagnostic surgery CAS, Spring 2001 © L. Joskowicz 16

Minimally invasive surgery • Provides treatment through small incisions • Uses imaging equipment for seeing and instruments for touching • Advantages: less damage, faster recovery • Disadvantages: hand/eye coordination, time • Examples: – brain tumor removal, laparoscopic surgery CAS, Spring 2001 © L. Joskowicz 17

Laparoscopic surgery CAS, Spring 2001 © L. Joskowicz 18

Brain surgery CAS, Spring 2001 © L. Joskowicz 19

Total Hip replacement -- principle CAS, Spring 2001 © L. Joskowicz 20

Total hip replacement procedure CAS, Spring 2001 © L. Joskowicz 21

What is required to perform surgery? • Knowledge intensive task – anatomy, procedures, cases – experience, skills, customization and generalization • Manual and cognitive skills – dexterity, precision, strength, tool manipulation – spatial orientation and navigation • Determination – information integration – judgement, decision, execution CAS, Spring 2001 © L. Joskowicz 22

Medical and surgical trends • Imaging improved dramatically diagnosis – started with X-rays last century – 30% of all cases use images • Move towards minimally invasive procedures – introduced in the mid ‘ 70 s, slow acceptance (laparoscopy) – the method of choice now • More precise and delicate procedures • Development of sophisticated surgical hardware • High degree of craftsmanship and skills CAS, Spring 2001 © L. Joskowicz 23

Socio-economical medical trends • Increase of aging population and associated problems: tumors, osteoporosis, Alzheimers • Larger population volumes • Universal, first rate, highly specialized care • Health care costs reduction (managed care) • Higher patient requirements • Legal and regulatory aspects CAS, Spring 2001 © L. Joskowicz 24

Surgical Needs Augment the surgeon’s capabilities with better quantitative planning, execution, and integration • Support for image-guided surgery • Passive and active devices for accurate spatial positioning, tracking, and execution • Modeling, planning, viewing, diagnosis systems • Systems integration: from diagnosis to post-op • Improve current practice and enable new procedures • Simulation and training systems CAS, Spring 2001 © L. Joskowicz 25

Current clinical status • Imaging – vast databases of medical images – digitized atlases – mostly uncorrelated unimodal qualitative interpretation • Devices – mostly passive and non-invasive (supports) – laparoscopic camera, – some real-time tracking • Planning, modeling, visualization – 3 D reconstruction, some registration CAS, Spring 2001 © L. Joskowicz 26

Part 2: Computers and Robots Technology and algorithms available today CAS, Spring 2001 © L. Joskowicz

How can computers help? (or are already helping…) • Image processing – single image: enhancement, noise reduction, segmentation, quantitative measurements – image stacks: 3 D reconstruction, segmentation – image sets: registration, comparison, data fusion • Planning and simulation – integrate medical images and CAD models – planning and simulation programs • Computer vision and graphics – camera modeling, image registration, rendering CAS, Spring 2001 © L. Joskowicz 28

Image processing CAS, Spring 2001 © L. Joskowicz 29

Planning and simulation CAS, Spring 2001 © L. Joskowicz 30

Virtual man project -- digital model CAS, Spring 2001 © L. Joskowicz 31

How can robots and sensors help? (or are already helping…) • Robotic devices – passive, semi-active, active devices – instrument and anatomy positioning and holding – cutting and machining • Real-time tracking – optical, video, electromagnetic devices – navigation tools CAS, Spring 2001 © L. Joskowicz 32

Robotic devices CAS, Spring 2001 © L. Joskowicz 33

Real-time tracking devices camera instrument Passive markers Instrument has infrared LEDs attached to it CAS, Spring 2001 © L. Joskowicz Active markers 34

Computer-Assisted Surgery (CAS) A computer-integrated system to enhance the dexterity, visual feedback, and information integration of the surgeon Key points: • The goal is NOT to replace the surgeon • A new paradigm for surgical tools • Address a real clinical need • Prove efficacy and cost-effectiveness CAS, Spring 2001 © L. Joskowicz 35

Elements of CAS systems CAS, Spring 2001 © L. Joskowicz 36

Elements of CAS systems • Preoperative planning – image acquisition, modeling, analysis, simulation – plan elaboration, tool and prosthesis selection – Output: preop images, 3 D models, prosthesis type and position, navigation and cutting plan • Intraoperative execution – passive, semi-active, active robot – real time tracking – intraoperative imaging (fluoroscopy, ultrasound) CAS, Spring 2001 © L. Joskowicz 37

State of the Art (1) • Main clinical procedures – neurosurgery: biopsies, tumor removal – orthopaedics: hip and knee replacement, spine, pelvis and femur fractures – maxillofacial and cranofacial – laparoscopy: laparoscope holders – new fields: dentistry, ophtalmology, prostate • Mostly rigid structures: bones!! CAS, Spring 2001 © L. Joskowicz 38

State of the Art (2) • Commercial navigation systems – main uses: neurosurgery and spine surgery • Commercial robotic systems – ROBODOC for total hip replacement – laparoscope arm holders • Research – very active, very interdisciplinary – a few dozen systems tested in-vitro CAS, Spring 2001 © L. Joskowicz 39

State of the Art (3) • Major players – INRIA Sophia Antipolis, Grenoble, Johns Hopkins, Brigham Women’s H. /MIT, Shadyside H. /CMU, Imperial College, many places in Germany and Japan • Interdisciplinary conferences and journals – started in 1994: MRCAS’ 94; Orthopaedic CAS meetings, visualization, etc, – Journals: Computer-Aided Surgery, Medical Image Analysis CAS, Spring 2001 © L. Joskowicz 40

Examples of CAS systems in use • • Image-guided navigation systems ROBODOC: Total hip replacement surgery LARS: Laparoscopic assistant Radiosurgery Brief overview follows; will be covered in detail later CAS, Spring 2001 © L. Joskowicz 41

Image-guide navigation • Purpose – accurate placement of instruments with respect to imaged anatomy for several procedures • Problem addressed – provide 3 D vision of unseen structures replace static 2 D fluoroscopy or larger openings – improve precision of biopsies, screw placements • Scope – non-invasive – creates surface model from preop images – registration of images to anatomy by direct contact CAS, Spring 2001 © L. Joskowicz 42

Image-guided navigation CAS, Spring 2001 © L. Joskowicz 43

Image-guided navigation (2) pedicle screw insertion CAS, Spring 2001 © L. Joskowicz 44

Status • In clinical use • Over 7, 000 neurosurgeries performed with commercial systems • Gaining popularity in pedicle screw insertion • Decreased the misplacement rate from 10 -40% to 5 -18% (clinical study of 700 cases) • More clinical applications under development CAS, Spring 2001 © L. Joskowicz 45

ROBODOC: Total hip replacement • Purpose – precise machining of cementless hip implant canal • Problem addressed – complications in canal preparation and implant fixation – improve positioning accuracy and surface finish • Scope – invasive, numerically controled machining – plan from preop CT, registered via pins – adapted commercial robot – custom bone fixator and bone motion detection CAS, Spring 2001 © L. Joskowicz 46

Artificial hip joint CAS, Spring 2001 © L. Joskowicz 47

Total hip replacement procedure CAS, Spring 2001 © L. Joskowicz 48

ROBODOC: Total Hip Replacement CAS, Spring 2001 © L. Joskowicz 49

ROBODOC system diagram CAS, Spring 2001 © L. Joskowicz 50

ORTHODOC Planning CAS, Spring 2001 © L. Joskowicz 51

ROBODOC robot diagram CAS, Spring 2001 © L. Joskowicz 52

ROBODOC robot CAS, Spring 2001 © L. Joskowicz 53

ROBODOC procedure CAS, Spring 2001 © L. Joskowicz 54

ROBODOC cutting CAS, Spring 2001 © L. Joskowicz 55

ROBODOC History • Developed by IBM Research and Integrated Surgical Systems • First active surgical robot – 1986: feasibility study – 1989: in-vitro testing of dog system – 1990: 26 dog cases – 1992: development of human system – 1994: first human procedure in Frankfurt – 1995 - clinical trials in the US for FDA approval CAS, Spring 2001 © L. Joskowicz 56

ROBODOC current status • • Sold by Integrated Surgical Systems Over 3, 000 cases performed 15 systems installed in Germany, 2 in Austria Excellent short term clinical results (3 year study) – no fractures, few failures (continue manually) • Long-term clinical results to be determined – key issue: does the artificial hip last longer? • Problems: OR time, pin insertion CAS, Spring 2001 © L. Joskowicz 57

Laparoscopic assistant: LARS • Purpose – laparocopic camera holding and precise navigation • Problem addressed – cumbersome, unintuitive, and unsteady camera positioning • Scope – non-invasive intraoperative device – video images interpreted by surgeon • Benefits – direct camera manipulation; stability, precise targeting CAS, Spring 2001 © L. Joskowicz 58

Laparoscopic assistant: LARS CAS, Spring 2001 © L. Joskowicz 59

LARS characteristics • Designed at IBM Research, 1993. Similar commercial devices available (AESOP) • Custom redundant 7 degree-of-freedom robot • Holds laparoscopic camera • Fulcrum motions: no motion at point of entry • Mouse-like controls on surgical scissors • Position memory and replay CAS, Spring 2001 © L. Joskowicz 60

Stereotactic Radiosurgery • Purpose – plan and deliver precise radiation doses • Problem addressed – precise positioning and dosing of radiation to avoid healthy organ damage • Scope – non-invasive intraoperative device – active beam postioning and planning – complex preoperative planning based on MRI images – registers preoperative plan with stereotactic frame CAS, Spring 2001 © L. Joskowicz 61

Stereotactic Radiosurgery CAS, Spring 2001 © L. Joskowicz 62

CYBERKNIFE system CAS, Spring 2001 © L. Joskowicz 63

CYBERKNIFE system CAS, Spring 2001 © L. Joskowicz 64

Stereotactic Radiosurgery: planning CAS, Spring 2001 © L. Joskowicz 65

Stereotactic Radiosurgery • • Developed at Stanford starting in 1992 Complex 3 D radiation plans Currently in clinical use Frameless procedure under development follow head with markers, video, or X-rays • Company Accuray has performed several clinical trials with frameless procedure CAS, Spring 2001 © L. Joskowicz 66

Developing CAS systems • Similarities – understand address real needs of surgeons – consider established procedures, context, use – work on problems that will make qualitative difference – constant feedback from user; test ideas and prototypes • Differences – system performace requirements CAS, Spring 2001 © L. Joskowicz 67

Developing CAS systems • understand address real needs of surgeons • consider established procedures, context, use • constant feedback from user; test ideas and prototypes • system requirements – safety and reliability – fail-safe systems: can always stop and proceed as usual – system integration CAS, Spring 2001 © L. Joskowicz 68

CAS systems design cycle • Prototype development • In-vitro experiments – system refinement • Cadaver studies – system refinement • In-vivo experiments – first animal and human trials • Clinical trials – double blind studies, Hospital and FDA protocols • Agency approval and commercial release CAS, Spring 2001 © L. Joskowicz 69

Summary • Great potential for robots and computers inside and outside the operating room • Great research and commercial interest, especially in the past 3 years • Just the beginning of the road: many things remain to be invented • Great role for applied computer science: – image processing, geometric planning, registration, graphics, vision, real-time systems, robotics, etc. CAS, Spring 2001 © L. Joskowicz 70