*Neurological Imaging *Stereotactic Neurosurgery *Surgical Navigation

header image

Vister3D: Common planning platform for frame-based and frameless stereotaxy

Vister3D is a new prototype of neurosurgical planning platforms for frame-based and frameless stereotactic interventions. Both interventions share the same diagnostic data, - and after archiving, the planning results can be involved in surgical preparation for a different technique. Vister3D offers an ease-of-use planning platform with a generalized solution for conventional arc-based stereotactic head frames:

    Riechert/Mundinger (Inomed Medizintechnik GmbH, Germany);
    MHT (MHT Stereotactic Systems LTD, Germany);
    Leksell (Elekta AB, Sweden),
and frameless navigation interface with Polaris type cameras manufactured by:
Northern Digital Inc., Canada.

Moreover, the 3D transparency, provided by the software, facilitates accuracy comparisons of frame-based and frameless methods even during the same surgical intervention.

Preplanning is supplemented by DWI (diffusion weighted image) read in and DTI (diffusion tensor imaging) data visualization with GPU-hosted registration to patient MRI. Tractographic analysis can be performed with on-site merging of preregistered archive of fusioned MR and DTI data.

The system has been validated in different brain surgeries: DBS electrode implantology for Parkinson's disease, tumor biopsy or thermolaesio planning for head-arc, frame-based reference and craniotomy planning for frameless navigation to enhance ablating poorly accessible tumors. The modular architecture supports new feature extensions with immediate use in both methodologies. The complex tasks of adding DTI images and running fiber tractography is enhanced by optional backup of studies in compressed archive at different stages of planning.

Vister3D: Tabulated interface, with modules built up according to clinical workflow

Patient data module for parsing DICOM data and archive studies from easily maintainable database to improve communication regarding diagnostics. Vister3D planning usually relies on three inputs:
    reference CT (always needed);
    fusion MR (to be registered with reference);
    diffusion tensor imaging (DTI) data (optional for tractography, to be registered with fusion MR);

CT-MR fusion module, targeting user selected subvolume with highly optimized fusion algorithm, aligns MR images to CT and visualizes results in orthographic planes;

MR-DTI fusion module, register DWI gradient-weighted dataset to MR (using 0 b-value DTI map, as reference) with fast, CUDA-based algorithm. ADC (mean apparent diffusion coefficient) or FA (fractional anisotropy) maps can be fusioned/displayed with MR for diagnostic purposes;

Tractography module, as planning supplement with automatic parsing of registered DTI data and ROIs positioned on anatomical images. Vister3D implements tractography both on original "raw" data and registered DTI data, as well. The unregistered approach relies on "one session" diagnostics with accurate scanner transform both for anatomical and DTI images;

Planning module integrates all operations needed for frame-based or frameless stereotaxy. The calculations are strictly 3D-based that permits an exchange of planning outcome between different platforms. Some of frame-based systems are supported with calculations optimized for anatomical symmetry. Rotated mounting of frame is also possible using common marker set (Vister3D separates marker- and stereotactic spaces for generalized stereotaxy). Real-time feedback is added for errors in marker localization and during trajectory planning for coordinates in image volumes and anatomical spaces along with polar settings;

Tracking/Navigation module in frameless mode: planning results are usable in optical navigation after surgical space is registered by fiducials. Different tracking modes can be initiated for imaging data using real-time 3D resampling with optional display of fiber models.

Patient data module
Data parsing module with display of basic imaging parameters. The root folder(F:0_MR) contains the patient list with DICOM data (in several formats: .dcm, .nii, .mif etc) and compressed archive files of previous surgical plans (.VTS). The parsing automatically detects patient's DTI data for tractography analysis. Large database can be parsed quickly in a multithreaded solution with options of importing patients' studies or imaging data during the parsing period. The displayed scenario is typical during study selection and data read in for patients undergoing frame-based DBS electrode implantation or tumor biopsy. Data parsing is extended to support study archives with registered DTI images. Back to modules

CT-MR fusion module
CT-MR fusion is based on MI (mutual information) optimization with optional smoothing of CT images. It works well with noisy CT reference and T1-T2 MR modalities, and the result is adapted for orthographic planning views. Manual alignment is possible in rotation with automatic registration. The fusion is optimized for freely selectable subvolumes but with suitable validation for the whole volume (local to global conversion of registration transform). The enclosed video illustrates the power of MI optimization if high level of metal artifact is present in CT reference images. The noise in CT was initially decreased by one-step 3D Gaussian smoothing(fusion_video). Back to modules

MR-DTI fusion module
After automatic data parsing of diffusion data found in patient folder, the program generates unregistered DTI mask file and calculates/adds reference DTI volume from tensor images.
Fusion can be executed in next optional ways:
• using patient (or scanner) transforms extracted from DICOM headers (MR and DTI) and neglecting patient’s move during scanning;
• aligning 0 b-value map as reference to anatomical MR and executing motion compensation with CUDA-based algorithm for the full DTI data set;
• adding CUDA-based deformable registration between MR and reference DTI map to find forward and backward displacement fields;
• build up diagnostic imaging with fusioned ADC or FA maps as reference DTI.

Comparison of axial and sagital views ofMR-fusioned DTI volumes(left side: 0 b-value map with nonlinear registration to MR, right side: DTI map for selected gradient registered to MR; in both cases MR mask is activated to improve registration accuracy). CUDA implemented nonlinear image registration of DTI reference consists of two steps: 1/affine step optimizing 12 elements of source-target transform and 2/nonlinear step wich uses geometric moments with diffeomorphic algorithm. Affine step is terminated by preset termination gain. Nonlinear optimization is limited by elastic termination gain, termination cost, max iterations, max time per resolution and max total time. The DTI volumes with nonzero b-value and gradients are registered to deformed DTI reference with affine transforms. This motion compensated data set is archived for tractography. Displacement fields are calculated and stored in stl files for MR to reference DTI and opposite registrations. With these antagonistic sets of displacement fields the seed and target ROI masks for tractography can be deformed into DTI volume and the resulted fiber models can be deformed back from DTI volume leaving the original diffusion data as intact as possible. The calculations are depended on location/orientation of images in scanner space and are adapted to user selected VOIs. This approach with identical VOIs on source and target volumes improves the optimization process and can be the basis for calculating fiber distributions within restricted regions. (DTIfusion_video). Back to modules

Comparison of axial and sagital views ofMR-fusioned ADC and FA maps(in both cases MR mask is activated to improve accuracy of deformable registration). ADC and FA maps can be used as reference DTI to register the whole gradient-based dataset. The preferred algorithm is the same as given previously for DTIs referenced by the 0 b-value map. (FAfusion_video). Back to modules

Tractography module
Preferences for using DTI-MR data fusion in fiber modelling during stereotactic planning:
• Unregistered (scanner transform based)approach is very simple, only seed/target ROI mapping needed between the CT reference volume and original, raw MR volume. From there the scanner and DWI spaces can be reached where the uncompensated diffusion data with mask file is used for fiber modelling;
• 0 b-value map as DTI reference,after automatic rotation compensation with scanner transforms, it is aligned to anatomical MR by mouse. Motion compensation is executed with CUDA-based affine algorithm for the full DTI data set with registration to DTI reference. In this case the optimized DTI-MR projection transform is used to reach the diffusion space where the fiber models are calculated with compensated diffusion data and mask file;
• CUDA-based deformable registrationis activated between MR and reference DTI map to find forward and backward displacement fields, followed by motion compensation as before. In this case the MR to DTI nonlinear displacement field is used to optimise seed/target ROI image data during tractography initialization. If tractography finished the resulted fiber models are mapped back from DTI to MR/CT volume using DTI to MR displacement field;
• Import/export functions for FSL;Vister3D implements streamline tractography but comparison with results from FSL (FMRIB's Diffusion Toolbox) tractography is possible. Vister3D exports bvals and bvecs files formatted for FSL and motion-compensated or uncompensated diffusion volumes and mask files in compressed .nii format. Seed and include ROi-s are archived as mask files for FSL. Preferably, this configuration runs under Windows/Linux dual boot on the same hardware. Vister3D is added under Windows10 and FSL is added under CENTOS7 Linux in dual boot environment. GPU versions of FSL commands 'bedpostx' and 'probtractx' can be started on Linux on high-end Notebooks supporting NVIDIA Optimus video card. This hybrid card (with Intel and Nvidia VGA control) permits to run GPU commands without switching off Linux graphic interface. The FSL-resulted probabilistic fiber distribution (fdt_paths.nii.gz) can be imported/displayed in Vister3D as overlay registered to MR or reference CT volumes (FSL import).

Tractography analysis in unregistered mode without motion compensation: upper scheme. Tractography analysis in affine/deformable registered mode with motion compensation: lower scheme.Back to modules

Selection of ROIs (spherical or masked types):
Sphere ROIs:3D-localized seed (yellow), include (green) and exclude types can be used. Seed ROI represents thalamic origin and include sphere is added for filtering nerve bundles. Left views display the distribution based on seed ROI only. These views can be used for initial 3D localization of include ROI. Right views show fibers after repeated calculation with 3D localized include ROI. Back to modules
Masked ROIs:Contur of seed mask (yellow) and include (green) types are set by mouse in orthographic views. The masked ROI volume is built up by movable/resizeable circle selectable for views (axial, coronar, sagital) with an optional switch to erase mode. This way an accurate 3D ROI mask can be created or edited. The mask depth can be any size from single voxel z step. Fibers tracked from seed ROI mask can be filtered with freely selectable include and exclude subregions. The video displays streamline intercepts along with results of FSL probabilistic tractography generated by FDT ( FMRIB's Diffusion Toolbox) (intercept_video). Back to modules

Comparing unregistered and registered tractography:
Nerve modelscan be calculated in unregistered mode using scanner transforms (blue) or after deformable registration and motion compensation (orange). Seed ROI represents thalamic origin. Results support the idea to use unregistered mode for fast initial 3D survey of fiber distribution. Back to modules

Import/export functions for FSL:

Vister3D exports b values and b vectors in parameter files, diffusion volumes (with or without motion compensation), data mask and ROI masks for FSL to perform probabilistic tractography. Other parameters like registration transforms and CT-MR imaging data are kept in Vister3D for surgical planning. FSL performs computations only in space of diffusion volume. The results of FSL tractography can be imported and used in registered mode for surgical planning. Vister3D is able to visualize and compare self-calculated fiber models to FSL results. See enclosed videos (Vister3D_FSL_video; FSLEyes_video).Back to modules

Tractography initialization for frameless navigation:
Frameless navigationcan be supplemented with display of nerve bundles (gradient based coloring): the diagnostic imaging involves fusioned CT-MR data. Real time resampling of imaging is controlled by orientation of stylus with optional offset for cross cursor at stylus tip. The cursor position can be used for 3D measurement between tip of device and target area. See sample of stylus controlled navigation (seed ROI located near skull bone) (navigation_video).Back to modules

Planning module

Planning strategies in 3D
Frame-based Interventions: 3D alignment planner to find the best representation of anatomy in imaging;
Rotated stereotaxy by means of separating marker and stereotactic spaces that helps to find generalized solution for stereotactic computation. Continuous error feedback is added during marker localization;
Biopsy planner represents one side planner for biopsy;
Two sides trajectory planner according to anatomical symmetry for DBS (for some frames constrained optimization is selectable to keep left/right mounting on the same arc);
Wide selection of models of DBS electrodes (for stimulation and multiply recordings), biopsy needle and TC (thermolaesio) electrodes;
Export protocols of head-arc settings for operating room and study file storing full dataset of planning.
Frameless Interventions: Orthographic trajectory planner with fusioned and different resampling views with optional data exchange with results of frame-based planning;
Frameless mode:Surgical space registration with fiducials or anatomical markers (comparative accuracy test with frame-based method is possible);
Stylus supported calibration module
Navigation of surgical device is possible if its simplified geometry is localized within the space of an attached motion sensor. This can be done with special calibration algorithm, usually based on statistical sampling of attached sensor's data (like pivoting) or identification of feature points by an external device of guaranteed accuracy (using stylus or special slots). The calculated parameters of geometry in sensor's space usually are: tip offset, axis orientation, handle direction). With this calibration approach, Vister3D is able to simulate several navigated devices or add new surgical tools;
Communication softwarewith Polaris family of motion tracking cameras. Communication with motion tracking cameras from Northern Digital Inc. is controlled by separate software which sets USB port connection. Configuration options can be added for sensor selection and parameters of data transfer.Back to modules

3D alignment planner:

Resampling reference imagesaccording to patient specific AC-PC plane. Center of rotation can be relocated and tilt can be defined about all axes. Sequential rotation is implemented according to Euler floating axes scheme. Upper figures display the original CT and reconstructed surface with data coming out directly from scanner. Lower row shows the resampled CT volume according to a user optimized tilt and rotation center relocation. The resampled surface from multiply transformations is displayed in right bottom (with courtesy of István Valálik MD, PhD., Neurosurgery Dept., St. John Hospital, Budapest, Hungary) Back to planning

Rotated stereotaxy:

Marker spaceis registered to CT reference through intercepts between marker lines and selected image plane. Stereotactic spacecan be identical to marker space or transformed from this base position. Continuous feedback of reconstruction accuracy of marker space ("Marker Check") is possible according to geometrical constraint introduced. The reconstruction of stereotactic space is based on geometry and location of marker plates that offers solution for global accuracy test as a function of marker localization. The planning is divided into several validating steps which are needed to finish step-by-step, in strict order. Clearing any step invalidates all succeeding steps. This figure displays the typical planning view of RM/MHT frames where the marker's space is identical to stereotactic space (no rotation added).Back to planning

Stereotactic frame (MHT) in surgical view rotated with 90° relative to patient's skull (marker's space). Left side shows the planning view with rotated axes of stereotactic space. Right side displays the final surgical view setting up the real orientation of axes of frame during surgery (patient is lying on his left side for convenient localization of right posterior entry point in DBS). Diagnostic background: fusioned CT-MR.Back to planning

Stereotactic frame (Leksell)relocated relative to marker's space (generalized solution for Leksell frame). Vister3D implements stereotactic calculation by distinguishing marker space from stereotactic space that helps integrating majority of head frames currently available. Diagnostic background: CT. Back to planning

Biopsy planner:
Biopsy planning in cerebellum with rotated mounting of MHT frame (270° relative to marker's space). After trajectory planning with parameter initialization in the main panel (left side) inset displays the biopsy sampling parameters (right bottom). The sampling locations are selected by simple mouse tracking along the predefined path. Diagnostic background: CT-MR fusioned data with highlighted MR. The figure shows the axes of registered frame as projected into diagnostic view (with courtesy of István Valálik MD, PhD., Neurosurgery Dept., St. John Hospital, Budapest, Hungary). Back to planning

Backup of results of biopsy planning with screenshots taken at each sampling position. Archive file stores frame settings with image screenshots (see example Surgery case in basal ganglia and brainstem...). Back to planning

Selection of models:
Models of Medtronic DBS electrodes (3387-89-91) with recording extensions, biopsy needle and TC (for thermolaesio with pull-out extension) electrode models are available in graphical interface and can be selected for planning. Back to planning

Orthographic trajectory planner:
Planning for frameless navigation is based on orthographic views with 3D cursor reflecting the actual slice position in each view. Trajectory localization can be started in all views by selecting entry or target points. The planning results are 3D transparent between views used for frame based method and orthographic views for frameless mode. In frameless mode the trajectories can be displayed also in 3 plane mode and surface reconstructed mode. In tracking mode the image at target position can be resampled according to vector identical to path direction (right bottom). Each viewing plane can display reference image and registered fusion image in semitransparent overlap mode. Back to planning

Tracking/Navigation module

Frameless mode; surgical space registration:
Registration panel in frameless navigation module. Upper view illustrates an optional configuration suitable for comparison with frame-based approach. Lower view illustrates the use of small metal fiducials attached to soft tissue. Surgical space can be registered with 4-6 fiducials and the software provides for full transparency between all integrated 3D spaces (diagnostic volumes, local/global surgical spaces as defined by NDI tracker, patient's anatomical space given by anatomical landmarks, and even stereotactic space of head frame if needed for comparison). The base view of navigation integrates solutions for marker-based registration, device calibration, feedbacks for calculated parameters, errors, sensor visibility tests, etc. Back to planning Back to modules

Targeting graphics helps navigating device toward target position along a preplanned trajectory. See video for illustration of real time targeting with graphics displaying distance and orientation mismatch between target and navigated device. The accuracy of targeting can be improved by using mechanical guiding support. The presented video represents phantom experiment planned with real data (targeting_brain_video). Back to modules

Tumor localizationfor craniotomy in frameless navigation mode. Different resampling views can be synchronized with stylus/tool location and orientation in real time. Reference or fusion images can be selected for viewing. Tumor identification is illustrated by videos with real time resampling according to change of stylus location/orientation (with courtesy from Neurosurgery Dept., St.John Hospital, Budapest, Hungary) (navigation_tumor_videoA; navigation_tumor_videoB). Back to modules

Vister3D: Surgical integration of tractography with preplanning and archiving supports

Vister3D creates very flexible environment for surgical planning: the complex task of DTI registration to anatomic MR can be executed in preplanning step without using CT input. The results of this step are stored incompressed archive fileand can be inserted later into CT referenced stereotactic planning. Compressed archive studies can be generated and merged - with data integrity check - at different levels of calculation (after CT-MR fusion, stereotactic frame registration, trajectory planning, DTI fusion etc). Tractography can be performed not only during preplanning step but during on-site stereotactic planning, as well. The fiber models are exported to vtk file and can be imported later at any phase of planning. This improves the reliability of computations and creates a highly verifiable environment.

Example; preplanning study merged with on-site DBS planning
DBS planning study with tractography data fusion,video displays fiber distributions near electrodes implanted on both hemisphere (intercepts with seed plane is also shown). Different colors of nerve bundles represent changes in target volume ("include" ROI in cortex) (with courtesy of István Valálik MD., PhD. from Neurosurgery Dept., St.John Hospital, Budapest, Hungary) .(tract_DBS_video). Back to modules

Archiving fiber models for independent study

Upper view shows theinterpolated mask of fibermodel suitable for isosurface reconstruction (stored in nifti format). Lower view displays themesh archivestored in .stl file. Back to modules


SurgiFront Ltd. has been owned by Ferenc Pongrácz, Dr. and his son Ádám Pongrácz. After working several years at Yale University (New Haven, USA) on the field of computational neurobiology Ferenc Pongrácz joined in pioneering work for implementing new motion tracking technology into medical applications. He worked for company Artma Biomedical Inc. (located in Salt Lake City, USA) which later moved into Vienna, Austria. He has many years of experience in developing software for various clinical fields listed from human motion analysis, integration of navigated endoscopic view into diagnostics, surgical planning and drill navigation in dentistry and recently stereoactic planning and navigation in neurosurgery. The application has been introduced in routine surgeries with close supervision made by qualified surgeons at Neurosurgical Department of St. John Hospital, Budapest, Hungary. Vister3D has been also integrated into treatment of Parkinson's disease at private medical care (DBS surgery and other minimally invasive procedures like radiofrequency ablation etc...https://parkinson.hu/ and at Neuromed; https://neuromed.hu/). The software development has been supported by experts actively participating in open-source fields for medical applications. More info on related works: profile/Ferenc_Pongracz.


SurgiFRONT Ltd.

Budapest, Zerind Vezér u. 29/B
1029 Hungary
phones: (+36) 305621806, (+36) 12758615
Contact email: info@surgifront.com

Clinical Sites

  • Neurosurgical Department of St. John Hospital, Budapest, Hungary (http://www.janoskorhaz.hu/idegsebeszet.html)

  • Neuromed Private Care, Budapest, Hungary (https://neuromed.hu/)

  • Selected Publications

  • M. Truppe, F. Pongracz, O. Ploder, A. Wagner, and R. Ewers, Interventional Video Tomography, in Proceedings of Lasers in Surgery, vol. 2395. San Jose, CA: SPIE, 1995, pp. 150-152.

  • W. Freysinger, M. Truppe, A. R. Gunkel, W. L. Thumfart, F. Pongracz, and J. Maierbaeuerl, Interactive telepresence and augmented reality in ENT surgery : Interventional Video Tomography, Lecture Notes in Computer Science • April 2006, (presented at CVRmed - MRCAS 97, Grenoble, 1997), 1205: 817-820.

  • Pongrácz F., Bárdosi Z. Dentition planning for image-guided implantology. In: Proceedings of CARS (Computer Assisted Radiology and Surgery), Eds: Lemke, HU et al. (Chicago, USA, International Congress Series 1268, 2004 ), pp: :1168-1173

  • Pongrácz F, Renner G. Localized volume matching for the detection of relative displacements in CT or MR images. In: Proceedings of CARS (Computer Assisted Radiology and Surgery), Eds: Lemke, HU et al. (Berlin, Germany, International Congress Series 1281, 2005), p.56-61.

  • Pongrácz F., Bárdosi Z. Proposal for prototyping applications for surgical navigation support. In: Int J CARS (Supplement 1 to CARS 2006, Osaka), Springer, Berlin Heidelberg New York, Eds: Lemke, HU et al. pp:185 -187.

  • Pongrácz, F., Bárdosi, Z. Dentition planning with image-based occlusion analysis. Int J CARS, Springer, Berlin Heidelberg New York, Eds: Lemke, HU et al., 2006, vol: 1 (3), pp:149 -156

  • Pongrácz F., Use of optical motion tracking in application development for surgical planning and navigation, Biomechanica Hungarica, 2008, Ed. Csernátony Z., vol: 1, No.1, pp:21-29.

  • Pongrácz, F. Visualization and Modelling in Dental Implantology (2009). in: “Handbook of Research on Dental Computing and Applications: Advanced Techniques for Clinical Dentistry: Chapter XI”, IGI Publications 2009, Editor: A. DASKALAKI, Max Planck Institute, Berlin, pp:159-169 (see also: irma-international: Chapter 8.9 Visualization and Modelling in Dental Implantology (2009).

  • Pongrácz F., Fusion of video and motion data: engineering tasks and clinical applicability. Biomechanica Hungarica, 2010, Ed. Csernátony Z., vol: 3, No.1, pp: 201-207.

  • Pongrácz F., Video Tracking in Clinical Environment: Framework and Computational Elements. In: : Proceedings of the 1st International CIS Workshop at Budapest, Budapesti Műszaki Egyetem, june 18-23, 2011 (9).

  • Pongrácz F.; Keret nélküli (frameless) stereotaxia és navigációs rendszerek, 5.Fejezet. Valálik I. (Szerk.), Stereotaxiás és funkcionális idegsebészet (Akadémiai Kiadó, Budapest, 2012, 627 old): 75-100. (in hungarian).

  • Pongrácz F., Valálik I.; Common Platform for Evaluating Frame- Based and Frameless Stereotactic Surgery, XXth Congress of the European Society for Stereotactic and Functional Neurosurgery, Cascais/Lisbon, Portugal; 09/2012

  • Pongrácz F., Valálik I.; Common IGS planning, support for frame-based stereotaxy and frameless navigation. 3rd National Conference of Neuro-Oncology, Cluj-Napoca (Kolozsvár), Rumania; 04/2013

  • Pongrácz F.; Visualization and Modelling in Dental Implantology. 33. Hellenic Dental Congress, Athens, 2013, Greece; 11/2013

  • Valálik I., Pongrácz F., Csókay A.; Technique and planning software for stereotactic brainstem and basal ganglia biopsies. XXIst Congress of the European Society for Stereotactic and Functional Neurosurgery, Maastricht, the Netherlands; 09/2014

  • Pongrácz F., Szloboda P., Valálik I.; Fiber tractography and brain atlas integration in stereotactic planning: improving interactivity with multithreaded and CUDA-based solutions. XXII Congress of the European Society for Stereotactic and Functional Neurosurgery, Madrid, Spain; 09/2016

  • Pongrácz F., Szloboda P., Valálik I.; Traktográfiai analízis és atlasz fúzió sztereotaxiás tervezéshez: interaktív módszerek felhasználási lehetőségei. Magyar Idegsebészeti Társaság és a Magyar Neuroonkológiai Társaság Kongresszusa, At Miskolc - Lillafüred, 2016

  • Pongrácz F., Szloboda P., Valálik I.; Tractography in frame-based and frameless stereotaxy: surgical integration with preplanning and archiving supports. XXIII Congress of ESSFN Edinburgh, 26 sept-29 sept, 2018