The development and application of image analysis procedures are performed under the software package IDL (Interactive Data Language, Research Systems Inc., Boulder, Co, USA). As hardware development platform a Silicon Graphics work station INDY is used with 128 MB main memory, true color display, connected via local area network to other computers and PC's, especially the PC attached to the laser scanning microscope (LSM 410, Zeiss, Oberkochen, BRD).
Beside mere memory problems with 3-D images of sizes about 15 MB pixels even the processing time increases considerably. E.g. even a RISC processor of MIPS 4400 type becomes slow for methods usually applied and well known in 2D image analysis.
Fluorescence images are scanned using a Confocal Laser Scanning Microscope (CLSM) Zeiss LSM 410. Essential setup features for the acquisition of FISH signals in tissue sections are as follows: Lens Zeiss PNF 100x, 1.3, zoom =2, realized by scanning unit). The scanned field of 62.5 x 62.5 m is sampled to 256 x 256 pixels giving a pixel size of 0.25 m in x and y direction. Excitation laser lines are selected according to the fluorochromes used. For propidium iodide (PI) used as DNA counter stain and FITC labeled signals both are excited by the Argon line 488 nm. The emission of both is measured simultaneously in two separate channels using a bandpass 515 - 565 nm for FITC and a longpass LP 590 nm for the PI channel respectively. The axial distance selected between two subsequent confocal images depends on the further evaluation strategy. If a spatial isotropic representation of the 3-D data is intended, the axial distance is identical to the lateral distance of the pixels, i.e. 0.25 m. For a 16 m thick section this results in a sequence of 64 scanned images. If only the sampling theorem should be met roughly a 0.5 m distance is enough, keeping in mind that the confocal axial resolution of a lens with NA = 1.3 is about the wavelength, i.e. 0.6 m [13]. The FITC emission is associated with the green channel and the PI emission with the red channel of a RGB true color image. The image data are stored on disk in TIFF format. Scanning one field of view of a section with 16 m thickness and using 0.25 m distance between individual images results in an uncompressed data set of 12.58 MByte.
Various methods of display of digital volumes are proposed in this quickly growing field of methodical developments [6, 2]. However, there are still large requirements in research to solve the technical as well psychological problems of display of 3-D informations.
To allow an observation familiar to pathologists, a GUI has been designed (Fig.1) displaying the 3-D image as a heap of orthogonal sections with arbitrary choice of the cutting planes in x, y and z. Beside the counter stain volume in red color channel and the signal in green, a second type of signal may be stored in blue color channel. Possible display by interactions with the GUI are (Fig.1, 2):
Other possible interactions for analysis are histogram based local thresholding, cell labelling, FISH signal count, global and local segmentation, cutting of globally segmented touching cell nuclei by interactively adjustable planes (knife), spheres or ellipses, where the latters can be dragged around the object of interest and than used as limit for the volume growing process, as well as active volumes for segmentation, 3-D morphological operations, etc.
For interactive volume analysis the orthogonal display of the volume can be positioned in x, y and z direction and a box, sphere or ellipses around the object of interest resized by dragging can be selected (Fig.4). By another mouse click the selected sub-volume is displayed in a separate window as a gallery of sub-sections (Fig.5). On left side all sub-sections and on right side zoomed sub-sections are shown.
During display automatically the number of FISH signals inside the selected sub-volume is calculated and the mouse cursor jumps to the counting button with respective number (Fig.2b). Clicking at the appropriate button stores the subvolume location, size and count. In the display field of orthogonal sections the box location is outlined for information of the already scanned places. The buttons SAVE, LOAD, CLEAR, REDRAW, DRAW Count and PRINT Count allow storage and display of the count list (Fig.2b). Also old count list files can be read and redisplayed for further inspection (Fig.6).
The complete automatic segmentation of cell nuclei is still a distant dream. We have developed a seeded volume growing technique based on several size and shape constraints, to segment the cell nuclei and to automatically count the FISH signal per nuclei. After slight volume opening for noise reduction the image is subject to global segmentation. It is automatically thresholded on the basis of local histograms and the 3-D connected components of the resulting two level image are labelled. Cells which are out of focus and too complicated to segment are rejected from further evaluation. Also the cells which are at the border of the image are rejected since the completeness of such a cell nuclei can not be ascertained. Cells which are touching each other are first selected for semi-automatic segmentation. The center of such a cell is selected by clicking the mouse over approximate centroid. This point will act as a seed and the seed is grown in all the directions using the threshold derived from local histograms.
The overshoot of the marked region into neighboring cells which touch each other is checked by the global shape constraints like maximum extension and in a second step by an adjustable ellipsoid surface which is fitted in such a way that the cell nucleus falls just within.
After separation of the touching cell nucleus, the image volume is subjected to a relabelling process to enable the newly segmented cells to have a distinct label (Fig.7). Segmented cell regions are selected in the signal channel for further processing.
Several different methods beginning from thresholding via linear transformations to non-linear ones, e.g. transformations derived from mathematical morphology [12, 10] were applied. For the actual material with small extension in x-, y- and z-direction FISH signals in 3-D are represented by small grain like structures.
A top-hat transformation was considered best to enhance the FISH signals for segmentation. The top-hat transformation image is the difference between an original grey scale image and an opened version. The radius of opening denotes the diameter of the top-hat. A threshold of top-hat can be considered as the height of the top-hat. By the way, the term top-hat is derived from 2-D image analysis where the operation can be considered as putting a top-hat upon the intensity profile. This image cannot easily be transferred to 3-D situations, where the signal intensity has to be considered as 4th dimension. The applied top-hat transformation corresponds with the simplified algorithm proposed by [16] for 2-D compact spots, where additionally a watershed transformation is necessary to detect small neighbored spots. The latter is not necessary by applying the full 3-D top-hat transformation. All operations are performed as 3-D operations.
After thresholding the top-hat transformation the resulting (binary) volume is processed by the selection of one connected object after the other, using again 3-D volume growing. A grain is accepted as FISH signal after a size and shape criterion and its being located inside the labelled nucleus.
The goal of quantitative FISH about pathological alterations is the count of gains and losses of chromosomes. The physiological detectable disomic nuclei (2 signals) in tumor tissue are not in the first place of interest. Of most importance is the number of appearing trisomies (3 signals), polysomies ( 3 signals) and monosomies (1 signal). The subjective computer-aided evaluation should equip us with data about the spatial distribution of signals in the nuclei as well as about the size and shape of nuclei in 3-D.
During visual inspection of the volume the coordinates and size of each selected box with the subjectively estimated signals are stored accompanied by the FISH signal count. This count list is subject to further analysis either by pooling count lists of several volumes or by using the box locations as expert selected seed for semi-automatical analysis.
First a global (threshold) based segmentation and object labelling is performed. The pathologist can display the result and decide which nuclei are to be counted and which are to be deleted. Selection or deletion are done by clicking at the labelled object. Improvement of segmentation is performed by selecting the object and proceeding as explained earlier under the section segmentation of nuclei. For labelled objects the FISH signals can be automatically counted, the box coordinates of the surrounding box, the count, signal and nucleus volume and the label number are stored. The FISH signals are counted of either all labelled nuclei or only of interactively selected ones. In the latter case a gallery of a box determined by the size of the selected object can be displayed, comparable to interactive processing.
Beside FISH signal count the nuclei as well as each signal are accessible for further feature extraction like calculation of volumes, mean total intensities and variations [4] and signal distribution in space.