Combined Fluorescence and Bright Field Imaging via a Single CMOS Detector Without Filters to Improve on Patient Safety during Endoscopic Procedures – An Experimental Study Analysing the Output of both LASER and LED Near Infrared Sources on In Vitro Samples

Many endoscopic systems combine white light and NIR light via LED multiplexing in order to image tissue stained with Near Infrared (NIR) fluorescent markers such as Indocyanine Green (ICG). This can result in unnecessary  in vivo  energy exposure which can cause irreversible damage to tissue. A simple, single detector system is proposed in this work comprising a single excitation channel at a wavelength of 780 nm. We have demonstrated that a single 1.6 Megapixel CMOS camera with quantum efficiency of less than 30% is appropriate to capture both fluorescent and non-fluorescent landmarks at NIR wavelengths. Experimental results indicate that a LASER source generating between up to 10 mW of optical power at 780 nm could be considered as an alternative to LED.


Introduction
There is a growing awareness of the benefits of fluorescence endoscopy for the complete and efficient removal of diseased tissue. Cancerous cells are stained with a fluorescent chemical marker that essentially illuminates under the correct excitation, to indicate location and to verify effective removal. There are many new LED-based systems making their way through clinical trials and subsequently becoming available commercially. To name a few, the KARL STORZ NIR/ICG Imaging System [1,2], Photodynamic eye [3], SPY [4], FLARE [5] and miniFLARE [5] are all experiencing success within the field. All of these systems are capable of exciting the chemical marker Indocyanine Green (ICG) with LED light at Near Infrared (NIR) wavelengths causing fluorescence to enhance clinical imaging, ICG is currently one of two FDA approved dyes at this wavelength range [6] and has been approved for medical use since the 50's [7].
Typically, when using ICG as an identifying marker for diseased tissue, a white light channel is required in order to view non-fluorescent/unmarked landmarks within the frame such as blood vessels [8][9][10]. The two channels individually do not allow for a complete resection of diseased tissue with minimal removal of healthy tissue as the imaging conditions are not optimal and this can prove very dangerous [11]. The most successful protocol is to combine data from the two channels which involves risks and complications. The main hazards being excessive excitation used thus exposing the patient to a higher risk of thermal damage under minimally invasive surgical conditions and incomplete, slow or inaccurate combining of the two data streams.
Unfortunately, the mechanism involved in efficiently focusing light through the liquid light guide (small aperture) for endoscopy produces heat as a by-product, this thermal energy can become intense and cause permanent damage to tissue [12][13]. Due to the number of endoscopy devices certified safe for clinical use, it is evident that the benefits outweigh the disadvantages with regards to the risk of tissue damage. Optical power of the NIR source should be balanced, the penetration depth should be optimal without adding to the unwanted thermal output [14]. Optical power used in this research is between 5 mW and 10 mW in agreement with relevant literature [15][16][17].
An illumination device manufactured by Cymtec Ltd was delivered to Imperial College London for the imaging of cancerous tissue ex vivo. The Cymtec light source multiplexed isolated Red, Green, Blue and NIR LEDs to allow surgical staff to adjust for white balance with optional NIR at 780 nm for ICG imaging. A homogenous output is observed at the distal end of the liquid light guide thanks to the patented light pipe technology. Surgical staff at Imperial College London explored frame interpolation using a single colour camera to detect ICG fluorescence [18]. This research follows on from the work completed in collaboration with Cymtec Ltd, using a lab-based setup comprising a single NIR illumination channel and single monochrome camera. Sudden closure of the company partner Cymtec Ltd and extreme cost of replicating this design required simplifying the system to focus on the NIR component only for in vitro tests.
Excitation power of the Cymtec light source is higher than expected at 50 mW/cm 2 , determined to be the photobleaching threshold of ICG according to literature [15,17]. An acceptable power range is documented as 1 to 10 mW/cm 2 [16][17] which is true for the previously listed devices within the wavelength range of 670 nm and 800 nm.
Literature focusing on health and safety suggests that excitation could be pulsed to reduce the thermal effects [19]. This experimental research will demonstrate the viability of a system comprising one excitation channel and a single CMOS detector. The aim of this work is to image ICG stained samples using an inexpensive camera in order to capture both fluorescence and bright field information using a single illumination channel without filters. Two systems, one comprising an NIR LED and one an NIR LASER will be investigated as there are currently no commercially available systems utilising LASER sources for minimally invasive fluorescence endoscopy and this could remove the need for expensive filters.

Materials and Methods
Samples of various concentrations of ICG solution are prepared for this experimental research, the ICG is mixed with Human Serum Albumin (HSA) in order to enhance stability and fluorescence [15,16,20]. The samples are illuminated with either LED or LASER excitation at 780 nm in order to excite the chemical marker causing it to fluoresce. This fluorescence is detected using a spectrometer and captured with a CMOS camera. Data collected is processed to determine if it is possible to capture fluorescence and bright field information using a single detector without the need for filters.

ICG Sample Preparation
ICG (Tokyo Chemical Industry UK Ltd) and HSA (PAN Biotech UK Ltd) are mixed in a 1:1 molar ratio consistent with relevant literature [15,[21][22], a 1000 µMol solution was prepared allowing a sufficient level of dilution for the desired range of concentrations analysed. In order to prepare 30 ml of stock solution at a concentration of 1 × 10 -4 Mol, 111.7 mg of HSA and 23.2 mg ICG are dissolved in 30 ml deionised water, this solution is diluted down to 1000 µMol or 1 × 10 -6 Mol by pipetting 0.10 ml of stock solution into 9.90 ml of deionised water. A solution is prepared daily and kept under dark conditions. The temperature of the environment is monitored throughout using a thermometer with external probe and recorded frequently, there was no significant changes in ambient temperature to note. Fig. 1 shows the stock solution being pipetted from a plastic container alongside three cuvettes filled with diluted samples of varying concentration. The stock solution does not fluoresce and is the lowest concentration of ICG : HSA achievable with the available apparatus, dilutions between 400 µMol and 800 µMol are prepared in accordance with relevant literature. It is essential to use a transparent mixing vessel to prepare the sample to ensure that the dry constituents are dissolved however, this is kept covered to protect the sample from degradation. For reference, the cuvettes have a pathlength of 10 mm and the glass thickness is 1.25 mm.

Experimental Setup
Two similarly arranged systems are assembled for this experiment. One comprises a LASER light source while the other utilises a single LED, (insert A and B) as seen in Fig. 2. Both the LASER and LED (Thorlabs L780P010 and Thorlabs LED780E respectively) emit light at a central wavelength of 780 nm, the output power of the LASER is manually set to either 5 mW or 10 mW by adjusting the current in mA, a calibration curve is plotted to confirm the corresponding current to output power by reading the results from a power meter while adjusting the current in small increments. Once the power threshold has been met, the relationship between current and optical power is linear. A simple circuit is setup to control the LED providing 1.75 V comprising an LED controller unit (Gardasoft CC320) for computer controlled on/off and pulsing. Forward optical power for the LED is listed as 7.2 mW as per the manufacturer's technical specifications, this corresponds with power meter readings taken from the perspective and positioning of the cuvette containing the samples.
For continuity, key apparatus such as the spectrometer (Ocean Optics USB4000), power meter (Thorlabs PM100A fitted with a S302C Head), thermometer and CMOS camera (BlackFly 16S2M) are disconnected and swapped out as and when needed so that the optical stages and mounts can remain fixed. With regards to the CMOS camera, the Quantum Efficiency of the incorporated sensor (SONY IMX273) at 800 nm is around 26%, resolution is 1440 × 1080 and the pixel size is 3.45 µm. The power meter detector and the fibre end of the spectrometer can interfere with imaging due to reflection so these are arranged in a flexible mode whereby they are simply set outside of the frame when unused. Two thermometers fitted with external probes are also used during data collection, one for monitoring the temperature of the ambient environment while the other is available to be placed inside the cuvette and submerged in the liquid contents of the cuvette as and when required. All experiments are completed under dark conditions with analysis post data acquisition. The insert C in Fig. 2 shows a schematic of the sample as seen from the perspective of the CMOS camera which is arranged perpendicular to the excitation source. (1) contains the liquid sample only, (2) contains the liquid sample with the probe end of the thermometer in position and (3) shows the fiber end of the spectrometer visible through the liquid contents. Images are not collected under condition (2) or (3) from the figure due to the unwanted reflection caused from the surface of the temperature probe and the spectrometer fibre respectively.
Although 780 nm is visible to the human eye, a beam viewer card was used during the LASER setup to ensure that the beam is striking the cuvette at the centre of the first surface and exiting the final cuvette surface without distortion. The beam is not collimated but an aspheric lens in used to minimise optical aberration. As the cuvette is positioned manually, inaccurate positioning can result in reflection and scatter which is visible at the furthest corners of the cuvette from the source. This is evident from the images collected and can be rectified with very gentle twisting of the vessel. The housing of the LED allows for a much simpler aligning to the central face of the cuvette and this is achieved with the LED off for safety. Fig. 3 shows the two configurations with cuvettes in position.

Data Collection
Data collected from the CMOS camera is analysed in a custom LabVIEW programme utilising the vision and motion palette. The programme displays an intensity analysis of pixels within a user-controlled Region of Interest (ROI). This ROI is saved to memory to keep the position and size consistent between images and can also be re-sized and re-positioned when needed. Data from the images is available in real-time and with units according to the parameter under investigation. It allows extraction of various features such as the mean intensity of the region given in grayscale values and the minimum and maximum intensity which provides the contrast available for example at the border of fluorescence. This can also be useful when looking at the intensity of the fluorescence across the pathlength of the cuvette to understand the effect of absorption of the fluorescent material. Numerical values are entered into a spreadsheet ready for plotting using software visualisation and analysis tools. For the purposes of this work, Origin (OriginLab 2020) package was used.
Spectrometer data is exported from the OceanView software (Ocean Insight) in .csv format which is then analysed in Excel. Data collection was repeated six times to enable computation of the mean and standard error. Unfortunately, the spectrometer traces collected show noise across the wavelength range for all repetitions and so a Savitzky-Golay smoothing function was applied to filter out the noise while preserving the shape of the peak. The smoothed spectrometer data is plotted to show the relative intensity of fluorescence for each excitation source and concentration on test. The power meter and temperature data are recorded manually from their respective display interfaces and input into Excel for processing before plotting, processing again involves calculation of the mean and the corresponding standard error.
The camera frame rate and LED pulses are software controlled. SpinView is used to set the camera frame rate through adjusting the exposure time in microseconds to a minimum of 4855 µs resulting in a maximum of 199.98 frames/sec, ~200 Hz. Gardasoft Maintenance software connects the LED controller to the laptop via Ethernet connection, values are entered manually with a minimum available pulse rate of 0.1 ms. The LED can also be set high (on) or low (off) and the time on phase and time off phase set independently.

Results and Analysis
Fluorescence data is collected under each excitation condition and the results are presented in Fig. 4. With the exception of the water sample in Fig. 4(c), a peak analysis of the data presented in Fig. 4 determined the fluorescence peak wavelength is around 815 nm, 35 nm longer than the excitation peak wavelength at 780 nm. The excitation peak is present only in the sample containing water, illuminated with LED as seen in Fig. 4(c). We do not observe the excitation peak with the water samples illuminated with LASER excitation as the beam does not diverge across the cuvette walls adjacent to the source, the detector head is positioned adjacent to the source and therefore nothing is detected across the wavelength range as seen in Fig. 4(a) and Fig. 4(b). Fluorescence intensity is highest under 10 mW LASER excitation with the LED producing a more intense fluorescence than the 5 mW LASER condition. Consistent with literature [16,23], the lowest concentration on test provides the highest fluorescence intensity under each excitation condition. The data presented is a representation of multiple days of repeat testing and so this cannot be put down to a rogue mix or an error in calculating the dilutions. The reference cuvette containing deionised water shows no fluorescence in Fig. 4(a) and Fig. 4(b) as expected however, Fig. 4(c) shows the LED spectra with a central peak at 780 nm. The reason for this is due to the divergence of the LED beam, the light that travels through the cuvette containing water at the excitation wavelength is detected by the spectrometer head. No fluorescence is detected due to the lack of ICG present in the sample. The results presented in Fig. 4(c) give a good representation of the term intensity which accounts for the y-axis of the spectra, the spectrometer and accompanying software defines fluorescence as a measurement of the relative irradiance and is not a quantified numerical value. The LED provides just over 7 mW of forward optical power and so the peak represented by the cuvette containing water only represents a portion of the source which has travelled through two cuvette walls (optical glass) and a volume of water to reach the detector positioned perpendicular to the source.
To verify that HSA has no fluorescence properties, a long exposure experiment was conducted using water, a 500 µMol solution of HSA and a 500 µMol solution of ICG : HSA as seen in Fig. 5. These images show the near and far walls of the cuvette as well as fluorescence from the sample containing ICG. The beam travels through the cuvette containing liquid samples from left to right, artefacts can be seen within the HSA and water images and this is from reflection, reflection from the fibre end of the spectrometer is clearly visible as a circular region of intensity near to the left-hand side of the cuvette.
To quantify the intensity of the fluorescence, the spectrometer data is combined with images captured with a single CMOS camera. No filtering was applied to the camera setup as confirmed by the spectrometer results. With the exception of water under LED excitation, there is no source wavelength within the results. The camera is an equivalent position to the spectrometer, in parallel to the source so that the cuvette and its contents are imaged.   The camera used is powered and controlled via a USB 3 port, the following Fig. 6 shows an example of images collected, the data extracted from the pixel analysis for this experiment is presented in Fig. 8(b), Fig. 8(c) and Fig. 8(d) alongside the corresponding 5 mW LASER and LED excitation. Each pixel within the selected region of interest contributes its own intensity values, the camera is 8-bit grayscale and so these values range from 0 to 255. A total intensity combines all pixel values within the region of interest, mean intensity divides this value by the number of pixels within the region of interest. For the following Fig. 6 the mean value for the 400 µMol sample is 74.946 with a standard deviation of 85.532. The mean value for the sample containing water is 7.346 with a standard deviation of 12.304. The standard deviation gives a good indication of the contrast present in the image and this is key to identifying diseased tissue within the frame and can ensure safe removal of virtually all diseased cells. The ROI is selected manually to encompass the rectangular area between the walls of the cuvette (to the left and right of the frame) and around twice the width of the widest saturated area (to the top and bottom of the frame). An example can be seen in Fig. 7 and this area is kept constant between frames as the camera remains in a fixed position. In each of the frames representing fluorescence, there is a cone shaped region of saturated pixels. The shape of this cone changes to reflect differences in fluorescence intensity due to changing experimental parameters. The average intensity values within the ROI would therefore highlight any significant differences in the area of this cone which is a feature on all fluorescing frames, as well as any fluorescence above and below the saturating region. As the ROI is fixed, a higher average intensity value would represent a wider cone shape as well stronger fluorescence above and below the cone. Fig. 7 shows the front end of the LabVIEW programme whereby a region of interest is selected manually, the walls of the cuvette can easily be determined as can the dome of the LED from the righthand side. The selected ROI encompasses all fluorescence within the cuvette. Also visible in the image just below the meniscus of the solution and just below the LED are spots of reflection from equipment much farther back from the apparatus, an unfortunate error which is at least present in all LED data including the water reference sample.   To assess the thermal effects of exposure, a thermometer with external probe is placed inside the cuvette and recorded every 30 sec. Data presented in Fig. 8(a) suggests that there is more thermal instability when using an LED source and it is important to note that there are no optics between the LED and sample to concentrate the light and heat further however, the LED was also not cooled as this was deemed unnecessary due to the short time on and strictly adhering to the operating instructions. The temperature of the LASER diode was maintained at 25 ºC via the mount and was fitted with optics. The temperature of the LASER diode was maintained at 25 ºC via the mount and was fitted with optics. Temperature changes recorded in the first 11 min of source on are insignificant however it is reasonable to comment on the perceived danger of long-term usage especially considering the addition of a lens with the LED. The average temperature increase over a period of 11 min is shown in Table 1.  The effect of exposure time on fluorescence was investigated, the mean pixel intensity of a 500 µMol solution within the region of interest with varying exposure time is presented in Table 2. Values are in Grayscale units; the ROI represents an area of 260804 and 287650 for the LASER and LED excitation conditions respectively.

Discussion
The results show that LASER should be considered as an excitation source with ICG imaging for this application. Overall rise in temperature inside the cuvette over a period of 11 min with the 10 mW LASER was on average 50% less than the LED, the 5 mW LASER was on average 80% less than the LED. These temperature differences are significant as this was an observation over a relatively short time, in real world applications whereby human tissue is exposed to light, it is reasonable to assume that procedures will typically take longer than 11 min and it is understood that temperature increases with time exposed. For in vivo applications, temperature increase should be insignificant so as to reduce the risk of thermal damage associated with longer exposures. Existing LED and Xenon bulb technology have shown to increase local temperature rapidly, up to 10 ºC in less than a minute of exposure [12] the experimental system offers around 20% of the maximum power of existing systems and does not include any focusing optics other than the collimating lens of the LASER. LASER excitation delivered to the sample is coherent and wavelength specific so there is little need to use filters, this is especially relevant given that excitation is within the visible range yet the fluorescence is not. Additionally, there is also little need for focusing optics which can result in costly design, manufacturing and maintenance of equipment. Furthermore, cost of the LASER diode used for this research was significantly less than the LEDs of comparable wavelength and power. This research aims to investigate the use of LASER as an alternative to LEDs in endoscopic light sources, there are drawbacks to using LASER such as the reduction in reliability lifetime as well as more stringent health and safety protocols amongst others.
The most important finding of this experimental research is that light at 780 nm can be used to image ICG stained samples without the need for a secondary, designated white light channel. At 780 nm, the ICG samples emitted fluorescence that was detectable by camera only at around 815 nm. The 780 nm excitation was detectable by both the human eye and camera therefore, if there is a region of intensity visible within the camera frame that cannot be seen with the naked eye, we can confidently say that this is fluorescence. If a region of the camera frame shows pixels with high intensity values where a bright region can also be seen with the naked eye, for example, reflection from the source -then we can confidently say that this is not fluorescence.
Using only the 780 nm channel, it is possible to view the fluorescent and non-fluorescent content within the frame. While the frame would not be viewed in optimal conditions, using a setup similar to this experimental analysis, the patient would be subject to just one excitation source therefore reducing the risk of thermal damage by 50% by eliminating a secondary source.
The camera used during this data collection was a monochrome camera and this was selected in preference to the colour alternative due to the quantum efficiency curve at NIR wavelengths. All images collected, including that shown in Fig. 7, shows the grayscale image of the setup whereby the fluorescence and rest of frame are visible, the fluorescence is saturating the frame however, a higher frame rate would reduce this saturation and allow a more precise boundary of fluorescence. The highest frame rate tested was 200 Hz using an exposure time of 4855 µs which significantly impacted on image quality of the non-saturating regions of the field which were left undetected. A frame rate of 120 Hz was acceptable in terms of contrast and detection of nonfluorescent landmarks within the frame, see Table 2.
Future work should consider the use of the colour camera to see if this can improve on clarity and enable better distinguishing between fluorescing material and reflecting (815 nm and 780 nm).

Conclusion
From this experimental work, LASER as an excitation source performed better than LED when used to excite the fluorescent marker ICG at 780 nm. A single J of Biomedical Photonics & Eng 6(4) 31 Dec 2020 © J-BPE 040303-8 monochrome camera as the sole detector is viable and could be used for sub-optimal imaging allowing for a significant reduction in patient exposure to energy that can be harmful.
As patient safety is a priority, the benefits of reducing energy exposure should be considered along with the harm that removing a white light channel would cause with the proposed system as future work. If the objective is to detect the presence of diseased (ICG stained) tissue and give a good indication of its size and location then a simpler single channel system may be sufficient and significantly reduce the risk of thermal damage to healthy surrounding tissue.
A variety of parameters and conditions have been investigated and the findings presented to demonstrate the similarities and differences between the current illumination techniques and how these are detected using a single monochromatic camera. The main objective is to enhance patient safety during minimally invasive procedures while maintaining high quality, real-time images with clear contrast between fluorescence and background.

Disclosures
All authors declare that there is no conflict of interests in this paper.