The importance of high technologies can’t be overestimated when they are used in such vital spheres as education, science, medicine, etc. For example, virtual reality technologies are already actively applied in science and medicine. One more project that can help surgeons to reduce the operating time was developed by our team. Using the OpenCV library we developed a way to increase the clarity of images during the process of endoscopy.
Something about medicine
Endoscopy is a modern instrumental diagnostic that allows looking inside hollow organs without a scalpel. Endoscopic research methods expanded the diagnostic and therapeutic opportunities for physicians. However, there is an unsolved problem of the loss of image clarity when transurethral contact laser lithotripsy is taking place. When urinary concretions are crashed, the laser radiation is absorbed by the stone. Then a micro crated is formed on the surface of the stone, but the micro-explosion leads to turbulent turbidity of the physiological fluid around the stone. As a result, the stone on the surgeon’s monitor becomes invisible.
How to fix this problem? You can use some image processing methods of noise suppression. If you correct an image, you can increase the clarity of a stone and speed up the operation time.
How we found the solution
In general, if you want to restore an image distorted by noise, you should minimize the distance between the original image and the restored one. There is plenty of denoising methods that have been invented since the end of the 20 century. They differ in applying different restoring models. But the common thing here is the function of averaging over a certain spatial neighborhood.
Nowadays various tools can help solve the problem of capturing and processing video streams. The best way is to use OpenCV. That is a computer vision and machine learning library. It has open-source code and interfaces in different programming languages, including Python, Java, and C++.
We investigated the 1CCD HD ENDOCAM Performance HD endocam. The analyzed video file has a resolution of 720 × 576(1.25:1) at a frame rate of 50, compressed by the AVC1 codec, digital format *.mp4 (*. m4v) and represented in the space YUV 4:2:0 (Y’CbCr). YCbCr can be easily converted to RGB for simple processing, as well as a reverse conversion is possible.
The essence of the developed technology is in the following. To increase the clarity of an image a real-time video stream or a video file of some format are processed in a loop. The first step is to provide pre-filtering of the current frame. Then the noise component is isolated when a stroke is detected. If there is a positive reaction of the noise detector, frame-by-frame processing starts. The image is stored in the buffer before the effect of the laser radiation. After that, the original image is subtracted from the noisy frame and the noise component is highlighted. After that, the noise component is subtracted from the subsequent noisy frames, the relevance of which is variable. There is also the image post-filtering, due to which its clarity is significantly improved.
The Laplace operator
To detect contour lines or improve the quality of images the Laplace operator (Laplacian) is used. It calculates the second derivative of the function by the formula:
To do this, we take one image channel (mainly grayscale) and collapse it with the next 3 x 3 kernel (Laplace kernel):
Then the variance (i.e. the standard deviation squared) of the response is calculated.
The described method is quite simple and can be implemented with one code in Python.
cv2.Laplacian(image, cv2.CV_64F).var ()
# Blur detection using OpenCV
If the variance falls below a predetermined threshold, then the image is considered blurred. Otherwise, the image is not blurred.
Software development
According to the set goal, we developed a software implementation of the algorithm to remove turbidity from the video stream. The method is based on digital filtration of the endoscopic image. It allows conducting the procedure more quickly and not to wait for the contours to appear. It reduces the duration of additional manipulations and the duration of the operation.
The OpenCV library provided enough functionality for the implementation of all image processing operations. It allowed us to develop the app with a concise code and provide possible modifications in the future.
As a result of testing, the following results were obtained:
- the main module copes with the task of capturing a video stream from various sources and placing frames in a buffer for processing and subsequent output, saving;
- the pre-filtering module allows increasing the clarity of the final image, eliminating it from random noise by applying a mask;
- noise detector dynamically determines the fact of “blurring” of the current frame based on the variance values for all buffer frames;
- algorithm (noise exclusion module) allows correctly selecting and excluding foreign objects in the frame (particles, general turbidity of the frame);
- post-filtering module equalizes the histogram of the frame and filters high-frequency interference.
Result
Our developed project on OpenCV is aimed at improving the working conditions of specialists in the field of medicine. Thanks to the use of the OpenCV library, we have achieved our goals and clearly demonstrated the work of our product. If you have an idea for a project, write to us, and our team will help you implement it!
The original article is published here: https://joy-dev.com/opencv-in-medicine-how-our-developers-helped-surgeons/
