… 1 successful computer vision disease detector

400 images, 6 days, 3 graduates …

Imagine it is winter, you wake up and you don’t feel so well. You decide to go to the doctor. Nowadays the doctor would advise you to stay home and take a COVID-19 test. A few years ago, it was a different story. The doctor would have taken a simple blood sample instead of sticking an enormously long tube inside your nose. The blood sample is then sent to the lab where lab assistants analyze your blood. There the lab assistants conduct some tests. One of the most important ones is the identification and counting of different white blood cells to identify possible diseases.

Forgotten the purpose of white blood cells? Let us go back to biology class in high school. There are five types of white blood cells: neutrophils, eosinophils, basophils (out of scope), lymphocytes and monocytes. Each white blood cell can be classified by structure (granulocytes or agranulocytes). The first three are granulocytes and the last two are agranulocytes, with each their function. For example, whereas neutrophils mainly target bacteria and fungus, eosinophils target larger parasites, such as worms. The number of white blood cells is often an indicator of disease. Therefore, an increase of eosinophils in the blood would indicate a parasitic disease.
In large laboratories, blood tests are handled by expensive devices. In smaller laboratory lacking the resources, this job is often done manually with the help of indicators which is labor intensive.

It is possible to distinguish the different types of white blood cells with the naked eye by recognizing some key features. The granulocytes are characterized by specific granules in their cytoplasm, where the neutrophils have a higher number of lobes than the eosinophils. The lymphocytes and monocytes are recognized by the absence of granules. The latter can be distinguished from the first by its grainy texture. However, not every white blood cell is recognizable with the naked eye. In this respect, a tool that automatically recognizes, classifies and counts white blood cells is really interesting.

This is where we, Yen De Clercq, Jonas De Vos and Joshua Van de Kerckhove come in. For our Young Professionals case in September 2020, we got the opportunity to explore data science in a medical field. During six interesting days, we explored the possibilities on how to solve this case with a hands-on approach and solved it as well.

By now the reader should know how awesome and relevant this case is. The question is however, will it perform? We try to automate steps in a process that aims at helping people with health problems. The consequences of passing on false information can be a little more severe than typical applications trying to classify dogs and cats.  Every machine learning practitioner knows that enough data is essential for attaining good results and let that be the first problem we faced. We had only 400 microscopic blood images. Luckily, there exists such a thing as data augmentation which applies transformations (zooming, rotating, flipping, etc) to the images, elevating the initial 400 to 12.000 augmented images.

Let us zoom in on one microscopic image.  You can typically see three components: blood plates, red blood cells and white blood cells. Our end goal is to be able to tell which white blood cell is present. Hence, the red blood cells and blood plates are redundant information. In fact, they will only bring noise to our algorithm. So, we decided to cut out the white blood cells out of the image in order to boost model performance. Manually cutting out the white blood cells out of 12.000 images would be incredibly time consuming. Time that Ordina most certainly does not like to pay out.

Building an automatic cutting tool can only be done if the algorithm knows the specific coordinates of the white blood cell(s) in the picture. Hence, the tool must first detect the white blood cell. Object detection is a field of computer vision that evolved a lot the last years. Nowadays the algorithms are pretty performant as transfer learning became the norm. Now what does Wikipedia tells us about transfer learning: ‘Transfer learning is a research problem in machine learning that focuses on storing knowledge gained while solving one problem and applying it to a different but related problem’.  What makes transfer learning so powerful is that third parties with more computing resources (Google, Microsoft, Facebook) have trained for weeks on millions of data points and open sourced these models. These models are already so powerful that it suffices to tune the model a bit towards your specific case and train it for an hour or so in order to gain good results. We made use of the YOLO-model. The interested reader can read more about it here. The model returned an xml-file consisting of coordinates of the bounding box around each object.

One obstacle we faced when cutting out the images based on the bounding box coordinates is that some white blood cells were cut in half or worse. Two reasons explain this behaviour. First of all, the white blood cells are in nature round, cutting out rectangles are suboptimal. Secondly, the model isn’t perfect. It can happen that the centroid of the bounding box doesn’t match the centroid of the real cell. We solved this by replacing the bounding box with bounding circles. The centroid of the bounding box became the center of the circle and the radius was set a bit higher in order to make sure the entire white blood cell was captured.

Subsequentially the cutout-white blood cell photos, of course without the bounding circles, went into our last model. Again a transfer learning model named VGG16 was used. It is a convolutional neural network model that is composed of 16 layers. The first layers capture high level information like edges, roundings, etc. The deeper you go the more specified the learned information becomes. By adding a few more layers to train and freezing the VGG16’s already learned weights, we reached 92% accuracy in 30 minutes on 4 of the following four classes: Eosinophil, lymphocyte, monocyte and Neutrophil. As time was limited, we did not tune the model much, making us suspect that the accuracy can still be improved. A second model was made which was built from scratch. After 30 minutes only 53% was attained which shows the power of transfer learning.

Once the model was ready, it could be integrated into an app. We made the app in Kivy. It is an easy to use python module, but it has its limits (scalability, deployment). A demo of the app can be seen by clicking on the this link. At the moment of writing, our colleagues of NCORE are building an app to overcome Kivy’s main shortcomings.

To conclude, it was a very exciting first case for us Young Professionals. We learned a lot about computer vision as well as coping with problems regarding resource limitations.

A special thanks to Yoni Geerinck for guiding us through this project. We look forward to our first project for one of Ordina’s customers.