Counting Mosquito Eggs with a Smartphone – Part 2: The Process for Designing an Automated Egg Counter
Assembling the Team
As a data scientist and project lead for AbtX, (Abt Global’s internal research and development lab), I often team with our regional entomologists, and together we have the responsibility (and often excitement!) of tackling problems in new ways where the path to success isn’t clear.
When we heard about the mosquito egg counting challenge, it set our team down a path of a lot of research, questions and tinkering. We wanted to know the environments where mosquito eggs were counted, how the process worked how much time it took, and about 1,000 more questions!
Using Human-Centered Design
Our human-centered design approach enabled us to flip the traditional model of software development on its head. We didn't develop an app for a group and ask how we did. Instead we enlisted the experts for their experience and insight to co-design and co-develop alongside us. This ensured that we converged on a useful solution and built in immediate feedback loops to avoid dead ends.
The first major challenge: understanding the counting process. We learned that field technicians and lab experts look for black dots on white paper. Sometimes they use magnifying glasses, microscopes or stereoscopes. Most of the time it's just counting by hand and maybe with a handheld clicker to help the tally.
The second challenge: understanding what tools are available. It was one thing to solve this problem with power computing and high-resolution sensitive microscope equipment in laboratory settings. It was quite another to imagine solving this in rural areas in Latin America and the Caribbean with limited access to labs, equipment and internet connectivity.
Building for a Smartphone Device
But our technicians routinely carried advanced equipment in their pocket: a smartphone. Basic smartphones have a digital camera able to take high quality images, access to the algorithms to process images and software that could put the pieces together for visual display, recording and transmission (at least where there’s a data signal or Wifi access). We just had to get it all to work.
Building the progressive web app
Our team began looking into ways to use computer vision — that is, digital object recognition -- to process an image of these papers with the eggs on them, identify the objects that are eggs and output the number of eggs that it "sees". We needed something that would be easily accessible, easy to use and available on a smartphone or laptop with little or no internet connectivity.
I'm happy to say that to-date, our proof of concept has made great strides — and it actually works! We are happy with it so far for two reasons: how we approached the basic design and the technology tools we've assembled to make it work.
The Progressive Web App and Computer Vision
Now, normally at this point your eyes might glaze over and we lose you as I talk about software development. But stay with me because this is actually pretty interesting.
In the web browser world, new standards and functionality improvements have been rolling out that enable regular websites to have the functionality of native apps built for a specific platform. What’s that mean in practice? From your browser, you can work offline, access device hardware such as cameras and place them on the home screen. Our web app is notable additionally for its development as a flexible, scalable and low-cost progressive web app that uses open-source technologies and can be used by iOS, Android and web browsers, even in offline environments.
Earlier web apps to count mosquito eggs were bulky (around 700MB), tedious to install, limited to desktop PCs, and didn’t expose their underlying source code or methodology. That made it difficult to implement effectively when you can't be confident in the data you're reporting.
Our web app comes in at under 10MB and can work on any desktop web browser or smartphone. Additionally, we use popular software libraries, with obvious algorithmic inputs, so that others could replicate these numbers in the laboratory or modify it for greater customization and precision. It combines good design, a small footprint, an eye towards usability, and modern, low-cost web frameworks for a tool that is simple, cheap, and easily handed off to local governments and programmers.
This is great news because it means that we can make things more useful, while becoming much simpler. When do you ever get that in technology?
Oh, and did we mention that we're going to be making this open-source?
The Future
We are confident that this tool will make a significant contribution to the teams working with mosquito surveillance. The graphs below describe the advantage of using our app compared with human counting of eggs. The MEC vision is extremely fast, yet its accuracy is no different from the accuracy humans achieve.
So here's where we are today:
We're testing and iterating on our proof-of-concept, and now we're back in the lab tweaking and learning as we go.
We've recently been able to work with the USAID-led Zika AIRS project to explore what this would look like in the field, and how we could roll it out into different countries.
As for next steps, we've begun experimenting with machine learning to build out an AI algorithm that is even more adept and more accurate with the egg counts. This tool is currently under trial in Jamaica, Paraguay, Honduras and El Salvador.
The thing that I find so exciting about this project is that the design process, the approach, the technology stack -- it's all reusable, scalable, and easy to incorporate into other areas. We are already looking at using the same basic components for use in so many other areas.
It's not just about counting mosquito eggs -- it's about helping global health teams to solve a problem that they've been wanting to solve, but not having the tools in place to do so. Any good technology is simply an extension of an original intent to solve a problem in the real world.
Read part one of this blog: The Challenge
