Finding Common Threads: How We Evaluated Diverse Gates Agronomy Grantees
The field of agronomy–the science of soil management and crop production–has seen rapid changes in recent years.
Traditionally, agronomy focused on educating farmers on practices to improve soil management and crop yields. However, international research in agronomy usually was disconnected from the needs of smallholder farmers in developing countries, with limited to no assessment of farmer demand for the research being developed. In addition, agronomy had typically not made much use of innovations in digital technology.
For many years now, the Bill and Melinda Gates Foundation has been a major funder in the field of agronomy research, with a $300 million agronomy investment portfolio. As the foundation realized it needed to emphasize demand-driven research and technology innovations in agronomy, it began to fund a series of research grant projects across multiple countries that focused on demand-driven research and digital technology.
After more than a decade of funding these types of individual grants, the foundation decided to instead begin investing in Centers of Excellence in Agronomy (CEIA) as a centralized, more systematic approach to maximizing agronomy research across countries.
This is where Abt Associates comes in.
The foundation wanted to assess common approaches, lessons learned, and common challenges across several of its longstanding individually-funded agronomy grant projects, and use the assessment/evaluation findings to inform the work of the CEIA.
So, the foundation hired Abt to take five of the grant projects and evaluate them as a “cluster.” These grant projects spanned multiple phases and timeframes (some had finished, some were ongoing), worked in over a dozen different countries in Africa and Asia, had different approaches to conducting soil and crop research, and each focused on different crops. One focused on bananas, another cassava, a third maize, a fourth legumes, and the fifth focused on multiple crops – rice, wheat, and maize.
This evaluation design included extensive qualitative and quantitative data collection and analysis in a short timeframe--and all of the data had to be collected, some of it through multiple country in-person site visits, analyzed, and written up in a report within three months.
Despite a rapid start up, the evaluation team was only in the middle of our first site visit in India when the COVID-19 pandemic hit. We had to hastily flee back to the USA and quickly revise our data collection plans.
Getting Some Perspective
While dealing with these challenges and pivoting to collect all data remotely from the U.S. (including over 90 key informant interviews), after an initial review of grantee documents, we developed a single framework to visualize and help compare evaluation findings across the grant projects. Because our evaluation was guided by a series of research themes–including whether and how agronomy practices and technologies were scaled up, sustainability, and impacts at the farmer and crop levels–we included them in our framework.
This approach worked! Mapping the findings to the framework was the key to being able to conduct this “cluster” evaluation.
We found the grant projects across countries were indeed using demand-driven approaches to guide their research, and all had components of “big data” in their research –they were collecting large amounts of data for soil mapping or developing local decision support tools to inform crop and soil management, and many of these had a large digital component.
Answers Leading to Questions (Leading to Answers)
Interestingly, these findings brought up some real questions for the foundation to consider. Was it reasonable to expect research grants working on scaling and sustainability to show impacts within a five year grant period--did research grantees really have the time and expertise to do so? Why did many grant projects struggle with creating business models the private sector could use to take over their digital technologies? Do public institutions really have the capacity and skills to maintain large datasets after a project finishes?
In the end, we were able to provide the foundation with findings and learnings that they are using to inform the next phase of their agronomy investments, which feels like a big success. The foundation convened a webinar event with grantees, lead agronomy researchers, and partners from CGIAR (Consultative Group on International Agricultural Research) centers from around the world (over 200 individuals were invited) to discuss the findings from our final report. Evaluation findings are not always used by funding agencies, but the recommendations that emerged from this evaluation seem like they are being implemented, and at the very least, discussed with interest.
Finally, we have many more findings for you to read. For more detailed information on this evaluation and our findings, please see our brief, executive summary, and full report, all of which can be found here.
