Summary
Established in the 1960s, the Philippine-based International Rice Research Institute (IRRI) is one of the premier agricultural research and training organizations dedicated to address food security among regions that heavily depend on rice as a staple in their diets.
However, keeping track of their projects’ impact is impeded by the slow process of accessing and viewing necessary information such as global rice production and consumption statistics.
The organization worked with Senti AI to develop a one-stop database that can be used by its staff, partners, and other members of the academe or government. Senti AI built an Extract-Transform-Load (ETL) pipeline with a dashboard through AWS for easier viewing and accessing of data needed for research, decision, and policy-making purposes.
The Challenge
For decades, IRRI conducted surveys across the globe as part of their research on developing better farming systems. The organization also conducts collaborative research, partnerships, and strengthening of the national agricultural research and extension systems of its partner countries. In the Philippines, specifically, they gathered data to monitor changes in rice farming.
However, processing and making use of these data for analysis proved to be difficult as these files are stored in silos and there is no centralized database. IRRI’s surveys were also stored in different data formats, which made it more time consuming to access the information they needed.
In its bid to deliver better projects with more impact, IRRI sought out its own unique resources that can aid in decision-making processes. The database is only part of the organization’s larger work which is to develop a platform that combines data from its two ongoing initiatives.
The Solution
Senti AI and AWS explored an Extract-Transform-Load (ETL) project with the organization to create a one-stop reference database that can be accessed by the organization and its stakeholders as well as partners. The database contains raw data taken from household surveys they conducted from 1965—2016, as well as external socio-economic data from national and international organizations that IRRI works with.
Senti AI made use of AWS Glue to organize raw data stored in S3, which were initially in csv and excel files, before coursing these to Amazon Athena where it will be connected to Qlik Sense for data visualization. The project output allows IRRI’s researchers or partners to access these socio-economic data.
The database’s highly customizable dashboard also has multiple filters that can speed up the search process while the data analytics can aid in measuring the organization’s progress and impact on communities. These features ultimately seek to improve the decision-making processes of the organization and other users of the resource.
The Effect: Accessible information for all
The deployment of the database and its dashboard reduces the time, effort, and cost IRRI and its partners incur when carrying out their mission to achieve food security.
It also serves as another platform that agricultural and social science researchers, academe, and policy-makers can use for consultations, research, and education.
