Agricompas

Agricultural Data Analytics: Knowledge and decision support

agricompas

Data Analytics Make Sense

by

By Simon Dzurjak, Data Analyst at Agricompas

With the advent of the big data era, the way we gather insights about processes affecting our everyday lives has changed dramatically, and the domain of agriculture and food production is no different.

As a result of the widespread availability of various sensors, from satellites and UAVs, to climate and soil measurement hardware on farms, we can now gather data at unparalleled rates and volumes. However, in its raw state, the data we collect often provides little, if any, insight. These raw measurements need to be processed and analysed through complex pipelines to be turned into a valuable product.

The engine responsible for driving the change of raw data into worthwhile insights is data analytics.

Data Analysis for EcoProMIS

Since joining Agricompas in July, I have been working on ensuring that the EcoProMIS data which is stored and arrives on our cloud platform, is as high quality as possible. Analytics and predictions are only ever as good as the data from which they are sourced, hence rigorous checking of data quality is always a necessary layer in data intensive projects. To help with this, I wrote a number of programs generating data quality statistics, which will ensure that any analytics we produce will be reliable and trustworthy.

The statistics produced by these scripts are then visualised and fed into internal dashboards, which provide a quick and intuitive way of keeping an eye on our data and making sure that it looks the way we expect it to.

Data analysis is an essential part of our work on the EcoProMIS project.

Eddy Covariance Data

So far, one of my most enjoyable (but challenging) experiences was working with large eddy covariance tower datasets. Eddy covariance towers are brilliant at logging and producing rich atmospheric chemistry flux datasets allowing us to understand the minute gas exchange processes in agricultural fields. This data is important for our crop modellers, allowing them to produce various models capable of predicting crop phenology and yields.

However, due to the way that the hardware transmits the data, it is common for gaps in data to form (e.g. as a result of signal loss). Luckily, with the power of statistics, we can accurately discern the missing values and fill the gaps in the data. Working closely with the crop modelling team, after many discussions and much research, we have successfully automated this process, and allowed ourselves to further iterate and improve on it if need be in the future.

Integrating Diverse Data

As for the future, I also look to my past and hope to bring some of my previous experience in working with remote sensing and Earth observation data into our platform. Integrating and tying all the data types together will be a big challenge, however I am certain that it will bear fruit and truly create a platform which will make farming easier, more precise and more sustainable.

With advances in computational power and the high quality data available to us, I believe that data analytics and artificial intelligence will play a key role in revolutionising agriculture, and it is rewarding to be a part of a team making this happen. Combining multiple sources of data together into a single product will allow us to create a platform capable of supporting decisions via accurate insights and improving the ease with which farmers grow crops like never before.

How can Value Chain Analysis be used to make cocoa farming more sustainable?

by

A smallholder farmer in Colombia cut opens a cocoa pod with a machete. Each pod contains around 20 to 30 seeds. (photo credit, FEDECACAO)

By Deborah Foy, Agricompas

Over the past few months, the team at Agricompas has been working closely with FEDECACAO in Colombia to conduct value chain analysis for cocoa. This blog introduces the concept of value chain analysis and highlights how value chains can be used to make cocoa farming more sustainable.

What is Value Chain Analysis?

At its heart, value chain analysis describes the pathway of cocoa from bean to bar. It extends all the way from pre-farm input providers right through to end consumers, linking different actors in the chain along the way and highlighting where there are both threats and opportunities.

Typically, the main segments of a value chain are summarised in a value chain map. This map becomes a graphical representation of the value chain that showcases the functions of stakeholders, their relationships and the agents supporting the process. A simplified value chain map for cocoa is shown below:

The Main Stages of a Cocoa Value Chain

A typical cocoa bean value chain involves the operation of four major segments – Primary Production, Commercialisation, Processing and Manufacturing, and Market.

1. Primary Production

Farmers are at the primary production stage of cocoa-based supply chains. In Colombia, this group encompasses over 52,000 cocoa smallholders. Whilst the area planted by the farmers varies between regions, a typical cocoa farm in Colombia is grown on an average plot size of less than three hectares. Farmers in the primary production stage can be classified in many ways, including:

  • Farm management practices (e.g. diversification of production)
  • Percentage of shade cover
  • Whether they are certified
  • Whether their production methods are traditional or there is a high degree of technification

2. Commercialisation

Commercialisation can include individual purchasers, producer associations such as our local partner, the Federación Nacional de Cacaoteros (FEDECACAO), and the purchasing agents of large firms. This stage typically includes transportation, quality control, fermenting and drying, as well as differentiation (e.g. certification of producers that have met criteria of an independent evaluating entity).  Some agents active in the commercialisation of cocoa – including our partner FEDECACAO – are involved in technical assistance and in the distribution of farming inputs such as fertiliser.

3. Processing and Manufacturing

In this stage, cocoa moves from commercialisation to firms involved in the transformation of cocoa and in the production of cocoa-based products (e.g. chocolate bars and cosmetics). In Colombia, two large domestic companies – Casa Luker and Nutresa – dominate the processing and manufacturing of cocoa. Together, they purchase between 80-90% of Colombia’s cacao production. Most central cocoa traders in Colombia have commercial relationships with one or other of these two major chocolate companies.

4. Markets

In this last stage, the processed cocoa is sold for internal (domestic) consumption or sent to export destinations. Colombia is unusual in that 81% of the final product is sold domestically, largely for drinking chocolate preparations (FEDECACAO 2019). However, exports of Colombia cocoa have been increasing since 2012. The country currently exports fermented and dried cocoa beans, shells and cacao residues, cacao paste, cacao butter, cacao fat or cacao oil, cacao powder without the addition of sugar and other food preparations containing cacao to around 72 countries.

Colombian exports of cocoa beans 2008-2018 in metric tons (FEDECACAO, 2019)

How is Value Chain Analysis Useful for Sustainable Agriculture?

Value chain mapping has many uses. Key among them are the following:

  • Identifying points of economic inefficiency: This includes economic analysis of points in the chain where resources are not being used in an optimal way.
  • Identifying the value added at each step of the chain: In cocoa, value added at each step of the value chain is typically unequally distributed – meaning that benefits from price increases tend to benefit the agroindustry more than smallholder farmers.
  • Identifying smallholder farmers who adopt practices that lead to sustainable agricultural outcomes: This can improve productivity through the creation of price premiums and incentive structures. The value chain can also help large firms to promote climate smart agriculture at primary production stage (e.g. through the reduction of greenhouse gas emissions).
  • Identifying how value chain links can be improved: Better value chain links (e.g. between producer associations and buyers) can also improve farmer access to extension services, information about prices and certification, as well as inputs for climate adaptation and resiliency.

In Colombia, there are multiple forces that shape the supply and demand of cocoa. At Agricompas, we’re using value chain analysis tools to map out the roles and responsibilities of different actors involved in the sector. We’re using these tools to gain insight into the net income derived by farmers and other stakeholders from cocoa production and trade. Ultimately, understanding how the value chain works will help us to identify ways to make it more sustainable.

Why the eddy covariance technique is an ally in the search of sustainable agriculture

by

By Agricompas Crop Model Team

In recent decades, agriculture has been under the scrutiny of society and the scientific community due to the negative impact that it generates on the environment. These impacts are of many types, including deforestation, eutrophication of water bodies, the reduction of biodiversity due to the intense use of pesticides, and the emission of greenhouse gases (GHG).

In relation to GHG emissions, these are released in the process of manufacturing inputs, such as, fertilizers. Also included are the GHGs released as a result of the transport process: first of inputs towards the production areas, and then of the product towards the consumption areas.

An eddy covariance system recording greenhouse gases emissions on a commercial rice field at Colombia for EcoProMIS project. (Agricompas)

A Complicated Task

But the most complicated task from a methodological point of view is to determine the GHGs that are released during the production stage. Among the GHGs released to the atmosphere during the production phase, the most important are carbon dioxide, methane (in systems where the soil is in anaerobic conditions), nitrous oxide, and ammonia.

Methodological difficulties are associated with the fact that these emissions are determined by dynamic factors such as climate, soil characteristics, and management practices, especially fertilization and irrigation.

Since it is impossible to survive without agriculture, efforts have focused on developing and implementing production systems able to maximize yields while reducing negative effects on the environment. A prerequisite for advancing in this direction is to measure the GHGs generated during agricultural production cycles.

Static Chambers

To understand better the methodological challenges involved in determining these gases under field conditions, let us take methane as an example. This gas is generated as a product of the decomposition of organic matter in the soil under non-oxygen conditions, typical of crops such as flooded rice.

Traditionally, static dark chambers have been used to collect samples that are later analysed by the gas chromatography technique in specialized laboratories.

This technique has a high sensitivity to determine low methane fluxes, is easy to handle, and has a low cost. But its main disadvantages are related to the low spatial representativeness and the inability to generate data at different time scales.

In other words, the measurements only represent the gas flux in a small area and at a specific time point, which leads to the question: can this technique generate data to represent what happens in inherently heterogeneous and dynamic agricultural systems?

Eddy Covariance

It is in this context that the technique of eddy covariance appears, as an alternative way to measure, among other variables, methane flows with greater spatial and temporal representativeness.

This technique employs a complex assembly of sensors arranged in a tower (which is why they are usually called eddy covariance towers) that records variables that ultimately allow the determination of the exchange of gases and energy between the crop and the atmosphere.

Although the foundations of the technique and data processing are complex, it provides useful information in the search for more sustainable agricultural systems.

This is because, in addition to determining GHG emissions, such as methane and carbon dioxide, the eddy covariance technique also provides information about the flow of energy between the soil, the plant, and the atmosphere. This means that information is also useful to improve the water use efficiency since the measurements allow the determination of water fluxes from the crops to the atmosphere (evapotranspiration).

All of this information is comparable in terms of accuracy with data obtained by reference instruments such as lysimeters. Therefore, the technique of eddy covariance is currently a powerful ally in the search for more sustainable agricultural systems.

Use with EcoProMIS

The EcoProMIS project has four eddy covariance towers in Colombia, two recording data on rice crops, and two on oil palm crops. The data collected by these stations are being processed to calibrate crop models that allow, in addition to predicting yields, to estimate GHG emissions.

Together with our partners (CIAT, Cenipalma, Fedearroz, IWCO, Pixalytics and Solidaridad), the final objective of the project is to generate “knowledge and decision support” to orient stakeholders towards sustainability.

EcoProMIS: a strategic alliance with Colombian rice and oil palm farmers and their federations

by

Libardo Ochoa García, Colombia Project Officer, Agricompas

The Ecological Production Management Information System, EcoProMIS, is a project implemented in Colombia and co-financed by the United Kingdom Space Agency (UKSA).

Agricompas has been developing the EcoProMIS idea over the last four years, with the support and collaboration of Colombian institutions Cenipalma, Fedearroz, CIAT and Solidaridad, and the companies IWCO and Pixalytics in the UK.

Collecting Farm Data
At the end of the EcoProMIS project, in 2022, a “platform” will be created that collects, with the active support of growers, agronomic field data and correlates it with meteorological and environmental data, as well as with satellite and drone earth observation data.

EcoProMIS processes this rice and oil palm production data in near real-time crop production knowledge with the help of crop models and algorithms (mathematical information processes).

Knowledge to Support Farmers
The above agronomic knowledge is combined with environmental, market and socio-economic knowledge, to generate practical information for decision-making in rice and oil palm crops.

Colombian rice and oil palm farmers will receive crop production knowledge to support their decision making at no cost, as long as they share and upload their crop data on the EcoProMIS platform. For each farmer and each plot, detailed supportive information will be generated.

A Strategic Alliance
The knowledge services that EcoProMIS will provide to growers are developed in close collaboration with our partners; the rice and oil palm growers and their federations. EcoProMIS activities include workshops in Casanare, Meta, Tolima and Magdalena wherein growers are interviewed and trained in data collection, concepts and ideas are shared, and partners participate in the development and testing of new knowledge services.

Knowledge Services
The EcoProMIS team are currently developing the first ‘knowledge services’ for rice and oil palm. These services predict yield and calculate water demand in both crops. Yield prediction is important for the grower to understand if the crop is performing well and if this is not the case to investigate limiting growth factors.

Further it helps the farmer with planning the harvest and processing logistics. Regarding crop water requirement; the farmers will be able to establish in time how much additional water is required in irrigated and rainfed systems per crop cycle for rice and per year for oil palm. Water is fast becoming a scarce commodity.

Support with Drought
A practical example of applying EcoProMIS knowledge in daily life by Colombian farmers can be illustrated with recent events during the first half of 2020. Rainfall was scarce and the average annual rainfall is expected to be below the historical average. The questions arise, how will water shortage affect the rice and oil palm crops? And how much water is available for agriculture or industry?

With the EcoProMIS platform, farmers will have access to near real-time knowledge about crop water needs vs. the expected rainfall and thus be able to make better management decisions. This will help farmers to decide whether or not to invest in or use irrigation. In addition the grower could use the information of water shortage to justify a yield loss claim in case of insurance against drought.

A Joint Sustainable Future
After the project is finished by mid 2022 Agricompas will commercialise the platform as the new independent knowledge creator and broker in the crop value chain. We will provide “Knowledge for Free” to our grower and federation partners and “Decisions for a Fee” to major value chain players such as input and equipment providers, insurers and banks, and processors and traders. Our ultimate goal is to empower and support growers and increase their productivity and profits while reducing the environmental impact and improve the socio-economic conditions of all stakeholders.

“Las cuentas, claras, y el chocolate, espeso”

by

By Annie Zamora, Principal Researcher, FEDECACAO

Anyone who has travelled to Colombia may have come across this local saying, which roughly translates as “I like my stories clear and my chocolate, thick.” It highlights the Colombian people’s preference for clarity and blunt talking. The saying also hints at our country’s passion for chocolate – we consume 50,000 tonnes of cacao annually, which is around a kilo per person.

But unlike Europeans, mostly we drink it rather than eat it. Drinking chocolate has a long tradition across the country. We typically start the day with a cup of hot chocolate for breakfast. In the afternoon, we have a tradition called “tomar las onces” (elevenses) which involves drinking chocolate with a snack like an arepa or an almojabana. If you come to the main office of FEDECACAO here in Bogotá in the morning, you can enjoy a delicious hot chocolate with bread.

A typical morning in FEDECACAO

At FEDECACAO, the Federación Nacional de Cacaoteros, we’re passionate about cacao, but even more passionate our improving the lives of cacao smallholder farmers.

Growing cacao in tropical environments can be difficult. Our farmers must constantly be on the lookout for pests and disease, they must fertilise the soil around the trees and – given most cacao grows near the equator – they must shield their cacao crops from the burning sun. Producing cacao pods also takes time; a cacao tree normally doesn’t reach maturity until it’s at least 3 or 4 years old. Our cacao farmers also live in remote areas with many social issues – many of them been badly affected by the decades-long armed conflict here in Colombia.

Despite these challenges, FEDECACAO has continued to provide support to cacao growers over the country. Since 2013, we have seen production increasing, and the lives of our smallholders have begun to improve. But there’s still much more work to be done.

That’s why FEDECACAO is excited to be part of the “Digitising of cacao production” project, together with our partner organisations in Colombia and the UK. Helping our farmers get the resources, knowledge and skills they need to ensure the highest yields will hopefully start to transform the sector and ensure a sustainable supply of cocoa for chocolate lovers, both in Colombia and around the world.

Hatching the proof of concept for sustainable agriculture

by

Roelof Kramer, CEO of Agricompas and Project Lead of EcoProMIS

The global corona crisis has a severe impact on our daily lives. National lockdowns are restricting travel and preventing us from seeing family and friends. It is scary and depressing how control of our daily lives has vanished without warning.

That’s why I bought ten fertilised chicken eggs on the internet and placed them in an incubator. Always good to have something to look forward to! Three weeks later my children hatched five healthy chicks which are now under a heat lamp provided with ample feed and water.

But wait! Did we really do that properly? In the middle of the night my wife had to check if the chicks were not too hot or cold, hungry or thirsty. Of course, they were all fine but “better safe than sorry” especially when growing crops or raising livestock as we do in agriculture.

The challenge of sustainability
For decades now our planet has had to feed increasingly more people while its resources are more and more strained. As a result, we are faced with deforestation, biodiversity loss, people displacement and soil & air pollution.

Although we see examples daily, little is done to create more sustainable systems as economic interests compete severely with social and environmental interests which are difficult to express in financial metrics.

On the one hand it is not in the short-term interest of politicians nor corporates to sacrifice public electability or shareholder profits while on the other hand NGOs, philanthropists or the angered public lack the resources, organisation and reach.

As a result, we create partial and disjointed solutions that reduce problems rather than solving them in a whole system approach.

The idea behind EcoProMIS
In 2016 Agricompas hatched the concept of an Ecological Production Management Information System or “EcoProMIS”. It is a system that can create valuable solutions while combining the competing interests of the different stakeholders in one commercially viable business model.

EcoProMIS is a data analytics platform that processes comprehensive crop data as detailed as per-field-in-near-real-time. It provides “Knowledge for Free” to growers & partners and “Decisions for a Fee” to stakeholders across the entire crop value chain.

These services aim to improve the efficiency of economic, technical, social and environmental processes.

Government support and funding
A project proposal was successfully submitted to the UK Space Agency in 2017 for four million pounds of funding in the International Partnership Program (IPP2).

EcoProMIS received this funding in order to improve the productivity and profitability of rice and oil palm growers in Colombia while minimising environmental impact and improving socio-economic conditions of the crops’ stakeholders.

From February 2018 onwards Agricompas and six partners including Pixalytics (UK), Cenipalma, CIAT, Fedearroz, IWCO and Solidaridad (Colombia) have been executing the 50-month project in Colombia with a delivery date by end March 2022.

Thereafter Agricompas will continue to operate the “commercial” EcoProMIS platform for the benefit of its users – partners such as the 16,000 rice growers, 5,000 palm growers and their federations and customers such as the input and & equipment manufacturers, traders & processors, banks & insurers, wholesalers & distributors, and governments & NGOs.

Taking action
During the last two years we have been installing professional equipment to collect weather and greenhouse gas data at four different locations across Colombia. We are building cloud-based IT architecture and applications to process data with machine learning and Artificial Intelligence into knowledge.

We are developing completely new offerings and testing these with prospective customers in a solid business model, while aware that this is a massively complex and ambitious project.

During the next two years we will continue to refine and optimise data collection, deepen the data processing, and test our offerings and business model.

Communicating our work
We want to share the exciting and ambitious work that we are doing. That is why, from now on each of the project partners will write a series of regular blogs about their organisation, activities, experiences and objectives within the EcoProMIS project.

Over the past two years, we have attended numerous expos and agricultural forums to share about our work.

Roelof Kramer, the EcoProMIS project lead and Agricompas CEO, in the British Embassy stand at the Agrofuturo 2019 exhibition in Medellin, Colombia

Hope for a better future
Yes, the world is going through a rough time and it is very depressing that even with our good intentions sustainable agriculture currently doesn’t exist. Across the world, we lack the comprehensive and detailed data collection, and the processing of data into knowledge and viable business models to offer services that satisfy all crop system stakeholders.

EcoProMIS is more than “just another project” that finishes when the funding runs out. It is a solid promise that is being developed by motivated partners into a whole system approach that creates true sustainable agriculture: a system that supports poor and rich growers, unites commercial and non-commercial parties, and supports NGOs and governments.

That is something really fantastic to look forward to! I wish you and your family health, hope and happiness.