Agricompas

Agricultural Data Analytics: Knowledge and decision support

Blog

Automatic satellite image processing for farm intelligence

by

By Simon Dzurjak, Data Analyst

Satellites orbiting the Earth allow us to gather insights about farms and the crops they are producing at unprecedented spatial and temporal resolutions. This is one of the key features of Agricompas’ approach to digitising agricultural value chains and providing value to the sector.

Remote Sensing and Big Data

Remote sensing and Earth observation have not been left behind in the current big data revolution, with a large and ever-growing number of both open and commercial datasets providing years, sometimes decades, of historical data, ready to explore using big data techniques.

At Agricompas, we are using imagery from satellites such as Sentinel-2 to monitor farms and develop models to predict the crop phenology, health and yields. This is in combination with ground-sourced data such as weather stations and soil tests as part of our EcoProMIS platform.

Processing Satellite Images

Whilst satellite imagery in its raw form is intriguing to look at, it does not tell us enough to be immediately helpful, therefore it must be refined further into various indices and biophysical parameters.

Crops are complex, living, plant organisms which interact with various wavelengths of the electromagnetic (EM) spectrum in a variety of ways. When monitoring crops using satellite imagery, we can use the various reflectance and absorption characteristics of plants in the visible and infra-red portion of the EM spectrum to our advantage and use mathematical methods to derive vegetation indices, each of which highlights a certain property of the crop.

Satellite image and analysis layer for palm farm in Colombia

The vegetation index data derived from satellite images can then be correlated with various biophysical parameters of the crops and insights about crop health and potential yields can then be modelled and monitored, giving farmers the insights they need.

Real-time Data for Farmers

Over the last few weeks, a large pipeline to mine valuable data from satellite images has been under development at Agricompas. The products derived by this pipeline will soon be used in machine learning models and crop modelling to develop accurate predictive models.

We are now in the final stages of putting finishing touches on the pipeline and testing it, ensuring that everything runs smoothly. Soon, the pipeline will be deployed to our cloud environment, where it will run automatically, and continually process the satellite images.

This will allow us to process an image as soon as it becomes available to us, letting us always have up to date and quick results about the farms we are monitoring.

Additionally, we are making sure that the latest imagery of the farms we are monitoring is available to the farmers via our knowledge services application.

We are extracting individual farm data from the large satellite images, so that the farmers can visualise the farms they are interested in, and do not have to manually locate their farm in the satellite image scene.

Practical Application

Farmers and other users will be able to visualise both true-colour imagery, as well as derived indices such as NDVI, giving a bird’s eye view of farms, highlighting issues such as unhealthy crops and allowing for quick responses before these issues turn into problems.

Automating the mining of the satellite imagery for valuable data has been a very exciting project to work on and brings a lot of additional value to the EcoProMIS platform. We are continuing to make sure that all of the data we work with is accurate, with advanced models producing reliable results, and supporting our users in undertaking the best decisions.

Cacao project in Colombia: final farmer workshops

by

By Carlos Perez, Solidaridad, and Annie Zamora, Fedecacao

The “Digitising of cocoa production in Colombia” project, funded by the UK Prosperity Fund, has been implemented since 2019 by Agricompas and a team of partners with the aim of generating a digital platform to support increased yields and environmental sustainability.

During February 2021 a select sample of cocoa farmers from Ríos de Chocolate association in Rionegro (part of the Santander region) and Producacao association in Andalucía (Valle del Cauca region) met as part of two final farmer workshops.

Results of our two-year project were shared and farmers were introduced to information about the market and value chain in Colombia and Europe.

Our sessions with the farmers started with an inspiring presentation by Galia Orme, owner of Choc Chick, an English chocolate company.

Listening to Galia Orme of ChocChick at one of the cocoa grower workshops

Galia shared with the farmers information about opportunities to access the UK chocolate market, the standards that are required for the UK market, and the most demanded products. In addition, she shared some of the technical qualities that are most prioritized, with special mention of the low-cadmium cocao produced in Colombia and valued by the European market. This was of encouragement to the Colombian farmers.

During these interactive sessions, Fedecacao and Agricompas also launched the value chain report that they had co-produced over the last year. An analysis of the sector, the report is an insightful and beautiful document, available in Spanish here.

The workshop included discussion around the value chain report and growers each received a copy of this publication. The report covers the dynamics of national and international cacao prices, characterisation of the production and consumption of Colombian cocoa, and statistics about the main destinations of the Colombian cacao exports.

Finally, the workshops included an update about the data collection technology and remote sensors used on the farms during during this innovative project and managed by project partner, the University of Lincoln.

A preliminary analysis was presented to growers including farm weather (temperature, rainfall) and production variables, designed to support growers to better understand their farms.

In addition to this information, Solidaridad shared with the farmers a plan to involve them with future data collection: Solidaridad will provide the selected cacao growers with a digital tool to continue collecting basic data on crops and weather conditions.

The “Digitisation of cocoa production in Colombia” project will conclude at the end of March 2021. It has been a successful project and contributed to identifying opportunities for the cacao sector in Colombia based on data collection and analysis.

Growers together after one of the workshops

The rice value chain: better knowledge, best decisions

by

By Nicolás Gonzalez, Business Development Manager at Agricompas

The last few months have been challenging, but also revealing for EcoProMIS and our approach to entering the market. While working hard to develop a comprehensive platform for data analytics, we are doing extensive market research with each of the stakeholders involved in the value chain for rice and agriculture in general.

EcoProMIS is a platform that collects and processes data into analytics from many different sources, providing a complete overview of the production chain. Establishing a link with each of the actors involved in the chain is fundamental. For this reason, our business and product development team work side by side with these stakeholders with the purpose of creating services that generate value and become the source of the company’s revenue.

For the market analysis, which is comprised of the study of both its size and the possible services to offer, a series of interviews and validations have been conducted with each of the stakeholders.

The main stakeholders we are engaging are drawn from the rice milling sector, agri-insurance, credit, seed, machinery, fertilizer and agrochemical suppliers. All of these have a key role in rice production and for which intelligent information is relevant in the decision-making of their business, improving competitiveness and optimising their processes.

Access to information is the critical factor

One of the common factors that the Agricompas team found in the interviews conducted with each of the key actors mentioned above, is that information for them is considered as their guide in daily operations, in the prediction of supply and demand, and in the management of decision making with respect to their business strategy.

EcoProMIS provides real-time data analysis tools for the agri-finance sector

Today there is no platform like EcoProMIS and the different actors mentioned above could see clearly the business advantage it would lend them: the possibility of knowing and predicting the yield of farmers’ production, seeing and mitigating risks for the credits that most of them offer, and the management of inventories and supply and demand.

Thanks to this market feedback, we have been able to generate valuable information products, taking our access to satellite images, artificial intelligence, climate information and data provided by farmers themselves, to generate differentiation in the market and value for platform users, making agriculture in Colombia more competitive through better decisions in crop management.

Best services, better conditions for farmers

With the development of the EcoProMIS platform, after market validation and the approval of rice farmers, we are creating a conducive environment for the innovation and growth in the agricultural ecosystem in Colombia.

The information for each of the actors in the chain translates into better products and services for farmers, because with this understanding of production, they are able to better identify their needs and their ability to pay and manage risks, all of which in turn translates into a considerable improvement in the structuring and costing of products.

Technology and access to information are part of the new generation of agriculture that is evolving worldwide. We believe that EcoProMIS is the platform responsible for bringing it to the life of each of the farmers who are part of the project and to the door of each of the companies that are part of the production chain.

EcoProMIS provides users not only with information, but with a valuable tool for daily operations, decision-making, and the way in which users innovate new products and services.

Agricompas and Information Workers working in a big data project for the Agro-Industry

by

by Carlos Torres, Senior Consultant, IWCO

Information Workers, or IWCO, is an important technical partner working on the EcoProMIS project led by Agricompas.

A Colombian company with 10 years of experience, IWCO is focused on helping its clients get the most and the best value from their data by turning it into information that allows them to make decisions in the short, medium, and long term.

This value comes from four fundamental areas in the process of data harnessing: exploration, extraction, refinement, and consumption of data.

Data Exploration
This phase is designed to understand the types of data, their characteristics, where they are located and their potential value, additionally, the identification of business needs that can be solved with data.

Data Extraction
During the extraction process, we help our clients (such as the EcoProMIS project) to take their data, put it into a structured format that can be simple to use and that enables our clients to immediately create on-demand queries to get answers to their business questions.

Data Refinement
Once the data is in a structured format, we help our clients design and create models to predict, understand, and even extract hidden patterns in the data.

Data Consumption
Business users such as Agricompas and Pixalytics are extremely important, that is why we care about helping them consume their data models – information that can now be prepared for analysis. Through knowledge transfer processes we enable our clients to obtain answers to their strategic questions.

Information Workers is a Microsoft, AWS (Amazon), and Google Partner. With broad experience and a process focused on culture, we have an emphasis on helping organisations to operate in the world of self-service.

Information Workers (IWCO) have a multidisciplinary team

We have a multidisciplinary team that includes Mathematicians, Statisticians, Economists, Software Engineers, Systems Engineers, and even Petroleum Engineers (!). This diversity means that our team is able to bring our customers’ different professional perspectives, something that has helped us contribute value in the EcoProMIS consortium of seven international partners.

Together with Agricompas, we created a plan designed to resolve some of the challenges for the EcoProMIS project, namely how we acquire, store, manage, and secure the agricultural analytics data.

After developing this plan, it was essential to choose a platform aligned with these necessities pragmatically, and allow us to obtain results in a short period. Another important step was to identify the sources of big data for the project, for example publicly available data (e.g. earth observation images from UKSA), streaming data from different sensors like weather stations, and data from legacy databases.

Big Data
As we engaged with all of this data, it became obvious that we could not process this multivariate data using traditional methods. The data from the EcoProMIS project is clearly in the realm of big data, complying with the three ‘V’s:

Volume: EcoProMIS has collected a lot of data from different sources.
Velocity: The various data streams are handled within different timeframes from the different sensors.
Variety: All types of formats – from structured to unstructured.

Adding Value
Another key step in our contribution to the project was to find a way to access, manage, and store the data. We needed to find a data platform that can support the storage and the capabilities of analysing petabyte-size files and trillions of objects.

Equally important was prioritising security and data protection, making sure the database is GDPR-compliant, and that data is securely stored. Linked to this is our role in providing data auditing and ongoing support.

Agro-Industry
Big data in the agro-industry plays an important role. With such a large amount of information out there, the data needs to be shaped or tested in a way that adds value to the agro-industry.

By doing so, the agro-industry can better identify problems and reach the goal of sustainable optimisation that is at the heart of the EcoProMIS project.

New year, new rice season

by

By Sam Adams, Head of Programmes

Over the last month or two the Agricompas science team have been making visits to rice fields in the Tolima region in Colombia. Travelling the five hours from Bogotá, the journey is a reminder of just how big this beautiful country is.

The two scientists are using these trips to better monitor, understand, and support the collection of accurate crop data. Inspections of soil and water took place, as well as looking at our cutting edge technology that collects weather data as well as greenhouse gas emissions.

These trips are in close collaboration with another member of the EcoProMIS project consortium, Fedearroz. Their technicians were on hand to demonstrate data collection techniques and to reflect on best practices and quality control measures.

Dr Gil and Dr Bojacá inspected the rice crop, which is just starting to germinate. This is the fifth cycle of rice crop data that the EcoProMIS project is collecting from both the Tolima and Casanare districts.

The rice seed in Tolima is just starting to germinate

In addition to these five cycles of data, we have access to a much larger historical database of rice farming, and our team is combining this information in order to create and strengthen our crop model. This model is thus becoming more reliable (‘robust’) as we add more data, including phenological, weather, and satellite imagery.

As we prepare to enter the fifth year of the project, it is exciting to see how a huge amount of collected data has now been  processed through our science and IT work into useful practical outputs that will directly benefit farmers.

Via mobile apps, these growers are able to access yield calculators and other tools to help them maximise their farm output and move towards our shared goal of sustainable optimisation.

Our science team will continue to make these regular visits as the project grows from strength to strength and we look towards rolling out this unique service to rice growers in Colombia and elsewhere.

Innovation, growth and data analytics

by

By Nicolás González, Business Development Manager

The world is changing. Every day we see how new technologies are being developed and many of us may have come to dread the idea of being displaced by a machine. But that is not the way of things. Technology is meant to make our lives easier, and we believe that the coexistence between traditional methods and newly developed ones is possible, if not meant to be.

In recent years, agricultural data analytics has become one of the top edge tendencies in terms of sustainable development globally. This means that there is a lot of research and a lot of projects currently trying to understand agricultural dynamics and how to use information in order to optimize processes and achieve sustainable objectives.

Although this is a very beautiful statement, the reality growers are facing on a daily basis, and how this information is to be gathered, processed and used, represents a huge challenge not only for farmers but for the whole agricultural value-chain.

Agricompas drones collect images of farms for innovative data analysis

Agriculture in LMIC countries still relies on manual labor. The culture of innovation, technology, insurance, good financial practices and data analytics is still in a juvenile stage.

Nevertheless, governments and private capital are incentivizing fast growth through technology and new services are becoming more common every day.

Innovation and Market Growth

For us as an agro data analytics company, innovation is the only way to break through such barriers and use our knowledge to evolve the growing market. This is our main drive, our oxygen and our compass. As Harvard Business Review’s article: Breaking down the barriers to innovation states:

To us, innovation doesn’t mean mere inventiveness. In our work we define it as: something different that creates value.

As we work with agricultural big-data, we face the challenge of gathering, processing and delivering useful information in the context of market needs and opportunities. By doing this, we look to make our EcoProMIS platform a value generator for growers, governments, and financial and insurance institutions.

Analytics for Decision Support

In order to capture what really drives the agricultural market in terms of financial services, risk management and productivity optimization, our innovation process aims to understand the market and develop tailored solutions that make the decision-making process more efficient, thus giving business intelligence the recognition and merit it deserves for the immense toolkit it provides us with.

We believe that by bringing sustainable agriculture into the digital era, better conditions for growers may be achieved. We believe that when real value is generated it can also be garnered.

We believe that corporate institutions can see a benefit as well, using high quality information and business intelligence, improving their margins, creating new delivery methods, enhancing R&D and, finally, increasing sales.

Our EcoProMIS platform sends farmers notifications using a suite of apps

With the richness of precise, accurate and relevant information, we enable an increase to the market size of agro insurance and provide the much-needed agronomic crop management data necessary for a new and creative product development ecosystem.

The world is changing. And so are we. Breaking through the barriers of convention, we have come to innovate and leave the world in better shape than how we found it. We believe that a new era for sustainable agriculture and analytics has come at last.