by Carlos Torres, Senior Consultant, IWCO
Information Workers, or IWCO, is an important technical partner working on the EcoProMIS project led by Agricompas.
A Colombian company with 10 years of experience, IWCO is focused on helping its clients get the most and the best value from their data by turning it into information that allows them to make decisions in the short, medium, and long term.
This value comes from four fundamental areas in the process of data harnessing: exploration, extraction, refinement, and consumption of data.
Data Exploration
This phase is designed to understand the types of data, their characteristics, where they are located and their potential value, additionally, the identification of business needs that can be solved with data.
Data Extraction
During the extraction process, we help our clients (such as the EcoProMIS project) to take their data, put it into a structured format that can be simple to use and that enables our clients to immediately create on-demand queries to get answers to their business questions.
Data Refinement
Once the data is in a structured format, we help our clients design and create models to predict, understand, and even extract hidden patterns in the data.
Data Consumption
Business users such as Agricompas and Pixalytics are extremely important, that is why we care about helping them consume their data models – information that can now be prepared for analysis. Through knowledge transfer processes we enable our clients to obtain answers to their strategic questions.
Information Workers is a Microsoft, AWS (Amazon), and Google Partner. With broad experience and a process focused on culture, we have an emphasis on helping organisations to operate in the world of self-service.
We have a multidisciplinary team that includes Mathematicians, Statisticians, Economists, Software Engineers, Systems Engineers, and even Petroleum Engineers (!). This diversity means that our team is able to bring our customers’ different professional perspectives, something that has helped us contribute value in the EcoProMIS consortium of seven international partners.
Together with Agricompas, we created a plan designed to resolve some of the challenges for the EcoProMIS project, namely how we acquire, store, manage, and secure the agricultural analytics data.
After developing this plan, it was essential to choose a platform aligned with these necessities pragmatically, and allow us to obtain results in a short period. Another important step was to identify the sources of big data for the project, for example publicly available data (e.g. earth observation images from UKSA), streaming data from different sensors like weather stations, and data from legacy databases.
Big Data
As we engaged with all of this data, it became obvious that we could not process this multivariate data using traditional methods. The data from the EcoProMIS project is clearly in the realm of big data, complying with the three ‘V’s:
Volume: EcoProMIS has collected a lot of data from different sources.
Velocity: The various data streams are handled within different timeframes from the different sensors.
Variety: All types of formats – from structured to unstructured.
Adding Value
Another key step in our contribution to the project was to find a way to access, manage, and store the data. We needed to find a data platform that can support the storage and the capabilities of analysing petabyte-size files and trillions of objects.
Equally important was prioritising security and data protection, making sure the database is GDPR-compliant, and that data is securely stored. Linked to this is our role in providing data auditing and ongoing support.
Agro-Industry
Big data in the agro-industry plays an important role. With such a large amount of information out there, the data needs to be shaped or tested in a way that adds value to the agro-industry.
By doing so, the agro-industry can better identify problems and reach the goal of sustainable optimisation that is at the heart of the EcoProMIS project.