For a long time, supply chain specialists have been dependent on traditional forecasting models and techniques to estimate demand for their goods and services. Both the digital transformation and the fourth industrial revolution –industry 4.0– are advancing rapidly. This has a fundamental influence on the way human beings live and work. The introduction of digital identifiers and the internet of things (IoT) produced a huge amount of data. One of the main impacts of this situation is that supply chain management decision-making is increasingly driven by data instead of experience. However, the traditional strategies and methods used in the different parts of the supply chain impose many constraints that prevent gaining full advantage of data.
At aioneers, we create a lot of dashboards and do lots of machine learning tasks. Some of those analyses are done regularly. To deliver these dashboards and machine learning, we need to do a lot of data transformation, like cleaning data and joining tables. To automate data transformation, we use Databricks clusters to run Python and, sometimes, R scripts. Databricks allows us to automatically run data transformation scripts on schedule, but it is missing one essential feature: email notification on how the data was loaded.
On a weekly basis, we exchange large amounts of data with our clients. This can for instance be information on different SKUs, such as how long they have been in stock, how they are selling, and so on.
For our customers, it may be easier to receive this data in Excel, as excel is a tool a lot of people are familiar with. Most of us already have it installed on our computer, which means no additional solutions or licenses have to be bought. A benefit Excel has over CSV is that it has a user interface and you do not need to spend a lot time developing. On top of this, we can also use different styles of formatting, colors and visualizations in Excel, so that it looks better and makes the data easier to understand for our clients.