At aioneers, we create a lot of dashboards and do lots of machine learning tasks. Some of those analyses are done regularly. To deliver these dashboards and machine learning, we need to do a lot of data transformation, like cleaning data and joining tables. To automate data transformation, we use Databricks clusters to run Python and, sometimes, R scripts. Databricks allows us to automatically run data transformation scripts on schedule, but it is missing one essential feature: email notification on how the data was loaded.
On a weekly basis, we exchange large amounts of data with our clients. This can for instance be information on different SKUs, such as how long they have been in stock, how they are selling, and so on.
For our customers, it may be easier to receive this data in Excel, as excel is a tool a lot of people are familiar with. Most of us already have it installed on our computer, which means no additional solutions or licenses have to be bought. A benefit Excel has over CSV is that it has a user interface and you do not need to spend a lot time developing. On top of this, we can also use different styles of formatting, colors and visualizations in Excel, so that it looks better and makes the data easier to understand for our clients.