Sergey Nikonov

Sergey Nikonov

Recent Posts

Comparison of Email Notification Methods for Data Processing

analytics, data science

At aioneers, we create a lot of dashboards and do lots of machine learning tasks. Some of those analyses are done regularly. To deliver these dashboards and machine learning, we need to do a lot of data transformation, like cleaning data and joining tables. To automate data transformation, we use Databricks clusters to run Python and, sometimes, R scripts. Databricks allows us to automatically run data transformation scripts on schedule, but it is missing one essential feature: email notification on how the data was loaded.

Sergey Nikonov
By Sergey Nikonov

Automating Recalculation of Excel Files in Python And Databricks

On a weekly basis, we exchange large amounts of data with our clients. This can for instance be information on different SKUs, such as how long they have been in stock, how they are selling, and so on.  

 

For our customers, it may be easier to receive this data in Excel, as excel is a tool a lot of people are familiar with. Most of us already have it installed on our computer, which means no additional solutions or licenses have to be bought. A benefit Excel has over CSV is that it has a user interface and you do not need to spend a lot time developing. On top of this, we can also use different styles of formatting, colors and visualizations in Excel, so that it looks better and makes the data easier to understand for our clients 

 

Sergey Nikonov
By Sergey Nikonov
Supply Chain Brief