Data gathering, ETL or ELT?
IT Tips & Insights from our team members
The famous phrase “Data is the new oil” from Clive Humby explain about the importance of data but figured out about the challenges involved in get data.
On crazy sea with of high and strong waves there are some brave guys, going through a lot of deep miles inside of sea looking for oil.
From data side the challenge is something like this. Getting data once is easy but get data always and keep them available, is hard to do and it that involves technology and process.
Nowadays there are various kinds of data format (txt, csv, parquet, Jason etc) that came from many kinds of data base flavors and with big volume that can arrive all day long or once a day.
This text summarizes the two process to get data ETL and ELT. Get data involves an ecosystem that need to be security, scalable, fail tolerant. Data arrives all time from everywhere.
What is ETL – Extract Transform and Load?
ETL is the process that involves connect on data source, once connected to data source
- Extract the data to a “staging” area,
- Once data is in staging are, it is transformed. Transform means change column format, change column name, do some calculations and in some cases, grouping etc
- When that data is read, it is loaded and become available for users.
What is ELT – Extract Load and Transform?
ELT is the process that involves connect on data source (like ETL ), once connected to data source
- Extract data,
- Load that data without any transformation,
- The transformation process occurs after all data is loaded by the user.
Getting data process is essential. This process brings data and keep it available to be used for all involved with data, Data Scientists, Data Analysts, Data Product and all stakeholders using data to making decision.
About Rogerio Alves
Rogerio is a husband, father and a learner musician who believe, data can help companies and people to stay resilient and happy.