ETL Example—Extraction
Explore how to build a simple ETL pipeline using Bash shell scripting. Learn to extract raw lottery data from a CSV file online, identify data issues, and prepare for transformation and loading to a PostgreSQL database. This lesson emphasizes choosing practical tools and handling external data sources efficiently.
We'll cover the following...
We'll cover the following...
ETL pipelines should only be as complicated as they need to be. It's easy to get swept away by the latest industry trends and feel overwhelmed with new software and tools. When building an ETL pipeline, we’ll need to choose the most appropriate tool for each step. But not all pipelines have to be complex.
To demonstrate this, let’s build an entire ETL pipeline from ...