A comprehensive ELT pipeline for analyzing passenger satisfaction data. Features a modern data architecture with Apache Airflow for extraction, dbt/Snowflake for transformation, Python/Pandas for cleaning, and interactive dashboards for visualization with NextJS.
The Skytrax Global Airlines Analytics Project is a comprehensive initiative designed to transform customer-review data for airlines into actionable insights. It focuses on collecting, processing, and visualizing reviews to provide analysis on passenger satisfaction, operational performance, and competitive dynamics among airlines. Utilizing a modern tech stack, this project aims to create a robust analytical framework that benefits various stakeholders, focusing on aggregating feedback while understanding the inherent biases in self-reported reviews.
This end-to-end analytics system integrates powerful data processing and storage technologies, making it a valuable asset for airlines looking to better understand customer sentiment. The project is built on cloud services and employs best practices in software development to ensure scalability and reliability in the insights generated.
Data Cleaning: The project includes a dedicated module to clean and standardize raw scraped data, ensuring consistent formats and enhancing data quality for analysis.
ETL Process: An efficient ETL (Extract, Transform, Load) process is implemented using Airflow, facilitating seamless data scraping from Skytrax, staging it in AWS S3, and loading it into Snowflake.
Data Transformation: Uses dbt (data build tool) for complex data transformations on Snowflake with continuous integration and delivery workflows via GitHub Actions, ensuring timely and accurate data updates.
Insightful Dashboard: A user-friendly dashboard website visualizes insights derived from the processed airline reviews, enabling easy access to key metrics for stakeholders.
Customer Satisfaction Analysis: Specific dashboards, like the Spirit Airlines dashboard, focus on in-depth analysis of customer satisfaction, providing targeted insights for airline management.
Collaborative Team Structure: The project is backed by a multidisciplinary team, ensuring that diverse expertise is applied to data analysis, engineering, and software development for a holistic approach to analytics.
Cloud-Based Architecture: The use of AWS and other cloud technologies ensures that the system is scalable and can handle the high volume of data generated by airline reviews effectively.
