YouTube Data Pipeline with Recommendation System
This project is built to help students with self-study. It allows users to create their own courses by adding study videos and playlists from YouTube. While watching a video, the platform providespractice questions to test their knowledge. The project includes a recommendation model that suggests videos based on the user's interests. •Developed a Django-based web application that recommends YouTube videos and practice questions using the gemini API. •Implemented a real-time recommendation engine using PySpark ALS and PostgreSQL. •Integrated PostgreSQL for efficient data storage and retrieval of generated questions. •Deployed the application on AWS and containerized services using Docker.
Tools & Technologies
- Python
- Django
- gemini API
- Apache spark (pyspark)
- Apache Airflow
- Youtube API
- Postgresql
- AWS