This repo demonstrates the development of a real-time data pipeline designed to ingest, process, and analyze stock market data. Using cutting-edge tools like Apache Kafka, PostgreSQL, and Python, the pipeline captures stock data in real-time and stores it in a robust data architecture, enabling timely analysis and insights.

create a streaming application as a simple real-time fraud detection system backed by Apache Kafka using a Python client.

Apache Kafka for python distributed streaming and processing systems

A streaming pipeline for a data lakehouse ( Kafka / MongoDB / Apache Drill)

Gregor is a Python program that produces fake data into an Apache Kafka topic for demo purposes