Data Engineer | Streaming and Production AI

I build reliable data and AI platforms that scale.

4+ years delivering measurable impact across enterprise systems: Kafka pipelines at 800,000 events/day, analytics acceleration from 5 hours to 55 minutes, and production AI latency improvements from 45s to 18s.

Chiheb Mhamdi, Data Engineer
Based in Nürnberg Open to collaboration

0

Kafka events processed in pilot production lines

0

Analytics report runtime reduced on BigQuery/Dataflow

0

Data lake ingestion acceleration with Airflow pipelines

0

P95 chat latency reduction in production RAG workloads

Positioning

I help organizations move from slow, fragile data workflows to production-grade platforms with strong observability, predictable latency, and clear operational runbooks.

Experience

Data Engineer — Bayerisches Landesamt fur Steuern (LFST)

10/2025 – Present · Nurnberg

  • Diagnosed PostgreSQL pool saturation and corrected limits to stabilize API behavior under concurrent usage.
  • Tuned Celery/Redis async processing, reducing p95 chat response from about 45s to about 18s during peak office traffic.
  • Hardened OpenShift deployments with timeout tuning, readiness probes, and runbooks for deploy/rollback/on-call handoff.

Data Engineer — Ancud IT

12/2021 – 09/2025 · Nurnberg, Germany

Selected clients: Continental AG, Valeo, Fraunhofer, Maisel, HKN

  • Built Kafka anomaly-detection pipelines on AWS processing about 800,000 events/day; contributed to about 14% downtime reduction over six months in pilot lines.
  • Reduced SharePoint-to-data-lake ingestion from about 3.5h to about 28m with Airflow DAGs, quality tests, and Prometheus/Grafana monitoring.
  • Designed a hybrid LLM platform (on-prem + cloud routing) for 45 users, reducing public API inference spend by about 35%.
  • Redesigned BigQuery/Dataflow pipelines handling about 22GB/day, cutting a recurring analytics report from about 5h to about 55m.

Case Studies

Real-time Anomaly Detection on AWS

Kafka + AWS Fargate pipelines for industrial sensor streams across pilot production lines, with anomaly alerts in about three minutes.

Impact: about 14% less unplanned downtime.

Data Lake Acceleration with Airflow

Nine Airflow DAGs, Python ETL, and observability stack on Kubernetes/Hetzner for reliable ingestion and quality checks.

Impact: 3.5h pipeline reduced to 28m.

Hybrid LLM Platform for Internal Teams

Routed inference between on-prem and cloud models using proxy architecture to satisfy EU data-residency and cost constraints.

Impact: about 35% monthly inference savings.

BigQuery/Dataflow Performance Redesign

Reworked schema and ingestion flow for analytics workloads with predictable runtime and better throughput.

Impact: report runtime cut from 5h to 55m.

How I Help Teams

Streaming Platform Engineering

Build and stabilize Kafka-based pipelines with strong delivery guarantees, partition strategy, and observability for production throughput.

Best for: real-time telemetry, anomaly detection, event platforms.

Data Platform Acceleration

Reduce slow batch workflows by redesigning ingestion topology, schema layout, and orchestration for predictable SLA performance.

Best for: BI/reporting bottlenecks and late data incidents.

Production AI Reliability

Improve RAG and LLM systems with practical latency tuning, queue/caching safeguards, and deployment runbooks for safer releases.

Best for: internal AI assistants and enterprise AI platforms.

Featured Public Work

Event-driven-app

Practical repository that explains event-driven architecture patterns in modern application systems.

View on GitHub

airflow-kubernetes-test

Hands-on deployment/testing repository focused on running Apache Airflow workloads in Kubernetes environments.

View on GitHub

data_enginnering_project_1

Beginner-friendly data engineering project showcasing practical Python-based workflow design and data pipeline thinking.

View on GitHub

Publications

I publish practical engineering deep dives on streaming, production AI, reliability, and system design. Explore the full animated publications page.

Certifications

Verified credentials — and the field notes behind them. Pick a cert to open how I prepared, what skills stuck, and how it shows up in real work.

Loading field notes…

Education

  • Master in Computer Science, Esprit Higher School of Engineering and Technology (2020 - 2022)
  • Bachelor in Computer Science, Esprit Higher School of Engineering and Technology (2017 - 2020)

Languages

  • Arabic (Mother tongue)
  • French (Bilingual)
  • English (Excellent)
  • German (Intermediate B1, preparing B2)

Open channel

Let's build data systems that hold under load.

Available for data engineering and production AI work focused on performance, resilience, and operational clarity.

Based in Nürnberg, Germany · CET · Open to conversations