Data Engineer | Streaming and Production AI

I build reliable data and AI platforms that scale.

4+ years delivering measurable impact across enterprise systems: Kafka pipelines at 800,000 events/day, analytics acceleration from 5 hours to 55 minutes, and production AI latency improvements from 45s to 18s.

Chiheb Mhamdi, Data Engineer
Based in Nürnberg Open to collaboration

0

Kafka events processed in pilot production lines

0

Analytics report runtime reduced on BigQuery/Dataflow

0

Data lake ingestion acceleration with Airflow pipelines

0

P95 chat latency reduction in production RAG workloads

Positioning

I help organizations move from slow, fragile data workflows to production-grade platforms with strong observability, predictable latency, and clear operational runbooks.

Experience

Data Engineer — Bayerisches Landesamt fur Steuern (LFST)

10/2025 – Present · Nurnberg

  • Diagnosed PostgreSQL connection-pool saturation (393 of 500 connections in use, about 120 idle per API pod), corrected pool limits, and documented a repeatable validation workflow.
  • Tuned Celery/Redis async processing, reducing p95 chat response from about 45s to about 18s during peak office traffic.
  • Hardened OpenShift deployments (route/nginx timeouts, Celery Beat access, readiness probes, resource limits) and wrote runbooks for deploy, rollback, and on-call handoff.
  • Wrote architecture diagrams and internal guides on incident response, RAG retrieval behavior, and LiteLLM/vLLM gateway routing.

Data Engineer — Ancud IT

12/2021 – 09/2025 · Nurnberg, Germany

Selected clients: Continental AG, Valeo, Fraunhofer, Maisel, HKN

  • Built Kafka anomaly-detection pipelines on AWS (VPC, Fargate, ECR) for Continental AG at about 800,000 events/day; alerts in about three minutes and about 14% less unplanned downtime over six months on two pilot lines.
  • Reduced Maisel SharePoint-to-data-lake ingestion from about 3.5h to about 28m with nine Airflow DAGs, 62 data-quality tests, and Prometheus/Grafana on a three-node Kubernetes/MinIO cluster (Hetzner).
  • Built a hybrid LLM platform on Kubernetes for 45 Maisel users (Ollama, OpenWebUI, LLM proxy), cutting public-API inference spend by about 35% (about €1,200/month) while meeting EU data-residency.
  • Redesigned Valeo BigQuery schemas and Dataflow ingestion at about 22GB/day, cutting a plant analytics report from about 5h to about 55m.
  • Automated HKN email infrastructure for 280 mailboxes (Docker, SOAP/XML, GitHub Actions CI/CD) with nightly archiving of about 40GB/month.
  • Delivered a Fraunhofer reinforcement-learning traffic pipeline (SUMO, Kafka, Protobuf, MLflow, Kubeflow): 68 experiments and about 11% lower mean vehicle wait time on the simulation benchmark.

Case Studies

Real-time Anomaly Detection on AWS

Kafka + AWS Fargate pipelines for industrial sensor streams across pilot production lines, with anomaly alerts in about three minutes.

Impact: about 14% less unplanned downtime.

Data Lake Acceleration with Airflow

Nine Airflow DAGs, Python ETL, and observability stack on Kubernetes/Hetzner for reliable ingestion and quality checks.

Impact: 3.5h pipeline reduced to 28m.

Hybrid LLM Platform for Internal Teams

Routed inference between on-prem and cloud models using proxy architecture to satisfy EU data-residency and cost constraints.

Impact: about 35% monthly inference savings.

BigQuery/Dataflow Performance Redesign

Reworked Valeo schemas and Dataflow ingestion for plant analytics at about 22GB/day with predictable runtime.

Impact: report runtime cut from 5h to 55m.

Mailbox Platform Automation

Containerized an exchange stack for HKN with SOAP/XML integration and GitHub Actions CI/CD for 280 mailboxes.

Impact: nightly archive of about 40GB/month.

RL Traffic-Control Simulation Pipeline

Kafka + Protobuf aggregation with MLflow/Kubeflow tracking for a Fraunhofer SUMO traffic-control benchmark (not live city traffic).

Impact: 68 experiments; about 11% lower mean wait time.

How I Help Teams

Streaming Platform Engineering

Build and stabilize Kafka-based pipelines with strong delivery guarantees, partition strategy, and observability for production throughput.

Best for: real-time telemetry, anomaly detection, event platforms.

Data Platform Acceleration

Reduce slow batch workflows by redesigning ingestion topology, schema layout, and orchestration for predictable SLA performance.

Best for: BI/reporting bottlenecks and late data incidents.

Production AI Reliability

Improve RAG and LLM systems with practical latency tuning, queue/caching safeguards, and deployment runbooks for safer releases.

Best for: internal AI assistants and enterprise AI platforms.

Featured Public Work

Event-driven-app

Practical repository that explains event-driven architecture patterns in modern application systems.

View on GitHub

airflow-kubernetes-test

Hands-on deployment/testing repository focused on running Apache Airflow workloads in Kubernetes environments.

View on GitHub

data_enginnering_project_1

Beginner-friendly data engineering project showcasing practical Python-based workflow design and data pipeline thinking.

View on GitHub

Publications

I publish practical engineering deep dives on streaming, production AI, reliability, and system design. Explore the full animated publications page.

Learn

Four rooms. The lexicon is the vocabulary. The interview lab is the conversation. The quiz checks whether the concept stuck. Talks are the decks — summary first, then a big slide carousel.

Certifications

Verified credentials — and the field notes behind them. Pick a cert to open how I prepared, what skills stuck, and how it shows up in real work.

Loading field notes…

Education

  • Master in Computer Science, Esprit Higher School of Engineering and Technology (2020 - 2022)
  • Bachelor in Computer Science, Esprit Higher School of Engineering and Technology (2017 - 2020)

Languages

  • Arabic (Mother tongue)
  • French (Bilingual)
  • English (Excellent)
  • German (Intermediate B1, preparing B2)

Open channel

Let's build data systems that hold under load.

Available for data engineering and production AI work focused on performance, resilience, and operational clarity.

Based in Nürnberg, Germany · CET · Open to conversations