LLM Gateway & Conversation Intelligence

Overview

LLM Gateway & Conversation Intelligence is a cost-optimized LLM platform built around an LLM gateway with model routing, semantic caching and cost tracking. It generates summaries, action items, sentiment analysis and quality scores from conversations.

Objectives

• Self-host and benchmark open-source models.
• Implement routing, caching and cost tracking.
• Build the conversation analysis pipeline.

Features

• LLM gateway with model routing by task and cost.
• Semantic cache for reusing similar answers.
• Per-request cost and usage tracking.
• Conversation summaries and action items.
• Sentiment analysis and quality scoring.

Technologies and profile

vLLM / Ollama, Qdrant, Redis and FastAPI. Profile: AI / Backend engineering. Type: LLM Platform / AI. One to two interns are planned.

Application documents

Attach a CV in PDF format. A short cover letter, GitHub or portfolio link and university internship agreement may also be provided.