Filter by topic (137)
  • Observability & tracing (OpenTelemetry GenAI semantics)

    How OpenTelemetry GenAI spans capture prompts, completions, tokens, cost and tool calls in a trace, and why plain logs cannot explain a bad answer.

    Evaluation & quality · AI EngineeringUpdated 3 min read

  • Open vs closed, hosted vs self-hosted models

    Compares closed and open-weight models, hosted and self-hosted, so you can pick by quality, data control and cost.

    Model foundations · AI EngineeringUpdated 2 min read

  • Orchestration Frameworks

    Shows what an agent orchestration framework runs for you, where it differs from a fixed workflow, and when a plain loop is enough.

    Agents & orchestration · AI EngineeringUpdated 2 min read

  • PII Redaction / Prompt DLP

    Explains how to detect and tokenize personal data and secrets in prompts and outputs before they reach an external LLM.

    Safety & governance · AI EngineeringUpdated 2 min read

  • Planning & Reasoning Patterns

    Compares chain-of-thought, ReAct and tree-of-thought prompting, with the cost of each and when a model needs none.

    Agents & orchestration · AI EngineeringUpdated 2 min read

  • Prompt / version registries

    Describes how a prompt registry stores versions, labels and eval scores so prompts can be fetched, tested and rolled back.

    LLMOps · AI EngineeringUpdated 2 min read

  • Prompt engineering (zero/few-shot)

    Explains zero-shot and few-shot prompting, why examples steer output without retraining, and when fine-tuning becomes worth it.

    Prompting & context · AI EngineeringUpdated 2 min read

  • Prompt templates & prompt management

    Covers parameterized prompt templates, safe interpolation, versioning and eval-driven testing, and why inline f-strings fall short.

    Prompting & context · AI EngineeringUpdated 2 min read

  • Provider key & secret management

    Covers storing, scoping, rotating and monitoring LLM provider API keys, with a gateway and spend limits to contain a leak.

    LLMOps · AI EngineeringUpdated 2 min read

  • Provider-Level Prompt Caching

    Explains how provider prompt caching reuses the KV cache for a stable prefix, what invalidates it and when it pays off.

    Emerging · AI EngineeringUpdated 2 min read

  • Quantization, distillation & model-size tradeoffs

    Compares quantization and distillation for shrinking models, what each costs in quality, and how to match model size to the task.

    Model foundations · AI EngineeringUpdated 2 min read

  • RAG (Retrieval-Augmented Generation)

    RAG retrieves matching document chunks and places them in the prompt; this article walks the pipeline and where grounding fails.

    Retrieval & RAG · AI EngineeringUpdated 3 min read

  • Re-ranking

    A cross-encoder re-ranker re-scores the retriever's top candidates so the best passages reach the context window; here is when it pays off.

    Retrieval & RAG · AI EngineeringUpdated 2 min read

  • Real-time data & live context

    Live context fetches current data at request time and injects it into the prompt; this covers the flow, cache TTLs and stale-data failures.

    Serving & infrastructure · AI EngineeringUpdated 2 min read

  • Response caching (exact & semantic)

    Exact caching keys on the literal prompt and semantic caching on meaning; this compares both, plus prompt caching, and the false-hit risk.

    Serving & infrastructure · AI EngineeringUpdated 2 min read

  • RLHF & DPO (preference tuning)

    RLHF trains a reward model then runs PPO, while DPO optimizes preferences directly; this compares both and the risk of narrow preference data.

    Fine-tuning · AI EngineeringUpdated 3 min read

  • Sampling, temperature & decoding

    Temperature, top_p and top_k control how a model picks each next token; this explains the three settings and which values suit which tasks.

    Model foundations · AI EngineeringUpdated 2 min read

  • Semantic Caching

    A semantic cache returns a stored answer when a new query embeds close to a past one; here are the questions teams ask about thresholds and fit.

    Retrieval & RAG · AI EngineeringUpdated 2 min read

  • Small / On-Device Language Models (SLMs)

    Small language models of roughly 1 to 14B parameters run locally when quantized; this covers fit, trade-offs and the data-privacy benefit.

    Emerging · AI EngineeringUpdated 2 min read

  • Streaming responses (token streaming / SSE)

    Token streaming sends model output over a long-lived SSE connection as it is generated; this covers transports, client duties and when to avoid it.

    Serving & infrastructure · AI EngineeringUpdated 2 min read

  • Structured Error Handling for Model Calls

    An LLM call fails in distinct ways, from 429 rate limits to a 200 response with bad output; this sets out how to route each failure class.

    Reliability · AI EngineeringUpdated 2 min read

  • Structured output / JSON mode / function schemas

    Learn how schema-constrained decoding and function calling force valid JSON, and why you still validate the values.

    Prompting & context · AI EngineeringUpdated 2 min read

  • Synthetic data generation

    Understand how synthetic training data is generated and filtered, and why unfiltered output can degrade the model you train.

    Data & pipelines · AI EngineeringUpdated 2 min read

  • System vs user vs developer roles

    See how system, developer and user roles rank instructions in a chat API, and why the boundary is a soft one you must back with code.

    Prompting & context · AI EngineeringUpdated 2 min read