A generative AI application in cross-section: data, infrastructure, model, orchestration, application. Open each layer to see what it does, what can go wrong and what to watch, then follow a request through all five.
For: engineers and SREs · architects · team managers · business and productPrerequisites: None.
Reading mode
Decision-maker reading: bridges first.
Here is a made-up scenario. A health insurer launches an assistant that answers its members’ questions. Three weeks later, customer service reports wrong answers about reimbursements. Yet the dashboard is green: the assistant answers quickly, the servers are running, no errors show up. Where do you start?
To find out, you need to see what the application is made of. Think of it as a five-tier cake. At the bottom sits the data: the documents the assistant searches for its answers. Above it is the compute infrastructure, meaning the servers and graphics cards (GPUs) that run the model. Then comes the model itself, which writes the answer. Next is orchestration, which prepares the model’s work: the instructions it is given (prompts), the document search and the agents that chain several actions together. At the top is the application the user actually sees.
Any layer can degrade quietly and any layer can be attacked. The diagram below cuts the cake open. Click a layer to read its card, switch from Observe mode (what to watch) to Risks mode (what can go wrong), then send a request to follow its path from top to bottom and back.
Drag the stack to rotate it, then click a layer (or press Enter) to open its card. Layers go from data, at the bottom, to the application, at the top.
Data
Freshness and coverage of sources
Input distribution against the baseline
Personal data detected in logs
Data drift: user questions change
LLM04Data and model poisoning
Stale or incomplete sources
Infrastructure
Time to first token (TTFT)
Tokens per second, queue, GPU memory
Cost per request
Saturation and queues
LLM10Unbounded consumption: runaway cost
A busy GPU says nothing about quality
Model
Quality score from automated evaluation
Gap against a golden dataset
Exact model version on every call
LLM09Hallucinations and misinformation
Concept drift: the world changes, the model does not
Silent update at the provider
Orchestration
End-to-end trace of every request (OpenTelemetry)
Relevance of retrieved documents
Tokens and tool calls per request
LLM01Prompt injection
Pipeline drift: chunking or template changed silently
LLM06Agent with excessive agency
Application
User feedback and drop-offs
Content policy violations
Personal data detected in outputs
LLM02Sensitive information disclosure in outputs
Guardrail bypass
Loss of trust before any complaint
What to observe, layer by layer
What can go wrong, layer by layer
Application
User feedback and drop-offs
Content policy violations
Personal data detected in outputs
OWASP LLM02Sensitive information disclosure in outputs
Guardrail bypass
Loss of trust before any complaint
Orchestration
End-to-end trace of every request (OpenTelemetry)
Relevance of retrieved documents
Tokens and tool calls per request
OWASP LLM01Prompt injection
Pipeline drift: chunking or template changed silently
OWASP LLM06Agent with excessive agency
Model
Quality score from automated evaluation
Gap against a golden dataset
Exact model version on every call
OWASP LLM09Hallucinations and misinformation
Concept drift: the world changes, the model does not
Silent update at the provider
Infrastructure
Time to first token (TTFT)
Tokens per second, queue, GPU memory
Cost per request
Saturation and queues
OWASP LLM10Unbounded consumption: runaway cost
A busy GPU says nothing about quality
Data
Freshness and coverage of sources
Input distribution against the baseline
Personal data detected in logs
Data drift: user questions change
OWASP LLM04Data and model poisoning
Stale or incomplete sources
Layer 1 of 5
Data
What it is
The fuel: internal documents, knowledge bases, training and RAG data.
What can go wrong
Data drift: user questions change
Data and model poisoning OWASP LLM04
Stale or incomplete sources
What to observe
Freshness and coverage of sources
Input distribution against the baseline
Personal data detected in logs
Layer 2 of 5
Infrastructure
What it is
The compute: GPUs, inference servers, cloud or cluster.
What can go wrong
Saturation and queues
Unbounded consumption: runaway cost OWASP LLM10
A busy GPU says nothing about quality
What to observe
Time to first token (TTFT)
Tokens per second, queue, GPU memory
Cost per request
Layer 3 of 5
Model
What it is
It predicts the most likely next word; it does not check facts.
What can go wrong
Hallucinations and misinformation OWASP LLM09
Concept drift: the world changes, the model does not
Silent update at the provider
What to observe
Quality score from automated evaluation
Gap against a golden dataset
Exact model version on every call
Layer 4 of 5
Orchestration
What it is
The glue: prompts, document retrieval (RAG), agents and tool calls.
What can go wrong
Prompt injection OWASP LLM01
Pipeline drift: chunking or template changed silently
Agent with excessive agency OWASP LLM06
What to observe
End-to-end trace of every request (OpenTelemetry)
Relevance of retrieved documents
Tokens and tool calls per request
Layer 5 of 5
Application
What it is
What the user sees: assistant, chatbot, business feature.
What can go wrong
Sensitive information disclosure in outputs OWASP LLM02
Open the Model layer. A language model predicts the most likely next word without checking facts. Its quality therefore shows neither in response time nor in GPU utilization. It is tracked with a score from an automated evaluation, which checks for instance that the answer sticks to the documents. In the insurer’s case, that score is what would have raised the alarm.
Switch to Risks mode and spot the OWASP IDs, the reference list of flaws specific to this kind of application. Prompt injection (LLM01), which slips hidden instructions into a question or a document, targets orchestration. Sensitive information disclosure (LLM02) happens in the application. Poisoning (LLM04), which corrupts the source data, targets the bottom layer.
Send a request. It crosses all five layers on the way down and back up. That recorded journey is called a trace: like parcel tracking, it shows where the request went and how long it stayed at each stop. The trace is what ties the signals of the five layers together and tells you where to look.
The same cross-section in 2 min 21, layer by layer, with a request crossing the whole stack.
Read the video text
Your dashboard is green. Yet your AI assistant has been answering wrong for weeks. An AI does not crash: it degrades silently. A generative AI application is a stack of five layers. Each one can drift, each one can be attacked. Let us open it, layer by layer, starting from the bottom.
Layer 1: Data. The fuel: internal documents, knowledge bases, training and RAG data.
What can go wrong: Data drift: user questions change ; Data and model poisoning (OWASP LLM04) ; Stale or incomplete sources.
What to observe: Freshness and coverage of sources ; Input distribution against the baseline ; Personal data detected in logs.
Layer 2: Infrastructure. The compute: GPUs, inference servers, cloud or cluster.
What can go wrong: Saturation and queues ; Unbounded consumption: runaway cost (OWASP LLM10) ; A busy GPU says nothing about quality.
What to observe: Time to first token (TTFT) ; Tokens per second, queue, GPU memory ; Cost per request.
Layer 3: Model. It predicts the most likely next word; it does not check facts.
What can go wrong: Hallucinations and misinformation (OWASP LLM09) ; Concept drift: the world changes, the model does not ; Silent update at the provider.
What to observe: Quality score from automated evaluation ; Gap against a golden dataset ; Exact model version on every call.
Layer 4: Orchestration. The glue: prompts, document retrieval (RAG), agents and tool calls.
What can go wrong: Prompt injection (OWASP LLM01) ; Pipeline drift: chunking or template changed silently ; Agent with excessive agency (OWASP LLM06).
What to observe: End-to-end trace of every request (OpenTelemetry) ; Relevance of retrieved documents ; Tokens and tool calls per request.
Layer 5: Application. What the user sees: assistant, chatbot, business feature.
What can go wrong: Sensitive information disclosure in outputs (OWASP LLM02) ; Guardrail bypass ; Loss of trust before any complaint.
What to observe: User feedback and drop-offs ; Content policy violations ; Personal data detected in outputs.
A request crosses all five layers, there and back. The trace ties their signals together: it tells you where to look.
Observing AI, from data to answer: www.meantimetolearn.com.
Once you can name the five layers, the next question is what to watch at each one. The guide Understanding generative AI observability covers the signals, the system types and the maturity levels.