Quiz: VictoriaMetrics for LLMs

Reading mode

EngineerDecision-maker

Ten questions to check what you remember from the course. Each answer is marked immediately, with an explanation and a link to the relevant lesson. Nothing is sent anywhere: your best score stays in this browser.

1. Question 1 of 10Where does VictoriaMetrics in Single mode accept OTLP metrics directly?One answer

   • On a dedicated port 4318.
   • On the /opentelemetry/v1/metrics path of its usual HTTP port, 8428.
   • Nowhere: you must always go through an OTel Collector.
   • On gRPC port 4317.

   Check

2. Question 2 of 10Which practices does the guide recommend to avoid a cardinality explosion?Several answers possible

   • Never label metrics with trace_id, request_id or session_id.
   • Add tenant_id to latency metrics to compare customers.
   • Beyond 1,000 tenants, use tenant_tier on high-frequency metrics and keep tenant_id for cost metrics.
   • Attach run_id as a label to the real-time metrics of an evaluation pipeline.

   Check

3. Question 3 of 10Bridge: businessThe instrumentation decorator computes cost from a per-token price grid. Why does the guide ask you to version and date that grid?One answer

   • Because VictoriaMetrics rejects cost metrics without an effective date.
   • To reduce the cardinality of llm_cost_usd_total.
   • Because prices change, and an undated grid makes historical cost series impossible to audit.

   Check

4. Question 4 of 10The first version of this guide used anomaly_score(...) in a MetricsQL query to detect RAG index drift. What is actually the case?One answer

   • anomaly_score is a metric written by vmanomaly (Enterprise); with the open source version, you compute a z-score against a 7-day baseline.
   • It is a native MetricsQL function available in the open source version.
   • It is a Grafana function applied at display time.

   Check

5. Question 5 of 10Why does the guide recommend recording rules for cost metrics?One answer

   • To extend the retention of cost series.
   • To precompute aggregations such as increase(llm_cost_usd_total[24h]) in the background, which dashboards would otherwise recompute on every refresh.
   • To replace vmalert alerting rules.
   • To get a finer resolution than the raw data.

   Check

6. Question 6 of 10Your availability SLO is 99.5% over 30 days. What is the corresponding error budget?One answer

   • 0.05%, about 22 minutes per month.
   • 0.5%, about 36 minutes per month.
   • 0.5%, that is 3 h 36 min of tolerated downtime per month.
   • 5%, about 36 hours per month.

   Check

7. Question 7 of 10The guide pairs the LLMSlowBurn alert, evaluated over 6 hours, with a second condition over 30 minutes. What is this short window for?One answer

   • To fire the alert before the long window confirms the trend.
   • To reduce the cardinality of the error series.
   • To compute the monthly error budget.
   • To let the alert clear quickly once the problem is fixed.

   Check

8. Question 8 of 10Bridge: peopleWhich practices presented in the course help protect the on-call team from LLM alerts?Several answers possible

   • Attach an action and a runbook link (runbook_url) to each alert, as for RAGIndexDrift.
   • Route alerts by team with Alertmanager, for example budget alerts to the billing team.
   • Send every alert, warnings included, to PagerDuty.
   • Review alert thresholds with the teams after go-live to spot false positives.

   Check

9. Question 9 of 10Bridge: organizationYour accounting obligations require keeping cost metrics for three years, while GPU metrics only need three months. With open source VictoriaMetrics, which implementation does the guide recommend?One answer

   • Set a different -retentionPeriod for each metric in the same instance.
   • Use -retentionFilter on the open source instance.
   • Run several instances, each with its own -retentionPeriod, and route series to them with vmagent.
   • Keep costs in OTel traces, which are retained longer.

   Check

10. Question 10 of 10Bridge: business linesLeadership and finance want to follow AI spending. What does the course provide for them?One answer

    • Admin access to Grafana so they can build their own views.
    • Read-only Grafana users, with SSO, on the same data as operations.
    • A copy of the data in a separate database reserved for finance.
    • Direct access to the VictoriaMetrics write API.

    Check

Start over

Going further

• Course overview
• Site glossary
• Bridges between the planes
• Labs

BridgesWhat this page changes beyond the technical team.

BusinessValue, cost, risk

You can make AI costs auditable: a dated price grid, recording rules and long retention for cost metrics.

OrganizationWho decides, who owns

You can treat retention and the single node versus cluster choice as governance decisions, within the limits of the open source version.

PeopleTeams, on-call, skills

You can design alerts that protect on-call: runbook attached, routing by team, thresholds reviewed with the teams.

Business linesProduct, finance, risk, customer

You can give leadership and finance read-only dashboards, built on the same data as operations.

Why bridges matter →