Quiz: HPC AI observability and security

Reading mode

EngineerDecision-maker

Ten questions to check what you retain from the course. Each answer is corrected immediately, with an explanation and a link to the relevant lesson. Nothing is sent: your best score stays in this browser.

1. Question 1 of 10Inside a GPU node, which link do the GPUs use to talk to each other?One answer

   • InfiniBand or RoCE
   • NVLink
   • The Lustre parallel file system
   • The Slurm scheduler

   Check

2. Question 2 of 10Which of these signal families belong to the four families to collect on an HPC AI cluster?Several answers possible

   • GPU: SM utilization, HBM bandwidth, ECC errors
   • Billing: hourly cost of each node
   • Model: loss, gradient norm, step time, AllReduce latency
   • Storage: Lustre throughput, metadata latency

   Check

3. Question 3 of 10Bridge: businessIn the lesson's illustrative example (constructed values, not a measurement), a GPU showing 87% utilization computes usefully only about 19% of the step. What does GPU utilization actually measure?One answer

   • The share of time the tensor cores spend on matrix multiplications
   • Power draw relative to maximum power
   • Whether the GPU has something in flight, including waiting on a collective communication

   Check

4. Question 4 of 10To spot a straggler slowing down the whole training run, which indicators does the lesson recommend measuring?Several answers possible

   • Step time per rank
   • The step breakdown into data load, compute and communication
   • Only the cluster's average GPU utilization
   • SM occupancy and tensor core utilization

   Check

5. Question 5 of 10Bridge: organizationHow do you concretely tie each GPU metric to a job, a user and an account, the basis for chargeback?One answer

   • With the hpc_job label that dcgm-exporter adds from mapping files written by the Slurm prolog and epilog, then linking the job to the user and account through sacct or a Slurm exporter
   • By joining the GPU UUID label of DCGM metrics with a Slurm metric carrying the same UUID
   • By asking each team to report its GPU hours in a spreadsheet
   • By installing a proprietary agent from the GPU vendor that sends data to its cloud
   • By reserving a whole node for each account

   Check

6. Question 6 of 10You want to trace calls to cuMemAlloc_v2 in the libcuda library with bpftrace. Which kind of probe should you use?One answer

   • A kprobe
   • A system call tracepoint
   • None: you have to modify the PyTorch code to instrument it
   • A uprobe

   Check

7. Question 7 of 10Which statements about Falco and Tetragon are correct?Several answers possible

   • Falco detects and alerts in real time on suspicious behavior, such as unexpected connections or modified binaries.
   • Both require instrumenting the training code.
   • Tetragon can block an unauthorized operation, such as a CUDA ioctl, before it completes.

   Check

8. Question 8 of 10RDMA traffic over InfiniBand bypasses the kernel. Where can visibility into this traffic come from?One answer

   • iptables rules set on each node
   • Adapter and subnet manager counters, with a per-job traffic baseline
   • eBPF probes attached to tcp_connect

   Check

9. Question 9 of 10Bridge: business linesYour cluster is shared between several customers. Which countermeasure does the lesson give against a job reading weights left in GPU memory by the previous job?One answer

   • Encrypting checkpoints on Lustre
   • Isolating the network with InfiniBand partitions (P_Key)
   • Explicitly wiping memory or resetting the GPU in the prolog or epilog and, as needed, isolating customers with MIG or confidential computing mode

   Check

10. Question 10 of 10Bridge: peopleAccording to the course, why should security and operations teams work together on an HPC AI cluster?One answer

    • Because security relies on the same stack as observability: same collectors, same storage, same dashboards
    • Because the law requires a single team for both functions
    • Because conventional security tools are enough to monitor RDMA traffic

    Check

Start over

Going further

• Course overview
• Understanding HPC
• Glossary
• Bridges

BridgesWhat this page changes beyond the technical team.

BusinessValue, cost, risk

You can tell reported GPU utilization from useful compute, and so argue hardware purchases on real efficiency.

OrganizationWho decides, who owns

You know how to tie each metric to a job, a user and an account, which underpins chargeback and answers to auditors.

PeopleTeams, on-call, skills

You see why security and operations gain from sharing one stack and working together.

Business linesProduct, finance, risk, customer

You know how to protect each customer's weights and data on shared infrastructure, and how to show research teams where their training time goes.

Why bridges matter →