Amazon AWS Certified Generative AI Developer - Professional AIP-C01 Practice Test 45

Question 440

A response shows citations but support rate is falling. Which telemetry control best identifies the failure?

Count hyperlinks in generated answers and call the proportion grounded accuracy.
Run grounded-answer quality evaluation on a stable verified question set.
Use only retrieval similarity scores to assess whether generation faithfully used evidence.
Change the benchmark questions every release and compare raw aggregate support rates.

Explanation

Correct Answer

Evaluate whether each cited passage entails the corresponding answer claim on a stable verified reference set; citation presence alone is insufficient.

Incorrect Answers

A valid URL can support a different claim.

Search ranking does not test statement entailment.

Uncontrolled test population changes confound the trend.

Question 441

A model-error rate alert should cause a tested rollback rather than merely populate a dashboard. Select TWO controls: a material error threshold and an accountable rehearsed rollback procedure.

Detect material model-error rate threshold.
Trigger a pager on any isolated user error without an agreed severity threshold.
Route rollback to the currently failing model version because its endpoint remains online.
Route alert to a rehearsed rollback/owner-response procedure.
Close the incident automatically when the FM claims the tool problem is fixed, with no measured recovery.

Explanation

Correct Answer

Alarm on a tested material error-rate threshold so release failure is detected promptly.

Tie the alert to an owned, rehearsed rollback using a verified previous model/prompt artifact, with final-state checks.

Incorrect Answers

Noisy alarms may cause unnecessary rollback and alert fatigue.

A rollback requires an actually validated prior artifact.

A model's narration is not operational recovery evidence.

Question 442

An audit team cannot attribute a business change requested through a model and agent tool. Select THREE correlated records linking the model request, authenticated actor/tool and actual downstream modification.

Store only the generated explanation for a database mutation, without tool execution records.
Use a shared service account for all users with no original actor identity.
Correlate model invocation/request ID.
Assume HTTP request acceptance means the database transaction committed successfully.
Identify authenticated actor and executed tool.
Bind to downstream authoritative modification receipt.

Explanation

Correct Answer

Correlate the actual model request ID with the workflow trace.

Bind the verified actor and tool name/arguments that actually executed.

Record the authoritative downstream modification receipt/status to prove the business side effect.

Incorrect Answers

Model text cannot prove which write occurred.

A service principal alone cannot attribute requested modifications.

A queued or timed-out call may have unknown final state.

Question 443

A performance dashboard aggregates free-tier and provisioned traffic. Which telemetry control best identifies the failure?

Split metrics by deployment category for valid capacity comparisons.
Compare one blended requests-per-second average across reserved and on-demand models.
Use total user count as the only denominator for provisioned-throughput saturation.
Rename the tier labels while leaving the underlying metrics combined.

Explanation

Correct Answer

Split usage, cost, error and latency metrics by supported on-demand/provisioned deployment and actual capacity, rather than combining unlike service tiers.

Incorrect Answers

Different capacity economics and quota behavior make the aggregate misleading.

Token demand and capacity allocations matter more than users.

Label changes do not correct an invalid comparison.

Question 444

An ingestion system drops document metadata but index availability stays green. What observability change gives the team actionable diagnosis?

Report green because the vector index responds to pings despite missing author/version fields.
Use only the number of stored vectors as proof all required metadata were preserved.
Alert on field-quality/lineage failures not just vector-store uptime.
Mask missing metadata in generated citations instead of diagnosing parser/index mapping failure.

Explanation

Correct Answer

Validate metadata field coverage, source lineage and ingestion success separately from index availability, alarming when source attributes disappear.

Incorrect Answers

Availability does not ensure indexing fidelity.

The same count may contain incomplete records.

Output formatting cannot restore lost provenance.

Question 445

A vector shard failure produces partial query results. Which monitoring signal or correlation is missing?

Use the surviving shards' top hit as proof the missing data do not exist.
Detect degraded search coverage and fail closed or respond with uncertainty.
Increase model temperature to compensate for missing index partitions.
Hide shard errors from the application so the user sees a normal complete-search result.

Explanation

Correct Answer

Detect shard degradation and incomplete retrieval coverage; fail closed or explicitly mark uncertainty instead of treating partial hits as complete evidence.

Incorrect Answers

Partial index availability cannot establish absence of matching documents.

Generation randomness cannot recover inaccessible source evidence.

Concealing degraded coverage misrepresents retrieval confidence.

Question 446

A model evaluation uses only questions from its prompt examples. Which evaluation design most reliably measures the actual task risk?

Evaluate only the examples already embedded in the system prompt.
Change the benchmark questions after each failure until the model passes.
Build a held-out representative dataset for unbiased answer quality measurement.
Measure response fluency rather than agreement with validated source answers.

Explanation

Correct Answer

Use a held-out representative dataset not reused as prompt demonstrations, with labeled expected outcomes and separate tuning/test subsets.

Incorrect Answers

The model can memorize or be conditioned on those examples, inflating scores.

Adaptive test selection biases acceptance.

Readable output does not establish correct task performance.

Question 447

A RAG chatbot is scored only on fluent responses. Which evidence is needed for an accurate comparison?

Count how often a reply includes links and treat that as factual correctness.
Use output grammar quality as the sole RAG benchmark.
Measure grounded factual correctness and citation support against source evidence.
Evaluate retrieval similarity alone without inspecting the final generated statements.

Explanation

Correct Answer

Score claim-level groundedness and citation entailment against approved source passages, separately from fluency and relevance.

Incorrect Answers

A citation can be unrelated or contradict a claim.

Fluent hallucinations can pass grammar tests.

Relevant passages do not guarantee the model used them correctly.

Question 448

A field extractor often omits mandatory values even though its reported exact-match score looks high. Select TWO field-level tests that distinguish incorrect extracted values from missing required values.

Use only whole-record exact match on the easiest rows and ignore missing fields.
Measure just JSON parse rate, not whether each field matches source data.
Discard hard-to-read document types to raise the extraction benchmark score.
Measure per-field precision and false extracted values.
Measure per-field recall and missing mandatory fields.

Explanation

Correct Answer

Per-field precision exposes wrong populated values/false extractions against labeled records.

Per-field recall and completeness expose missing mandatory attributes even when returned fields are accurate.

Incorrect Answers

A single summary can hide which required fields fail.

Syntactic success does not prove extraction correctness.

Excluding failure cases misrepresents intended workload performance.

Question 449

A model produces inconsistent answers to paraphrased questions. Which evaluation design most reliably measures the actual task risk?

Run semantic consistency tests while tolerating benign wording variation.
Require byte-for-byte identical prose for every paraphrase as the only consistency metric.
Compare outputs from changing models and prompts without recording versions and sampling settings.
Ignore contradictory numeric facts when the replies share similar sentence structure.

Explanation

Correct Answer

Evaluate the meaning of replies to equivalent paraphrases using a fixed reference task, allowing benign wording changes but flagging factual contradictions.

Incorrect Answers

Different wording can be semantically equivalent.

Uncontrolled changes confound diagnosis of inconsistency.

Stylistic similarity cannot prove consistent business answers.

Question 450

An FM judge approves all outputs from a similar model. Which evidence is needed for an accurate comparison?

Use one judge from the same model family as its own final approver with no calibration.
Treat a 10/10 judge score as proof of human approval when no reviewer participated.
Calibrate judge against human-labeled reference cases and monitor disagreement.
Remove disagreement cases from the calibration set so judge accuracy appears higher.

Explanation

Correct Answer

Calibrate FM-judge decisions on human-labeled accepted/rejected examples and monitor disagreement, especially with a related model family.

Incorrect Answers

Correlated failure modes can produce optimistic scores.

An automated score is not independent human evaluation.

Selective omission conceals the important error distribution.

Pass Your Amazon AWS Certified Generative AI Developer - Professional AIP-C01 Exam Easy!

Question 440

Explanation

Correct Answer

Incorrect Answers

Question 441

Explanation

Correct Answer

Incorrect Answers

Question 442

Explanation

Correct Answer

Incorrect Answers

Question 443

Explanation

Correct Answer

Incorrect Answers

Question 444

Explanation

Correct Answer

Incorrect Answers

Question 445

Explanation

Correct Answer

Incorrect Answers

Question 446

Explanation

Correct Answer

Incorrect Answers

Question 447

Explanation

Correct Answer

Incorrect Answers

Question 448

Explanation

Correct Answer

Incorrect Answers

Question 449

Explanation

Correct Answer

Incorrect Answers

Question 450

Explanation

Correct Answer

Incorrect Answers

AWS Certified Generative AI Developer - Professional AIP-C01 Practice Tests