Amazon AWS Certified Generative AI Developer - Professional AIP-C01 Practice Test 48

Question 470

A monitoring report hides confidence intervals and sample size. Which evaluation design most reliably measures the actual task risk?

Publish a 99% success number without the number of evaluated cases.
Combine populations with different task difficulty into one denominator with no stratification.
Report valid denominators, uncertainty and tested population per metric.
Claim the reported proportion is exact true-world performance with zero uncertainty.

Explanation

Correct Answer

Report each metric's real sample size, denominator, tested cohort and uncertainty interval; scores without those details are easily misinterpreted.

Incorrect Answers

One error in a tiny sample is not comparable to a large controlled test.

An aggregate can hide important subgroup failures.

Empirical estimates have sampling error and distribution limits.

Question 471

A model change appears to improve accuracy by dropping hard cases. Which evidence is needed for an accurate comparison?

Drop difficult failures from the new version's benchmark and compare its higher score to the old set.
Retune acceptance thresholds after seeing the new model's failures.
Compare the same fixed representative evaluation population across versions.
Use only the new model's fluent output as proof it improved over an old model.

Explanation

Correct Answer

Compare model versions on the same fixed representative test set, including hard cases, with matched evaluation rules and denominators.

Incorrect Answers

Different populations create artificial improvement.

Post-hoc rules make the comparison biased.

A paired outcome comparison is required for regression evaluation.

Question 472

A retrieval evaluation includes duplicate near-identical queries only. What should be added to this test program before release?

Increase realistic query coverage rather than inflating pass counts.
Add dozens of paraphrases of one easy query and count each as independent breadth.
Replace difficult user intents with simple benchmark repetitions.
Score retrieval solely by numberOfResults returned rather than relevant distinct evidence.

Explanation

Correct Answer

Deduplicate near-equivalent queries and expand real scenario, source, language and failure coverage so pass counts reflect diverse tested tasks.

Incorrect Answers

Near-duplicates inflate sample count without covering new failure modes.

The test set no longer represents production demand.

Large hit counts do not establish query coverage or correctness.

Question 473

A production AI failure cannot be reproduced because input facts and the serving versions are missing. Select THREE items to preserve so a reviewer can replay and investigate that exact failure.

Save only a screenshot of the final answer without source inputs or version IDs.
Rerun the failure on an unspecified latest model and assume it matches the original environment.
Drop the expected business outcome because evaluation can infer it from generated text.
Store source input fixture and expected output.
Pin model/prompt/data versions.
Retain reproducible failure and correction evidence.

Explanation

Correct Answer

Store a governed source input and expected-output fixture that reproduces the failing case.

Pin the exact model, prompt and document/index versions and inference settings for the replay.

Retain the observed failure receipt and correction evidence so regression tests can verify real repair.

Incorrect Answers

A screenshot is not executable replay evidence.

Model and source changes can alter the failure.

A model response is not an independent ground truth.

Question 474

A model generated citation resolves but contradicts answer. Which evidence is needed for an accurate comparison?

Evaluate whether cited text entails each claim rather than link existence alone.
Award a perfect citation score if the web link returns HTTP 200.
Use model confidence to decide if the citation probably supports a claim without reading it.
Count any retrieved document from the right domain as confirming every generated fact.

Explanation

Correct Answer

Check each substantive claim against the cited passage's actual text and effective revision, not merely that a URL resolves.

Incorrect Answers

A reachable page can contradict the answer.

Confidence is not evidence of textual entailment.

Topic similarity is weaker than claim-specific support.

Question 475

A team reports a single success score for complex agent workflow. What should be added to this test program before release?

Use a single self-reported task-success sentence as the entire workflow score.
Count executed tool calls alone without checking whether the business goal was met.
Ignore retry/compensation paths because the happy-path workflow succeeds in demos.
Break down plan correctness, tool success, recovery and final task completion.

Explanation

Correct Answer

Evaluate plan correctness, authorized tool execution, error recovery and final confirmed outcome as separate components of complex agent task quality.

Incorrect Answers

The model may omit tool failures or unauthorized side effects.

More activity is not proof of successful task completion.

Production reliability requires evaluating recovery and partial states.

Question 476

After an SDK upgrade, Bedrock Converse rejects a request containing an image content block for a model version that supports text-only message content. The caller already has invocation permission. Which diagnostic isolates the malformed request?

Increase model context length to repair missing required role and content keys.
Validate the selected model’s supported content-block types and the Converse message role/content structure before sending the request.
Pass a plain string as the full Converse messages field and assume the service wraps it automatically.
Switch to an unrelated inference Region without checking the request schema.

Explanation

Correct Answer

Converse messages are typed structures and model modality support is version-dependent. An unsupported image block is a request contract error, not evidence that AWS invocation permissions failed.

Incorrect Answers

Token capacity does not change the API request schema.

Converse requires typed message objects and content blocks.

Region selection cannot repair malformed JSON message structure.

Question 477

A request to Bedrock fails with AccessDenied. What specific check can confirm the actual failure mechanism?

Check IAM action, resource ARN, guardrail/profile access and active principal.
Change the request prompt wording without inspecting IAM resource permissions.
Give only S3:GetObject access to the caller as a blanket remedy for Bedrock inference denial.
Disable authorization checks in the frontend because the AWS SDK already authenticated the role.

Explanation

Correct Answer

Inspect the effective IAM principal, bedrock action and model/profile ARN, plus any guardrail or destination-Region SCP constraints behind AccessDenied.

Incorrect Answers

Natural-language changes do not grant authorization for a denied API call.

Object-read rights do not authorize InvokeModel.

Successful credential authentication does not imply access to the requested model resource.

Question 478

An application sends a Converse inference request to the Bedrock control-plane endpoint and receives an unknown-operation error; the same credentials can list models successfully. Which endpoint diagnostic is appropriate?

Compare the requested API action with the target endpoint and move Converse inference to bedrock-runtime while keeping administrative catalog calls on bedrock.
Use the Bedrock control client for Converse inference because it can list models.
Invoke a foundation model directly through the agent build-time endpoint and ignore operation mismatch.
Change only the request body while leaving the application connected to a wrong API service endpoint.

Explanation

Correct Answer

Listing models succeeds on the control plane, but Converse is a runtime inference operation. Classify the request and endpoint service before treating an operation-not-found response as a model authorization or networking failure.

Incorrect Answers

The control plane does not implement the Converse runtime operation.

Build-time agent management is not standard FM inference.

Service operation routing is resolved before model payload validation.

Question 479

An application invokes an exact Bedrock model ID successfully in us-east-1 but gets a model-unavailable ValidationException in a second Region despite an approved caller identity. What availability diagnosis is required before modifying IAM?

Reuse the model ID from a deployment in another Region without checking destination availability.
Grant all S3 permissions when the actual model identifier is not available in the Region.
Retry the same invalid model ID indefinitely without inspecting model cards/profile routes.
Inspect the model version’s supported Regions and inference profile/capacity requirements for that caller Region before blaming permissions.

Explanation

Correct Answer

Bedrock model and inference-profile availability varies by version and Region. A valid ID elsewhere does not establish direct on-demand availability in this Region; check catalog/support first.

Incorrect Answers

Regional catalogs and deployment modes differ.

Unrelated permissions cannot create a missing model offering.

Repeated requests cannot fix unsupported regional configuration.

Question 480

A chatbot passes small JSON cases but returns unfinished objects for long requests. Runtime traces show a max-token stop reason at a low configured output-token limit. Which diagnostic can confirm the mechanism and guide a safe retest?

Parse an incomplete JSON prefix as a successful result because generation ended normally at the token limit.
Raise only the API Gateway timeout while maxTokens remains unchanged.
Treat closing braces guessed by the application as evidence that the missing output fields were actually generated.
Correlate stop reason and token usage with the configured/model-supported output limit, then retest with sufficient bounded output budget and schema validation.

Explanation

Correct Answer

When the response stops at max_tokens, the immediate failure is output truncation, not necessarily a schema-model bug. Inspect token usage and allowed limits before evaluating a corrected full response.

Incorrect Answers

A length-limited stop can leave required fields missing.

Timeout settings cannot extend the model's configured output token cap.

Auto-completing syntax does not validate business semantics.

Pass Your Amazon AWS Certified Generative AI Developer - Professional AIP-C01 Exam Easy!

Question 470

Explanation

Correct Answer

Incorrect Answers

Question 471

Explanation

Correct Answer

Incorrect Answers

Question 472

Explanation

Correct Answer

Incorrect Answers

Question 473

Explanation

Correct Answer

Incorrect Answers

Question 474

Explanation

Correct Answer

Incorrect Answers

Question 475

Explanation

Correct Answer

Incorrect Answers

Question 476

Explanation

Correct Answer

Incorrect Answers

Question 477

Explanation

Correct Answer

Incorrect Answers

Question 478

Explanation

Correct Answer

Incorrect Answers

Question 479

Explanation

Correct Answer

Incorrect Answers

Question 480

Explanation

Correct Answer

Incorrect Answers

AWS Certified Generative AI Developer - Professional AIP-C01 Practice Tests