Operating, Monitoring, and Securing ML and AI Solutions

Question 50

A retail company runs a customer support agent on Amazon Bedrock AgentCore Runtime. The agent calls a foundation model on Amazon Bedrock and an order-lookup tool that is implemented as an AWS Lambda function. Users report that some answers stop mid-sentence and that order lookups fail intermittently. The built-in AgentCore metrics in Amazon CloudWatch show a rising error rate, but the team can't tell which step of the agent's workflow fails.

Which approach should the ML engineer take to find the root cause?

A. Turn on AWS CloudTrail logging for the agent runtime and search the trail for failed InvokeAgentRuntime calls and their error codes.

B. Enable CloudWatch Transaction Search, instrument the agent code with the AWS Distro for OpenTelemetry (ADOT) SDK, and inspect the agent's traces in the CloudWatch generative AI observability dashboard to see the latency, errors, and token usage of each model call and tool call.

C. Turn on AWS X-Ray active tracing for the order-lookup Lambda function and inspect the function's service map for errors.

D. Configure a SageMaker Model Monitor data quality monitoring schedule on the agent's requests to detect drift in the questions that users ask.

Question 51

A health care company deploys a SageMaker real-time endpoint for patient risk scoring. Regulatory requirements mandate that all API calls to SageMaker AI, including the identity of who invoked the endpoint and the timestamp, must be logged and auditable. Additionally, the operations team must monitor inference latency and receive alerts when the p99 latency exceeds 500 ms.

Which services should the team use to meet these requirements? (Select any two options.)

A. AWS CloudTrail to log and audit all SageMaker AI API calls, including caller identity and timestamps.

B. AWS X-Ray to trace all endpoint invocations and capture the identity of each caller for audit purposes.

C. Amazon CloudWatch to monitor ModelLatency metrics and configure alarms when p99 latency exceeds the 500 ms threshold.

D. SageMaker Model Monitor to track inference latency metrics and generate alerts when latency thresholds are breached.

E. Amazon QuickSight to build a real-time dashboard that visualizes API call audit logs and latency metrics with alerting capabilities.

Question 52

A legal services company runs a contract question-answering assistant on Amazon Bedrock with on-demand inference. Every request starts with the same 6,000-token system prompt that contains instructions and a library of standard clauses, followed by a short user question. Users send several thousand requests per hour during the business day and expect interactive responses. The legal team has approved the current model, so the company must keep using it, but token costs have grown every month.

Which change will reduce the inference cost the most without changing the model or slowing down responses?

A. Purchase Provisioned Throughput for the model so that the requests run on dedicated model units.

B. Enable prompt caching and place a cache checkpoint after the static system prompt, so that repeated requests read the shared prefix from the cache.

C. Submit the questions as Amazon Bedrock batch inference jobs and return the answers to users when each job completes.

D. Use Amazon Bedrock Model Distillation to create a smaller student model from the current model and route all requests to the student model.

Question 53

A data scientist needs to grant a SageMaker notebook instance access to read training data from an ...