Amazon Says AI Agents Wrote Its AWS Emulator

Amazon released AWS Emulator on Nov. 11, a local tool that imitates Amazon Web Services so developers can test cloud applications on their own computers.

The company said autonomous artificial intelligence agents wrote and reviewed the emulator’s implementation. Matthew Goldberg, an Amazon engineer, wrote that his team did not write or review the emulator’s code. Human engineers built the guardrails and checks that determine whether changes can be released.

Amazon uses three groups of AI agents on the project. One manages tests that measure how closely the emulator matches AWS. Another finds differences between the emulator and real AWS services. The third fixes individual issues.

Before a change can be merged, it must pass checks for compilation, coding standards, security and regressions. The agents use publicly available AWS application programming interface models and compare the emulator’s behavior with the real cloud. Amazon’s public evaluation suite says each fidelity test was checked against at least two sandboxed AWS accounts.

AWS uses “fidelity” to describe how closely the emulator matches its cloud services. A test passes when the emulator produces the same result as AWS for every call in a covered workflow. The comparison includes status codes, created resources, side effects and errors. For example, a successful response from the emulator would fail the test if AWS returns an error.

A passing test covers only the workflow it evaluates. AWS’s evaluation materials state that behavior without a fidelity test remains unverified. The available tests therefore do not confirm that every possible use of an AWS service will work in the emulator.

AWS warns developers not to use the emulator to assess whether an application is secure. The tool does not evaluate AWS Key Management Service key policies or Amazon Simple Storage Service bucket policies. A request denied by real AWS may succeed in the emulator.

AWS instructs developers to test the complete application and its security settings on real AWS before deploying it to production. The emulator is intended for local development and testing and is not designed to run production workloads.