question: when I re-run the tests against AWS, I find that a KeyMaterialId field in the decrypt_response_with_encryption_context block of this snapshot is added, which is not captured in this PR. Can you explain why? Did you execute these tests against AWS, or manually update the snapshot file?

You can see more details about our parity testing approach here: https://github.com/localstack/localstack/tree/master/docs/testing/parity-testing

I executed the tests against AWS that generated the new block (decrypt_response_with_invalid_ciphertext) in test_kms.snapshot.json. I don't expect any changes in response for decrypt_response_with_encryption_context with my changes in test.
Let me test this again to double check.

@simonrw I am not seeing KeyMaterialId or any other changes in block decrypt_response_with_encryption_context in test_kms_snapshot.json
I am testing with following command after setting required env variables to update the snapshot and to execute the against the AWS. I can also see keys created in my AWS account.
TEST_PATH="tests/aws/services/kms/test_kms.py::TestKMS::test_encrypt_decrypt_encryption_context" make test
This is updated snapshot block after running above test:

diff --git a/tests/aws/services/kms/test_kms.snapshot.json b/tests/aws/services/kms/test_kms.snapshot.json
index 17ebf79f2..e33911e7e 100644
--- a/tests/aws/services/kms/test_kms.snapshot.json
+++ b/tests/aws/services/kms/test_kms.snapshot.json
@@ -1565,7 +1565,7 @@
     }
   },
   "tests/aws/services/kms/test_kms.py::TestKMS::test_encrypt_decrypt_encryption_context": {
-    "recorded-date": "11-05-2023, 22:46:49",
+    "recorded-date": "01-07-2025, 17:01:36",
     "recorded-content": {
       "encrypt_response": {
         "CiphertextBlob": "ciphertext-blob",
@@ -1594,6 +1594,16 @@
           "HTTPHeaders": {},
           "HTTPStatusCode": 400
         }
+      },
+      "decrypt_response_with_invalid_ciphertext": {
+        "Error": {
+          "Code": "InvalidCiphertextException",
+          "Message": ""
+        },
+        "ResponseMetadata": {
+          "HTTPHeaders": {},
+          "HTTPStatusCode": 400
+        }
       }
     }
   },

could you please suggest anything wrong with these steps test against AWS?

If you look at the How to write parity tests section of our docs, you should find the instructions on how to run the tests against AWS.

Specifically:

Run the test against AWS: use the parameter --snapshot-update (or the environment variable SNAPSHOT_UPDATE=1) and set the environment variable as TEST_TARGET=AWS_CLOUD.

You have run the tests with your AWS credentials, however you are not configuring the LocalStack test suite to target AWS.

I am testing with following command after setting required env variables to update the snapshot and to execute the against the AWS. I can also see keys created in my AWS account.
TEST_PATH="tests/aws/services/kms/test_kms.py::TestKMS::test_encrypt_decrypt_encryption_context" make test

I can't really explain what is going on here unfortunately. You are clearly

• updating the snapshot with SNAPSHOT_UPDATE=1 since it has changed, and
• targeting AWS with TEST_TARGET=AWS_CLOUD since you say the keys are created (but they should only be temporary, they should be scheduled for deletion after the test completes, right?)

but when I re-run the test with the same configuration, I find the KeyMaterialId property has been added

diff --git i/tests/aws/services/kms/test_kms.snapshot.json w/tests/aws/services/kms/test_kms.snapshot.json
index e97a3097e6d7..33e987b371d7 100644
--- i/tests/aws/services/kms/test_kms.snapshot.json
+++ w/tests/aws/services/kms/test_kms.snapshot.json
@@ -1565,7 +1565,7 @@
     }
   },
   "tests/aws/services/kms/test_kms.py::TestKMS::test_encrypt_decrypt_encryption_context": {
-    "recorded-date": "27-06-2025, 08:31:49",
+    "recorded-date": "04-07-2025, 10:40:02",
     "recorded-content": {
       "encrypt_response": {
         "CiphertextBlob": "ciphertext-blob",
@@ -1579,6 +1579,7 @@
       "decrypt_response_with_encryption_context": {
         "EncryptionAlgorithm": "SYMMETRIC_DEFAULT",
         "KeyId": "<key-id:1>",
+        "KeyMaterialId": "4ba2a196f27b13af15cce4730e67b632eff7b53b68f987b7331d209fddc0b103",
         "Plaintext": "plaintext",
         "ResponseMetadata": {
           "HTTPHeaders": {},

I executed the tests against AWS that generated the new block (decrypt_response_with_invalid_ciphertext) in test_kms.snapshot.json. I don't expect any changes in response for decrypt_response_with_encryption_context with my changes in test.

If you are executing the test against AWS and capturing snapshots, then it captures all snapshot.match values, not just the one that you have changed. So it's not unexpected that even though you have not changed the code that contributes to the decrypt_response_with_encryption_context snapshot content, that it may change.

This is a challenge we face regularly, you see the last time this snapshot was executed was 2023! We regularly find the AWS responses have changed since the last time the snapshot was captured.

When I re-run the entire kms test suite against AWS I see

• other snapshot responses have additional keys (also relating to key material so likely related)
• some tests fail which is interesting, they need revisiting

question: do you think it's worth logging an error message on line 1047 where we catch and silence any exceptions?

Yes, agree. I should have done it. We should log the error before line 1047 to avoid hiding potential bugs caused by catching all exceptions without logging.

Do you think this change should be part of this PR or a separate one?