Depending on what your size and other things, sometimes it's just easier to have dev instances you test against. Like S3 emulator, you could download one of many and test it but unless your test suite is extensive, the costs against S3 test bucket would probably cost you .03 cents for assurances that everything will work against AWS. If that's breaking the bank at the company, you probably have bigger problems.
Not sure if your serious.
You picked the simplest service that isn't expensive at all to run. Have you tried with an RDS cluster for example, how does that work?
Does your integration test create a new cluster to use? That's slow to set up.
Does it use an existing one? Now you might have failures due to noisy neighbors and your cost will not be small.
And that's just RDS, as soon as it's not serverless, costs aren't trivial anymore.
Again, while local cloud emulator is great for cost sensitive customers, at some point, you should test Dev against almost exact environment you will be running Prod on. If not, you are asking for eventual outage in Prod when some slightly different AWS thing bites you.