upvote
There were two different technologies. One was client-side scanning for known CSAM, which created a huge backlash and is described here: https://educatedguesswork.org/posts/apple-csam-intro/

The other is detection of images that may contain nudity, whether sent or received, when the owner/admin/parent enables the feature. It is relatively uncontroversial and is described here: https://support.apple.com/en-us/105069

reply
> The other is detection of images that may contain nudity, whether sent or received, when the owner/admin/parent enables the feature. It is relatively uncontroversial [...]

That's a new version. The one that as announced the same time as client side scanning to block uploading CSAM to iCloud worked like this.

1. It could be enabled on a child's device by the parents. It was not on be default.

2. If the child received a sexual image (not necessarily just CSAM...if an adult sends a dick pic to a child that is not CSAM but would have been flagged) the image is blocked, the child is notified, told their parents are worried the image may harm them, and asked if they still want to see it.

3. If the child says no, they do not want to see it, that is the end of the matter.

4. If the child says that they do want to see it and they are at least 13 they are shown the image and that is the end of the matter.

5. If the child says that they do want to see it and they are under 13, they are again told that they parents are concerned, and that if they view it their parents will be notified, and asked if they still want to view it.

6. If they say no that is the end of the matter.

7. If they say yes they see it but the parents also are notified and will be able to see it.

This should have been pretty uncontroversial, but there were objections on the grounds that if someone say sends their dick pic to your under 13 child and the child goes all the way through to step 7 and decides to view it, that is a violation of the sender's privacy because that message was only intended for the child.

reply
The controversial part is having the system enabled by default with age verification required to turn it off, and having the system impact non-Apple/Google apps. The UK for example wants Apple and Google to forcibly enable nudity blocking on all devices in the UK, and they want the system to bypass app/DRM security to scan all content visible on a device.
reply
That is what they actually deployed. They were planning on performing client-side scanning of all images uploaded to iCloud for CSAM and reporting it to the authorities, but backpedaled after public push-back.
reply
The original proposal was to use a visual hash designed to identify known bad CSAM images even if cropped or otherwise edited. Your computer would scan all your stuff for these images and report you to apple who would report you to the cops. This presented a number of issues.

Accidental false positives could lead to horrific outcomes up to and including oh look bob got shot by the cops for resisting.

It was possible to produce apparently matching innocuous images and then poison people's machines with them.Oops did you click on that picture of a tree have fun with the cops. Like an advanced form of swatting.

Although inspired by a desire to find CSAM Apple could be forced to scan for ANYTHING by repressive regimes including America and China.

Although initially targeting images client side scanning of messages is a pretty obvious next step. Again obvious good motivation exists and is completely justifiable who doesn't want to stop the next mass shooting or terrorist attack... and then we can basically use it to find people critical of the regime. Do remember we are presently prosecuting a political figure for a picture of sea shells and a guy in texas is rotting in prison for distributing political literature.

reply
> Apple could be forced to scan for ANYTHING by repressive regimes including America and China.

Including Europe. Europe is ruled by people who think 1984 was an instruction manual.

reply