Apple has revealed plans to install software on its users’ phones to scan photos for images of child sexual abuse.

The new programme will be rolled out across Apple iPhones in the US as part of a series of new child protection features. An AI-powered tool called neuralMatch will scan photos before they are uploaded to iCloud Photos. The images will be compared against a database of known abuse images.

Any pictures that show a strong enough match will be flagged. Apple will also have the ability to manually review the reported images.

Should the image be found to contain abuse, the user will have their account disabled and the US National Center for Missing and Exploited Children (NCMEC) will be notified.

According to Apple, the technology provides “an extremely high level of accuracy and ensures less than a one in one trillion chance per year of incorrectly flagging a given account”.

The company’s system uses a hashing process that takes a photo and encrypts it, creating a unique string of numbers and digits. All the sample images have been have been hashed using this process. The same hashing process is used on locally stored photos on the user’s device. The Apple system can then detect if the strings generated by the photo match one in the abuse database.

Apple currently uses the technology to scan photos already stored on iCloud. The new plan will see the technology moved on device.

In addition, the company will begin scanning end-to-end encrypted messages with a similar AI tool to spot sexually explicit images.

While the encryption means that the content of messages is secured and cannot be read during transit, the content is still vulnerable before it is sent and once it arrives.

However, rather than reporting the images to Apple, the system will be used to alert parents and children that they are receiving or sending explicit photos.

The initiative was proposed to a collection of US academics this week. At present, the software will be added to iPhone next month as part of iOS 15.

Child Abuse or Privacy Abuse?

The use of the internet to share images of child abuse is a major problem facing law enforcement and tech companies.

Previous warnings have suggested that the problem may largely be hiding in plain sight. In late 2020, figures revealed that over 90% of the 69 million sexual abuse images reported in the US were not hidden on some dark web database, but were being sent over Facebook messenger.

Home Secretary Priti Patel later criticised Facebook, especially over its plans to rollout end-to-end encryption on its messenger. The move would make it impossible for either Facebook or authorities to scan messages for content.

As such, Apple’s move touches on a larger debate around privacy. While end-to-end encrypted messengers have proven popular with consumers – WhatsApp suffered a major backlash over a new privacy policy that included greater data sharing – the practice has proven controversial.

Opponents of end-to-end encryption say that it enables criminal activity, allowing people to send illicit content through commonly available services. Meanwhile, privacy activists warn that practices like Apple’s image searching capacity opens up innocent people to surveillance.

Apple has claimed that the new system skirts the line between respecting and maintaining its users’ privacy while helping fight the issue of child abuse images.

Paul Bischoff, a privacy advocate at Comparitech, said: “Although there are privacy implications, I think this is an approach that balances individual privacy and child safety. The important thing is that this scanning technology is strictly limited in scope to protecting children and not used to scan users’ phones for other photos.

“If authorities are searching for someone who posted a specific photo on social media, for example, Apple could conceivably scan all iPhone users’ photos for that specific image.

“The hashing system allows Apple to scan a users’ device for any images matching those in a database of known child abuse materials. It can do this without actually viewing or storing the user’s photos, which maintains their privacy except when a violating photo is found on the device.

Recommended

• Edinburgh Uni startUps raise £6.27m on new AI acceleration programme
• Footballers facing a torrent of racist, homophobic abuse on social media
• SCDI partners with Pawprint to tackle employee carbon footprint

However, there are concerns that the same technology could be used to scan iPhone users’ photos for other purposes. The Pegasus scandal has shown the extent that surveillance technology can be used to spy on innocent parties.

Chris Hauk, consumer privacy champion at Pixel Privacy, noted: “While I am all for clamping down on child abuse and child pornography, I do have privacy concerns about the use of the technology. A machine learning system such as this could crank out false positives, leading to unwarranted issues for innocent citizens.

“Such technology could be abused if placed in government hands, leading to its use to detect images containing other types of content, such as photos taken at demonstrations and other types of gatherings. This could lead to the government clamping down on users’ freedom of expression and used to suppress “unapproved” opinions and activism.”

As one of the most popular families of smartphones in the world, there are around 1.9bn iPhones out there being used by about 1bn people. As such, if the initiative were rolled out on a global scale, it would put a huge section of the global population under surveillance.