Sydney's Duplicate Image Problem: How the City Stacks Up Against London, Singapore and Toronto

Sydney's network of councils, state agencies and public institutions is sitting on a growing crisis of duplicate and AI-generated imagery — and the tools most of those bodies are using to catch it were built before the problem existed at scale. That gap is now costing time, budget and, in some cases, legal exposure.

The issue landed squarely on the agenda this year after the NSW Department of Planning flagged internally that its online heritage database — which covers more than 2,500 items listed under the State Heritage Register — contained hundreds of duplicated property photographs, some mislabelled, some scraped from real estate portals without authorisation. The department has not publicly disclosed the full scope of the audit.

Why does it matter right now? The surge in generative AI tools since late 2023 has made it trivially easy to produce near-identical variants of an image that defeat older hash-based detection. Libraries, archives and government asset registers globally have been scrambling. Sydney is no exception, and the problem is colliding directly with two of the city's most politically charged pressure points: housing approvals and heritage protection.

What Sydney Is Actually Doing

The City of Sydney Council's digital asset team — based at Town Hall House on George Street — began trialling a perceptual hashing and machine-learning pipeline in March this year to audit its public image library, which runs to more than 80,000 files accumulated since 2009. The tool flags near-duplicate images for human review rather than auto-deleting, a deliberate choice made after an earlier automated cull in 2022 accidentally removed originals from the Surry Hills library redevelopment documentation.

Separate from council, the State Library of New South Wales on Macquarie Street has been running a deduplication project under its Digital Excellence Program since January 2026, targeting its digitised collection of roughly 5 million images. Librarians there are working through a backlog that grew during the COVID digitisation push of 2020 and 2021, when contractors uploaded material in bulk with minimal quality control.

Neither program is fully funded. The State Library's Digital Excellence Program received $3.4 million in the 2025-26 NSW Budget — enough, according to budget documents, to cover roughly 18 months of operations across three separate digitisation workstreams, of which duplicate detection is one.

How Sydney Compares Globally

London's approach is more centralised. The Greater London Authority consolidated image rights and deduplication responsibilities under a single Digital Infrastructure unit in 2024, giving it authority across borough councils — a structural option that NSW's fragmented council system makes extremely difficult to replicate. Transport for London's asset image register, covering station signage and infrastructure documentation, is now audited quarterly using a vendor tool built by a UK-based firm under a three-year contract awarded in late 2023.

Singapore is further ahead still. The National Heritage Board there embedded automated duplicate detection directly into its Collections Online portal in 2022, requiring all contributing institutions to pass images through the system before ingest. The result is a near-zero duplicate rate on new additions, though legacy content remains a problem.

Toronto sits closest to Sydney's position. The City of Toronto Archives undertook a manual deduplication audit across 1.2 million digital records in 2023 and 2024, a process that took 14 months and was funded through a one-off federal cultural infrastructure grant. City archivists there have publicly noted the audit consumed staff capacity that would otherwise have gone to new acquisitions.

The practical gap between Sydney and Singapore comes down to governance, not technology. The tools exist. What Sydney's institutions lack is a mandate that forces standardisation at the point of upload rather than retrospective cleanup years later.

For anyone dealing with this at an operational level — property developers submitting heritage impact statements to the NSW Heritage Council, say, or creative agencies pitching imagery to council tourism units — the near-term reality is more paperwork, not less. Councils are increasingly asking for image provenance declarations alongside submissions, and the Heritage Council updated its submission guidelines in April 2026 to include a requirement that photographs be accompanied by metadata confirming their origin. Getting that documentation right before lodgement will matter more as the audit culture tightens.