Sydney's Duplicate Image Problem: How the City Stacks Up Against London, Amsterdam and Singapore

Sydney's public institutions are sitting on tens of thousands of duplicate digital images — redundant scans, mislabelled photographs and identical files stored across disconnected servers — and the effort to clean up that mess is exposing just how far behind the city's digital infrastructure has fallen compared to peers like Amsterdam, Singapore and London.

The problem matters now because NSW government agencies are deep into a multi-year digitisation push tied to the state's broader public records modernisation agenda. As Metro West construction churns through Western Sydney and councils from Blacktown to Bayside rush to digitise planning documents ahead of expected housing-approval surges, the volume of image assets held by local government has ballooned. Without deduplication protocols, agencies risk indexing the same image multiple times, inflating storage costs and producing unreliable search results in public-facing archives.

What Sydney Is Actually Doing

The State Library of NSW, on Macquarie Street in the CBD, began a structured deduplication review of its digital collections in late 2024, targeting the Holtermann collection and several twentieth-century photographic series. The Library uses perceptual hashing software — a method that detects visually identical or near-identical images even when file names differ — as part of its collections management workflow. City of Sydney Council's property and heritage image database, maintained through its Customs House library branch at Alfred Street, Circular Quay, has similarly flagged duplicate asset removal as a 2026 priority under its digital services roadmap.

Parramatta City Council, which manages one of the largest heritage photographic collections in Western Sydney, contracted an external records management firm in early 2026 to audit its digital asset library ahead of the Parramatta Light Rail Stage 2 corridor documentation project. The audit — details of which have not been made public — is understood to cover image files dating back to the council's first digital capture programs in the early 2000s.

The NSW State Archives, based at Kingswood in Western Sydney, published guidance in March 2026 advising government agencies to adopt deduplication as a standard step before any migration to cloud storage platforms. The guidance references ISO 15489, the international standard for records management, as the baseline framework agencies should follow.

How Global Cities Compare

Amsterdam's Stadsarchief — the city's municipal archive — completed a full deduplication sweep of its 750,000-image online collection in 2023, reducing redundant files by roughly 18 percent and cutting annual storage costs measurably, according to figures the archive published in its 2023 annual report. The Dutch institution uses an open-source tool called DupeGuru alongside custom metadata reconciliation scripts, and its approach has since been adopted by archives in Utrecht and Rotterdam.

Singapore's National Archives, under the National Library Board, embedded automated duplicate detection into its ingest pipeline by 2021. Every image submitted to the National Online Repository of the Arts programme passes through a hash-check before cataloguing. London's Metropolitan Archives at Clerkenwell adopted a similar protocol in 2022 as part of a broader London Digital Archive programme funded partly through the UK's Arts Council.

Sydney's institutions, by contrast, have generally treated deduplication as a retrospective cleanup exercise rather than an upstream ingest control. That means problems compound with every new digitisation batch — a structural difference that records management specialists have pointed to in submissions to the NSW Legislative Council's standing committee on state development over recent years.

The practical stakes are not abstract. When a resident searches the State Library's online catalogue for historical images of, say, George Street or the old Darling Harbour wharves, duplicate entries degrade search precision and can cause the same image to appear under conflicting copyright or provenance records — creating real headaches for journalists, researchers and heritage consultants who rely on the archive's metadata as authoritative.

For Sydney's institutions, the immediate next step is aligning local practice with the NSW State Archives' March 2026 guidance before the next budget cycle. Agencies that can demonstrate compliant digital asset workflows are better positioned to access the state's Digital Restart Fund, which has backed previous infrastructure upgrades across government. Public users who spot obvious duplicates in the State Library or City of Sydney catalogues can flag them directly through each institution's online catalogue feedback form — a small but direct contribution to a cleanup that, in Amsterdam and Singapore, ultimately took years of sustained institutional will to complete.