Sydney's Duplicate Image Problem: How the City Stacks Up Against London, Singapore and New York

Sydney's public sector holds tens of millions of digital images across dozens of agencies — and a growing number of them are duplicates. The NSW State Archives and Records Authority has been working through a backlog of redundant image files across government databases since mid-2025, a process that archivists and records managers describe as one of the most labour-intensive digital hygiene challenges the sector has faced since the shift to cloud storage accelerated after 2020.

The timing matters. With the NSW government under pressure to cut operational costs and improve service delivery ahead of the next state election, digital asset management has moved from a back-office concern to a budget line item. Duplicate images inflate storage costs, slow database queries, and create legal and compliance headaches — particularly for agencies managing planning records, court documents, and health files, where image provenance can carry real legal weight.

What Sydney Is Actually Doing

The City of Sydney Council's digital records team, based out of its administrative offices near Town Hall, began a deduplication audit of its planning portal image libraries in March 2026. The portal, which covers development applications across suburbs from Pyrmont to Zetland, had accumulated years of resubmitted documents where the same site photographs appeared multiple times under different file names. Council staff are using automated hash-matching tools to flag identical files before human reviewers confirm deletion — a two-step process designed to avoid accidental loss of genuinely distinct records.

Parramatta City Council launched a similar program in January 2026 through its Digital Transformation Office on Darcy Street. The council's records team identified duplicate image bloat as a secondary problem stemming from its 2022 migration to a new document management system, when thousands of files were imported twice. The State Library of NSW, on Macquarie Street in the CBD, has been running a longer-term deduplication project across its digitised photographic collections since 2023, using a combination of perceptual hashing and metadata cross-referencing to sort originals from copies within its catalogue of historical images.

None of these agencies have publicly released figures on how many files have been removed or how much storage cost has been recovered. Requests to the NSW Department of Customer Service, which oversees digital infrastructure policy for the state, had not been returned by the time of publication.

How Sydney Compares Globally

Other major cities are further along. Transport for London completed a deduplication sweep of its CCTV image archive in 2024, reducing redundant storage load across its network of more than 15,000 cameras — a project it reported on through its annual digital infrastructure review. Singapore's Government Technology Agency, GovTech, embedded automated deduplication protocols into its Whole-of-Government data standards in 2022, meaning new image uploads to any linked agency system are checked for duplicates at the point of ingestion rather than retrospectively. New York City's Department of Records and Information Services published guidance in late 2023 requiring all city agencies to conduct biennial deduplication audits under its Digital Records Management Policy.

Sydney's patchwork approach — with individual councils and state bodies running separate projects on different timelines — reflects a broader gap in centralised digital governance. Unlike Singapore's top-down model or New York's mandatory audit cycle, NSW has no single policy requiring agencies to deduplicate image holdings on any fixed schedule. That leaves institutions setting their own priorities, often only acting when storage costs become impossible to ignore.

For residents and businesses interacting with government systems, the practical effect is slower search results in public-facing portals and occasional confusion when multiple copies of the same development application photo appear in search returns on the NSW Planning Portal. Anyone who has lodged a DA in suburbs like Marrickville or Penrith in recent years and noticed their submitted images appearing multiple times in the online tracking system has encountered the problem first-hand.

The immediate next step, according to publicly available NSW government digital strategy documents, is a whole-of-government storage rationalisation review scheduled for the second half of 2026. Whether that review will produce binding deduplication standards — rather than another set of advisory guidelines — is the question records managers across the sector are watching closely.