Sydney's Duplicate Image Problem: How the City Stacks Up Against London, Singapore and New York

Sydney's public institutions are sitting on tens of millions of duplicate digital images — redundant photographs, scanned documents and heritage records stored multiple times across fragmented government servers — and the bill for fixing it is rising faster than the storage costs themselves. The issue landed back on the agenda this month after a City of Sydney Council audit flagged the problem in its digital asset management review, drawing comparisons with infrastructure overhauls already completed in London, Singapore and parts of New York.

The timing matters. NSW is mid-way through an ambitious digitisation push tied to the state's broader public sector modernisation agenda. The State Library of New South Wales on Macquarie Street has been scanning fragile colonial-era materials since 2019. The City of Parramatta Council — which manages one of the fastest-growing local government areas in the country — accelerated its own records digitisation program after the 2022 merger consolidation work wrapped up. Both institutions, along with dozens of smaller councils and state agencies, now hold sprawling image repositories that were built quickly and without a unified deduplication standard. The result is predictable: the same photograph of, say, the 1890s Parramatta riverbank sitting in four different folders across three different systems, each copy incurring storage, licensing and retrieval costs.

What Other Cities Did First

London's approach is the most cited benchmark. The Greater London Authority consolidated its visual assets under a single digital asset management platform, a process completed in stages between 2021 and 2023 across more than 30 borough councils. Singapore's National Archives ran a parallel deduplication program from 2020, using perceptual hashing — a technique that identifies near-identical images even when file names, formats or metadata differ — to cull redundant files from its national digitisation archive. New York City's Department of Records and Information Services published a public-facing deduplication policy in 2022 after an internal review found roughly 18 per cent of its digitised photograph collection was duplicated at least once.

Sydney has no equivalent published policy. A spokesperson for the NSW Department of Customer Service, which oversees the state's digital infrastructure strategy, has not responded to requests for comment before deadline. The City of Sydney Council confirmed its digital asset review is ongoing but declined to provide figures ahead of a formal report expected later this quarter.

The practical consequences go beyond wasted storage. When emergency services, planners or journalists submit freedom of information requests for historical imagery — say, aerial shots of Western Sydney flood corridors or before-and-after photographs of the Barangaroo foreshore — duplicate files slow retrieval, inflate response times and sometimes return contradictory metadata. Infrastructure NSW has flagged image data integrity as a secondary concern in its Metro West project documentation, though the agency has not attributed specific costs to the problem publicly.

The Local Fix Is Fragmented

Several Sydney institutions are moving independently. The Powerhouse Museum, which relocated its main collection to Parramatta's Riverside Precinct, began a deduplication audit of its digitised object photography in early 2026 as part of a broader collection management overhaul. The State Archives NSW at Kingswood in Western Sydney is understood to be trialling automated deduplication tools across its post-1990 scanned document holdings, though the program has no confirmed public timeline or budget allocation.

What Sydney lacks — and what London, Singapore and New York built — is a cross-agency coordination framework. Without one, each institution re-solves the same technical problem at its own expense. A 2024 report by the Australian Library and Information Association estimated that Australian cultural institutions collectively spend between $40 million and $60 million annually on redundant digital storage, though that figure covers all file types, not images alone.

The path forward is clearer than the timeline. Institutions looking to act before any state-level policy emerges have three practical options: adopt open deduplication standards like those published by the Digital Preservation Coalition, run perceptual hashing tools across legacy repositories before migrating to new platforms, or join one of the existing cross-council digital asset consortia already operating in Victoria and Queensland. NSW has neither mandated nor funded any of those options. Until it does, Sydney's digital image libraries will keep growing — and so will the waste inside them.