Sydney's Duplicate Image Problem: How the City Stacks Up Against London, Singapore and New York

Sydney's public sector is sitting on a sprawling mess of duplicated digital imagery — and the institutions tasked with fixing it are doing so without a unified plan. Across the City of Sydney Council, the State Library of NSW on Macquarie Street, and the cultural holdings managed through Create NSW, cataloguers and archivists are independently running deduplication software on collections that, in some cases, overlap substantially. The absence of a shared metropolitan-level framework has left each body reinventing the wheel.

The timing matters. Federal funding attached to the National Cultural Policy — Revive, released in January 2023 — includes a digitisation stream that requires recipient institutions to meet data quality benchmarks by mid-2027. Duplicate image records directly undermine those benchmarks. Institutions that cannot demonstrate clean, deduplicated catalogues before the next assessment round risk losing access to further grant tranches. That pressure is now arriving at exactly the moment Sydney's collections are growing fastest, partly because of the Metro West construction corridor through Parramatta generating fresh archaeological and documentary photographic records.

What Sydney Is Actually Doing

The State Library has been running Veridian-based cataloguing software for years, but deduplication has historically been a manual, collection-by-collection exercise. Staff in the Mitchell Reading Room on Macquarie Street handle requests for historical photographic prints that frequently appear in multiple catalogued forms — scanned from originals, copied from microfilm, and re-digitised from earlier low-resolution scans. According to the Library's own published digital strategy from 2022, the institution acknowledged that image metadata inconsistency was a known problem requiring resolution before 2025. That deadline has passed.

The City of Sydney Council's open data portal, which hosts planning and heritage imagery for suburbs from Redfern to Pyrmont, uses a separate system again. Council staff confirmed in published meeting minutes from March 2026 that a data audit of the heritage image repository had been commissioned, with results expected in the third quarter of this year. No completion date for any deduplication work was specified in those minutes.

Compare that to Singapore, where the National Heritage Board completed a sector-wide deduplication exercise across 14 institutions in 2024 using a centralised hash-matching protocol, consolidating roughly 2.3 million image records into a single national index. London's Wellcome Collection and the Museum of London Archaeology completed a joint deduplication project in 2023 that cut redundant records by an estimated 40 percent across their shared photographic archive. New York's DPLA-aligned institutions operate under a metadata aggregation framework that flags duplicate entries automatically at the point of ingest, a standard Sydney's cultural sector has yet to adopt at scale.

The Cost of Getting It Wrong

Redundant image records are not just a storage problem. For researchers at institutions like the University of Technology Sydney's library in Ultimo, or postgraduate students working through the Powerhouse Museum's MAAS collection in Ultimo and Parramatta, duplicate entries mean wasted time chasing records that are functionally identical. In heritage planning — a live issue along the Parramatta Road urban renewal corridor — duplicate cadastral and site photographs can create conflicting evidentiary records in development applications, complicating assessments that run through the Department of Planning.

The financial dimension is real. Cloud storage costs for unmanaged image archives are not trivial for mid-sized public institutions. A 2024 report by the Australian Institute for the Conservation of Cultural Material estimated that Australian collecting institutions collectively spend tens of millions of dollars annually on storage infrastructure, a figure that deduplication programs in comparable jurisdictions have demonstrably reduced.

For institutions still working through their audit processes, the practical next step is not complex: adopt hash-based fingerprinting at the point of ingest rather than retrospectively. The State Records NSW has published guidance on this. Institutions waiting for a metropolitan coordinating body to appear before acting are likely to miss the 2027 federal benchmarks. The State Library's digital team, the Create NSW grants unit at 4 Parramatta Square, and the council's open data group could convene a working group without waiting for legislation. Peers in Singapore did not wait either.