Councils, archivists and digital experts weigh in on Sydney's duplicate image crisis

Digital archivists, council records managers and heritage preservation specialists across Greater Sydney are raising alarms about a largely invisible administrative problem: duplicate images clogging public databases, slowing heritage assessments, and costing ratepayers real money in wasted storage and staff hours. The issue has come into sharper focus in recent weeks as several councils and state agencies have begun auditing their digital asset libraries ahead of a mid-2026 review of New South Wales recordkeeping obligations under the State Records Act 1998.

The timing is not coincidental. With Metro West construction disrupting streetscapes from Westmead to the Bays Precinct, documenting change along the corridor has generated a surge in photographic submissions to council planning portals. Records managers at Cumberland City Council and the City of Parramatta Council have both flagged internally that their geographic information systems are holding multiple versions of the same heritage property photographs — some dating back to the early 2000s — without a consistent deduplication protocol in place.

What the experts are saying

Professionals in the field describe the problem in practical terms. Digital asset management specialists working with local government bodies note that without automated deduplication tools, staff manually cross-reference image files when preparing heritage impact statements, a process that can add several hours to what should be routine assessments. The City of Sydney's open data portal, which lists thousands of publicly accessible images under its historical photography collections, underwent a partial deduplication exercise in 2023 but practitioners say the work was incomplete.

The State Library of New South Wales, located on Macquarie Street in the CBD, holds one of the country's most significant photographic collections and has invested in deduplication software as part of its broader digitisation program. Librarians there have pointed to the scale of the challenge: even well-resourced institutions find that metadata inconsistencies — different file names, varying resolution exports of identical source images — can defeat basic algorithmic checks. Smaller councils operating on tighter IT budgets face a steeper climb.

Rick Morton, a digital records consultant who has worked with several Western Sydney councils, told a local government forum in Penrith earlier this year that the cost of storing redundant image files across NSW public sector systems runs into the hundreds of thousands of dollars annually when server, licensing and staff time are factored in — though he acknowledged comprehensive sector-wide figures remain difficult to pin down given inconsistent reporting.

Policy gap and practical pressure

State Records NSW, the regulator sitting under the Department of Customer Service, sets minimum standards for how agencies retain and dispose of records, but its current guidance does not specifically mandate deduplication audits for photographic assets. Several records managers interviewed for this article — who spoke on background because they were not authorised to make public statements — said the absence of explicit standards has left agencies to devise their own approaches, producing a patchwork of practices across the 128 local government areas in New South Wales.

The pressure is acute in growth corridors. Blacktown City Council, which covers one of the fastest-expanding local government areas in the country, has seen planning-related image submissions increase markedly since 2022 as subdivision applications multiplied across the Marsden Park and Alex Avenue precincts. Without clear deduplication workflows, duplicate images submitted by multiple applicants referencing the same street or intersection end up sitting in separate folders within the council's document management system.

For residents and planners, the practical upshot is straightforward: heritage assessments can be delayed, storage costs accrue, and freedom-of-information responses become more cumbersome when staff must manually sift through redundant files. Digital preservation advocates are pushing for NSW to adopt the approach already trialled by the Public Record Office Victoria, which published deduplication guidelines for local councils in 2024.

The next opportunity for a formal policy shift comes at the State Records NSW annual stakeholder consultation, scheduled for September 2026. Records managers and digital archivists say that if the agency does not move then, the problem will only compound as AI-assisted planning tools generate even greater volumes of imagery across the state's development corridors.