Sydney's Duplicate Image Problem: What Happens Next and the Key Decisions Ahead

Sydney's public sector is sitting on a digital storage problem that administrators have largely ignored for years. Duplicate images — identical or near-identical files stored multiple times across different servers, platforms and databases — are costing councils and state agencies real money in storage fees, slowing document retrieval systems, and creating legal headaches when the wrong version of a planning photograph ends up in a development assessment file. The question now is who makes the call to clean it up, and how.

The issue has sharpened in 2026 for a specific reason. The NSW Government's push to accelerate housing approvals under its Transport Oriented Development program — which rezones land within 400 metres of certain train stations — has dramatically increased the volume of site photography, architectural renders and heritage assessment images flowing through council systems. More images in, processed faster, means the duplication problem compounds at speed.

Where the Bottlenecks Are Appearing

Two institutions illustrate the scale of the challenge. The State Library of NSW on Macquarie Street holds digital collections numbering in the millions of files, and librarians have flagged internally that the migration of older photographic collections to newer cloud infrastructure has left duplicate master files scattered across legacy servers. Separately, Cumberland Council — which covers suburbs from Auburn to Greystanes in Western Sydney — processes hundreds of development applications each month, many of which require multiple site photographs submitted at different stages. Without an automated deduplication layer in its Pathway DA management software, the same image can sit in three separate folders attached to a single application file.

The City of Sydney Council's digital records team, operating out of its Town Hall House offices on George Street, began a file audit in March 2026 after routine storage cost reviews flagged unexpected blowouts in its Microsoft Azure contract. No outcome from that audit has been made public yet, but the audit itself signals that even the best-resourced local government in the state is grappling with the same underlying problem.

The duplication issue is not unique to Sydney, but the city's scale makes it acute. NSW councils collectively manage hundreds of terabytes of planning-related imagery, and commercial cloud storage pricing — which can run above $20 per terabyte per month for frequently accessed data — means unchecked duplication carries a direct budget cost that lands on ratepayers.

The Decisions That Now Have to Be Made

There are three distinct choices facing institutions over the next six to twelve months, and none of them is straightforward.

The first is whether to use automated deduplication tools or manual review. Automated tools — products like NetApp ONTAP or open-source alternatives — can identify and flag duplicates at scale, but they require configuration rules specific to each organisation's file-naming conventions, and a false positive can delete a file that is legally required to be retained under the NSW State Records Act 1998.

The second decision is about governance: who owns the deduplication policy? In councils with multiple directorates — planning, engineering, communications — the same photograph of, say, a heritage item on Parramatta Road might legitimately live in three different departmental drives for different operational reasons. Collapsing those to a single master file requires a cross-directorate agreement that, in practice, often stalls without executive-level direction.

The third and most consequential choice is timing. Institutions that defer until their next major IT contract renewal risk carrying compounding costs and compounding legal exposure. The NSW Information and Privacy Commission has noted, in published guidance updated in February 2026, that retaining excessive or redundant records can itself create compliance risk under privacy law if those records contain identifiable imagery of individuals — a real consideration when site photographs capture passers-by on public streets.

For Sydneysiders watching the housing approval pipeline, the practical stakes are concrete. Delays caused by retrieval failures in DA systems — where officers cannot quickly locate the correct version of a site photo — add days to assessment timelines that are already under political scrutiny. With the Minns government under pressure to deliver approvals faster, the unglamorous work of digital housekeeping has, unexpectedly, become a marginal factor in the city's biggest policy fight.