AI watermarking promises to promote transparency about human and AI-generated content, but will it create confusion and even false claims?
All text generated by Claude now has a digital signature, or watermark, showing that it was created by artificial intelligence (AI). In early August, Anthropic announced the global change to comply with the European Union AI Act, a set of rules intended to limit the risks that come with AI.
“The EU’s transparency obligations require generative AI providers to make synthetic or manipulated content machine-readable and detectable, where technically feasible,” says Deb Szabo, AI business strategist and Claude Certified Associate.
Szabo says Australian businesses using newer Claude models are receiving a transparency feature driven by European law, despite Australia choosing voluntary guidance rather than an equivalent mandatory labelling regime.
In Australia, there are no specific laws or requirements for labelling AI-generated content, although Szabo notes that the Government recommends voluntary disclosure through labels, watermarking and metadata.
“Transparency is worthwhile, but the label must tell people something useful. “AI was involved” is not the same as “AI created this”, and neither automatically means someone has been deceptive,” she adds.
How the AI watermarking works
Large language models like Claude are pattern matching systems that have been trained on reams of text to predict the next word in a sentence. Each time the model selects a word, it chooses from a list of suitable words, typically at random.
With the watermarking system, the choice of certain words is generated by a pattern or key and several preceding words. Claude uses a version of Google DeepMind Synth-ID Text approach.
David Kiriczenko, Managing Director of Australian technology consultancy, CICS, says watermarking is an attempt to establish content provenance, “giving platforms, businesses and regulators a signal that AI was involved in producing content”.
Kiriczenko expects provenance to become a much bigger part of AI governance in the next few years, but it will raise questions about the level of AI or human involvement. “The difficulty is that a watermark doesn’t answer the question people want answered: ‘Who wrote this?’.”
“A person may write an entire document and use AI only to correct grammar. Another person may ask AI to generate the entire document. Both could potentially carry an AI provenance signal. Treating those two cases as equivalent would be problematic,” he says.
There are also technical limitations. “Statistical text watermarks can survive copying and some light editing, but substantial rewriting, translation or paraphrasing can weaken or remove them. That potentially creates an arms race between watermarking systems and tools designed to circumvent them,” he says.
“This becomes particularly important in education, recruitment, journalism and professional services, where AI is increasingly being used legitimately as an editing, research and productivity tool,” he adds.
Will other AI services follow suit?
Anthropic has said that the watermarking does not affect the quality of the text and that it shows no difference to output that hasn’t had any watermarking applied. Essentially, the key is like an equation that produces a number that determines the word choice, but looking at the order of words chosen would reveal the key, and AI-created content.
OpenAI has researched text watermarking, but it hasn’t broadly deployed it in ordinary ChatGPT responses. Google already uses SynthID to watermark text generated through Gemini. Kiriczenko believes watermarking and other moves to authenticate output could become a competitive issue among AI services.
“If major regulated AI providers watermark their output while other models do not, users who specifically want to avoid detection may simply migrate to unmarked tools. That could perversely disadvantage providers that adopt stronger transparency standards,” Kiriczenko says.
He believes that watermarking should be one part of a broader provenance framework rather than treated as definitive proof of authorship. “We will likely need a combination of watermarking, cryptographically verifiable content credentials, organisational audit trails and clear policies around acceptable AI use,” he says.
“The important distinction is this: a watermark may be able to tell us that AI touched a document. It cannot tell us who had the idea, whether the information is true, or how much human judgement went into producing it,” he adds.
Is watermarking likely to introduce claims of AI mis-use?
Article 50 of the EU AI Act excludes AI performing standard assistive editing that doesn’t substantially change the user’s original input or its meaning. Szabo says that distinction matters enormously for businesses.
“If I dictate my ideas to Claude and ask it to turn them into clear copy, that’s not fundamentally different from dictating to a secretary or working with an editor. Claude assisted with the expression, but the ideas, judgement and final approval remain mine,” she says.
While a Claude watermark can indicate the likelihood that Claude was involved, it can’t prove Claude authored the work, that the person acted dishonestly or that the underlying ideas belong to AI. “An absent mark cannot prove AI was not involved either. The real business risk is people treating a watermark as evidence of misconduct,” Szabo notes.
“AI is already embedded in everyday tools used for writing, image editing, background removal, colour correction and design. We need to distinguish normal AI-assisted work from fully synthetic content created to deceive people,” she says.
In essence, AI watermarking and labelling could become far more common as the technology is embedded into more tools and systems. Rather than simply asking if AI was used, organisations — and customers — may want to understand how it was used and whether it assisted a human-created process or generated the content, text, image, video, entirely itself.
“I use it for proofreading … but everything that I write in my article is my own words, but it has been touched with AI,” Szabo says. “Just because it says AI-generated content doesn’t mean that it wasn’t my experience, my thoughts.”
Szabo says businesses need to understand how AI is being used and where it can add value, rather than treating all AI-assisted work as the same. “My biggest thing is giving capacity back to business owners,” she says.
“With all that tedious manual handling stuff that used to take forever to do, they can actually get to serving the customers, to making the wine, to being the creative artist.”
To read about the rise of AI deepfakes, click here.