Unstructured data quality: what generative AI requires

Unstructured data quality determines what generative AI says inside an organization. Contracts, emails, minutes, legal opinions, and reports hold the knowledge that a language model lacks, and an internal assistant draws its answers from them. When the contract stored in the repository is the version from before the third amendment, the assistant reports the wrong term with the same confidence it gives a correct answer.

In February 2025, Gartner predicted that through 2026 organizations will abandon 60% of AI projects unsupported by AI-ready data. The most common technique for bringing company content to a model, retrieval-augmented generation (RAG), gives accurate answers when the retrieved documents are good, and most companies' documents are outdated, duplicated, or incomplete. That quality is addressed in stages, with people and with AI itself, starting from a defined business problem.

In RAG, the system searches the collection for passages related to a question and passes them to the model along with it, and the model writes its answer from those passages. The model does not know that an amendment exists if the search did not retrieve it, or that two documents refer to the same supplier. The collection limits the answer, however good the model is.

What makes a document high quality?

A document is fit for AI when it is relevant to the question, unique in the collection, complete, correct, and current, and when it carries the context needed to interpret it. Structured data has a dictionary that says what each field means and who is responsible for it. Documents usually have none: they sit in folders with different naming conventions, on different servers and clouds, and they were written for another purpose.

A contract shows the problem. The signed version sits with the legal department, the amendments in another folder, the service acceptance records in the requesting unit's system, and the exchange about a price adjustment in the contract manager's inbox. Each piece is correct, and none of them states on its own what the current term is or how much of the contract value remains. The ERP records the amount and the payment, while the document that explains the amount stays outside it, one of the gaps discussed in the myth of the monolithic ERP.

Connecting the pieces is also hard for the model. Generative AI struggles to tell whether "J. A. Smith" in one document is "John Smith" in another, a task known as entity resolution. In a contract collection, the same supplier appears under its legal name in one document, under its trade name in another, and only by its tax ID in a third. Documents call for the same clarity as structured data: who produces them, who uses them, who answers for their quality, how long they are kept, and who may access them.

What to do: choose one document type, such as the contract, and write on a single page who produces it, who is responsible for it, where the current version lives, and which other documents complete it.

One problem at a time, starting with the cleanest collection

The volume of documents in an organization makes it impossible to fix everything at once. Priority comes first from the value of the business problem and then from the cost of preparing the documents it requires, and a collection that is already used and checked every day goes first. At a company that services truck refrigeration units, the technicians' assistant draws on schematics for about 20 equipment models from two manufacturers. These documents are consulted again and again, use a small vocabulary, and are therefore trusted.

Assessing the collection comes before building anything. A "less is more" principle applies: absent a reason to trust a document, leave it out. Experts read a small sample in depth, looking for errors, inconsistencies, and confusing language. If the sample is poor, or if there are not enough good documents, the project is reconsidered before it consumes budget. AI agents depend on the same foundation, as the case of agentic AI in procurement shows, where a complete supplier registry comes before the first agent.

What to do: before buying a tool, pick a few contracts from the collection and ask the people who use them to mark, for each one, whether it is the current version, whether the amendments are attached, and whether the supplier is identified by its tax ID. The result tells you whether the project starts with that collection or another one.

Human curation and AI-assisted preparation

Preparation runs on two tracks. The human track starts with a glossary: a group from the units involved defines what a contract, an amendment, a statement of work, or a customer is, and records each definition. That agreement is hard to reach: an advertising agency had no common definition of a "pitch," the central document of its business.

Curation comes next: choosing the best document for each subject, tagging it with metadata, and scoring it on recency, clarity, and relevance. At an investment bank, a team of about 20 analysts scores each research report against 20 criteria. That curation began before generative AI and allowed the bank to bring its assistant for financial advisors into production sooner.

The second track is AI itself, which summarizes documents, classifies them by type, tags key data elements, finds exact and near duplicates for removal, and drafts knowledge graphs that show how one piece of information relates to another. A US auto retailer uses generative AI to translate each manufacturer's language for components and features into a standard set of descriptions, so consumers can compare cars across brands. There is a limit: AI was not good at picking the best document among several similar ones, even with a grading rubric. The division of labor leaves people to decide what needs to be done and machines to provide scale and a lower cost per document.

What to do: build a glossary of the 10 to 20 core terms in the collection with the units that use them, use AI to classify documents and remove duplicates, and have a person choose the reference version of each one.

Golden questions before production

Even with a prepared collection, the application is tested before it reaches users. One method is the "50 golden questions": the team writes the questions the system must get right, checks the answers, and fixes the collection or the application wherever it fails. The test is repeated over time, because foundation models change often, and with more rigor for regulated applications or those that affect people's well-being.

Testing also exposes contradictions in the business itself. An insurer validating its assistant for claims handling found that customer contracts, the call center, the knowledge base, and the claims department gave different answers. The company named the claims department as the owner of the answer and, from there, changed the assistant, the customer contracts, and call center training.

In a contract collection, the golden questions are the ones the team asks every week: what the current term is, how much of the contract value remains, which amendments changed the amount, which index governs price adjustments, and who signed off on the latest delivery. Each one has a correct answer and a document that supports it, which makes the check objective.

What to do: write the 50 questions with the contract managers and inspectors, each with its correct answer and supporting document, and release the assistant only when it answers every high-risk question correctly.

Quality at the source

The best way to deal with quality is to prevent errors where documents are created, and that is the long-term work. The people who write documents learn to assess them, to tag key elements as they write, and to store them in a central repository. The assistant in use feeds that improvement: each error is logged, its root cause is investigated, and the process that produced the faulty document is corrected. For critical decisions, a qualified person keeps reviewing the answer.

Brazil's Federal Court of Accounts (TCU), the body that audits federal spending, applies two of these rules in a public service. Since late July 2026, the integrated search on the court's portal, which covers about seven million documents across 14 publicly accessible categories, has offered AI-generated summaries. Each summary is stored after its first access and automatically invalidated when the document is updated, and topics that already have an official statement of their own, such as the court's binding precedents and selected case law, are excluded from summaries.

For a contract, the source is the workflow. A signed amendment is entered linked to the contract it changes, an acceptance record is linked to the delivery and the payment, and the current term and remaining balance are calculated from those records, available to anyone who needs them.

What to do: define, for each document type, what gets tagged at creation and which repository it lives in, and hold a monthly review of the assistant's errors with the owner of the process that produced each document.

Conclusion

Generative AI answers with what it finds in the organization's collection, and the quality of that collection is built piece by piece: one business problem at a time, the most trusted collection first, a glossary and curation with named owners, AI to scale the preparation, golden questions before production, and each error fixed in the process that produced it. The first step is to choose one document type and measure a sample of it.

Contracts are among the first candidates, because they concentrate deadlines, amounts, and obligations. Converge Gestão de Contratos® brings each contract's term, amendments, acceptance records, budget, and payments into a single workflow, with a traceable history and an audit trail. To assess your organization's contract collection, talk to Takter.

Sources

• Thomas H. Davenport, Roger W. Hoerl, and Thomas C. Redman, To Create Value with AI, Improve the Quality of Your Unstructured Data, Harvard Business Review, May 2025.
• Gartner, Lack of AI-Ready Data Puts AI Projects at Risk, February 2025.
• Federal Court of Accounts (TCU), Funcionalidade de IA da Pesquisa Integrada já está disponível no Portal TCU, July 2026 (in Portuguese).