Why AI Ingestion of Exam Content Threatens Exam Security, Assessment Validity, and Legal Defensibility
Acknowledgments:
The author thanks the following individuals for their review and feedback.
Randy Gross, CISO – CompTIA (LinkedIn)
Jill Burroughs, COO – Certiverse (LinkedIn)
Much of the legal debate over AI training data has focused on publicly available content. For certification and credential exam item banks, the legal analysis needs to be expanded to account for the unique nature of how exam item banks are protected and acquired.
Some training, retrieval, and exam-preparation systems may be ingesting content that was never public in any lawful sense: secure test items obtained through breach of confidentiality, candidate agreement violations, theft, or vendor leakage.
The certification and credentialing community has lived with exam security threats for decades: piracy sites, exam dump operations, candidate misconduct, and vendor leaks. Countermeasures include rotating items, adaptive delivery, psychometric analysis, access controls, and confidentiality agreements across development and delivery. Those measures address different threats. Their effectiveness depends on the design and operating evidence of the particular program.
AI tools can add a route by which exposed content is repackaged and used. The concern deserves investigation, but a general claim about model capability is not evidence that a particular model acquired or reproduced a secure item.
For example:
Matching beyond exact wording. A tool may connect a paraphrased question with an exposed item. Whether it does so reliably requires evidence; changing wording alone should not be assumed to restore security.
Explanations and variants. An exposed answer accompanied by an explanation can help a candidate recognize a related item. The distinction between legitimate study material and misuse of a secure item bank turns on the content, source, and use.
Reuse of a partial leak. Exposed items may be repackaged into practice material or connected to a retrieval system. A partial leak does not establish access to the whole bank, and a plausible generated question is not proof of copying.
Given all of this, AI-scale web scraping is a serious new variable in the ever-evolving game of “whack-a-mole” in protecting and preserving exam integrity. Exam dump sites, unauthorized question repositories, and piracy platforms are visible on the web, but that does not make the content lawfully public. In some cases, that content may be used to build commercial exam preparation, tutoring, or assessment products that undermine the credential ecosystems those items are supposed to protect.
While operational countermeasures remain important, legal protection also deserves attention. Copyright, trade secret law, and related claims may provide remedies. The distinctive argument is that unauthorized acquisition and exposure can damage the assessment value of secure items. That is a theory to support with evidence, not an established rule that every exam-bank claim is stronger than other AI litigation.
Secure exam content is not ordinary scraped web content, and credentialing organizations should not let the AI debate be framed as if it is.
The Exam Security Landscape
Exam Dump Sites Are Not a New Problem
Exam dump sites publish unauthorized test questions, answer keys, and other exam content, often marketed as “real exam questions,” “actual tests,” or “verified answers.” They have existed since computer-based testing became widespread. Organizations like CompTIA (where I served as General Counsel), Cisco, PMI, Microsoft, and other major credentialing organizations have spent years fighting them: DMCA takedowns, litigation, candidate sanctions, item invalidation, and ongoing psychometric monitoring.
For a particular site, the organization needs to establish whether the material is actual exam content, an unauthorized reconstruction, or independently created practice material. Candidate agreements, vendor access, and item-development records can help explain how secure content was held and how it may have escaped. Retired content is not necessarily content the owner authorized for public use.
The exam security community refers to unauthorized exposure as compromise. Exposure can require an item to be retired, restricted, or investigated because prior access may affect what a candidate response measures. The appropriate response depends on the extent of exposure and the program’s security and psychometric evidence.
What Makes Secure Exam Content Different
Secure exam content is not ordinary published content. It differs from a textbook, a journal article, or even a proprietary database in several important ways.
Development requires a record of investment. Item writing, expert review, editing, piloting, statistical analysis, and translation can all contribute to cost. A damages analysis needs the organization’s actual development and replacement evidence, rather than a universal cost-per-item assumption.
Secrecy supports assessment value. Advance access can undermine an item’s usefulness in measuring competence. The consequence depends on who received the content, how broadly it spread, and whether the item remains in use. Exposure should not be equated automatically with total loss of all legal protection or economic value.
It is tightly access controlled. Credentialing organizations build layered confidentiality structures around item banks: restricted access systems, role-based permissions, confidentiality agreements for SMEs, NDAs for vendors, proctoring protocols, and candidate agreements that explicitly prohibit reproduction.
Harm can reach the program. An organization may need to investigate affected results, replace items, or change delivery. Those consequences should be supported by evidence connecting the exposure to the response. Candidate confidence and public protection can matter, but they should not be asserted as proven losses without a factual basis.
Accreditation and assessment validity
ISO/IEC 17024 addresses bodies operating certification of persons and the development and maintenance of certification schemes. Accreditation obligations concern the certification body; they do not themselves impose a legal duty on an unrelated AI company.
A body relying on secure assessments may need to explain how a compromise affects the validity and integrity of its results. The applicable standard, accreditation arrangements, and transition requirements need to be checked for that body. A legal claim against a third party does not answer whether the organization can continue using affected items.
Why AI Scraping Arguments Do Not Fit Cleanly for Secure Exam Items
Many AI model developers have trained on massive corpora that include web-scraped content. The standard justification offered is that the web is publicly accessible and that training on it constitutes transformative fair use. While courts and legislatures consider those arguments in the context of publicly available information, they do not extend cleanly to exam dump content. At minimum, AI companies with meaningful diligence processes should identify the sources that are legally and ethically different from ordinary public web pages. Many of the major offenders are not hard for the credentialing community to identify.
And this is not limited to foundation model training. The more immediate risk may come from downstream exam-preparation products, AI tutors, and retrieval-augmented study tools. Some products are marketed expressly for certification study and use AI to explain practice questions, generate rationales, identify weak areas, or create study plans. Other tools allow users or vendors to upload PDFs, scraped web pages, question sets, or “practice exams,” which are then chunked, embedded, indexed, and retrieved in response to candidate prompts. When an AI exam-preparation system is built on or connected to dump-site content, it can transform compromised secure items into explanations, flashcards, practice tests, tutoring sessions, and answer rationales. At that point, the product is not just helping candidates study. It is operationalizing the compromise.
Why the Training Pipeline Matters
AI companies and other commentators often describe training as if it were a single act: the model “learns” from data. That framing obscures what is actually happening.
Before secure exam content can influence an AI system, it usually has to be collected, copied, stored, cleaned, indexed, embedded, or otherwise routed into a training or retrieval process. That matters because the legal and evidentiary questions are not limited to what the model outputs. They also include how the content was acquired, retained, processed, and commercially used. The Copyright Office has recognized this, noting that AI development can involve multiple potentially infringing acts across data collection, curation, training, and output generation, including downloading, storing, and creating intermediate copies. The AI development cycle is a sequence of distinct acts, not a single moment of learning, and that distinction matters for how liability and evidence are analyzed.
Legal Considerations
Copyright infringement and trade secret misappropriation present different questions. Fair use may be asserted against copyright claims; secrecy, reasonable protective measures, improper acquisition or use, and the defendant’s knowledge matter to a trade secret claim. The facts supporting one theory do not automatically establish the other.
Trade Secret Misappropriation
Copyright protects expression. Trade secret law protects information. For a secure item bank, trade secret analysis may reach the core injury more directly because it addresses the confidentiality violation itself.
Access controls, confidentiality obligations, and evidence of value from secrecy can support a trade secret analysis. The route by which a particular item reached a dump site matters. So do the alleged user’s knowledge and conduct. Under 18 U.S.C. Section 1839, protection and misappropriation have defined elements. Broad public exposure can affect whether information remains secret; it does not necessarily erase a claim arising from earlier misconduct. The timing and extent of disclosure require separate analysis.
The fact that a model does not reproduce items verbatim should not end the trade secret analysis. A company may use misappropriated item-bank content by incorporating it into a training corpus, retrieval database, evaluation benchmark, or fine-tuning set. How that theory applies will depend on the evidence and the governing law.
Copyright Infringement
An exam item may contain protectable expression in its wording, scenario, answer choices, or diagrams, while facts, ideas, and functional elements receive different treatment. 37 C.F.R. Section 202.13 provides a group registration route for eligible secure test items. Conditions include preparation for a qualifying secure test, separately copyrightable items, the same author or authors and claimant, and publication-status requirements. The route is not automatically available to every question bank.
The key copying may not happen only at the output stage. Downloading, storing, processing, and indexing content for training or retrieval can implicate the copyright owner’s reproduction right at multiple points in the pipeline, regardless of what the model ultimately outputs.
Fair use requires the source, use, and harm to be examined
Public accessibility does not establish permission. It also does not, by itself, defeat or establish a fair-use defense. The analysis examines the statutory factors as applied to the particular conduct.
The Copyright Office’s May 2025 pre-publication report on generative-AI training considers acquisition, intermediate copying, training, and outputs as potentially distinct acts. It is a policy and legal analysis, not a court holding resolving secure-item-bank claims.
For a credentialing organization, the evidence should connect the protected expression, the challenged use, and the alleged harm. Item replacement costs and impairment of an assessment may be relevant to the broader dispute, but should not simply be substituted for the copyright market-harm inquiry. Removal of ownership information can raise additional issues only if the elements of the applicable claim are met.
The relevant conduct may involve a model developer, a data supplier, a retrieval service, or an exam-preparation product. Identifying the actor and the actual use is more useful than assuming all AI services operate through the same pipeline.
What an effective legal response needs
A credentialing body does not need to wait for a definitive AI-training case to examine its rights and evidence. It does need to distinguish a suspected similarity from proof of acquisition or use.
Ownership and registration. Author and contractor records support the chain of rights. Registration timing can affect enforcement and remedies under 17 U.S.C. Sections 411 and 412. The secure-test procedure has eligibility and deposit requirements that need to be assessed before assuming a group filing is available.
The security record. Actual access controls, confidentiality terms, distribution records, and enforcement practices help establish what was protected and how. A policy that was never followed is weaker evidence than a contemporaneous record of the controls in use.
Controlled evidence work. Counsel, exam-security specialists, and appropriate technical experts can assess suspicious content while protecting the item bank. Uploading live items into an ordinary public AI account to test a suspicion can itself create further exposure. Evidence collection should avoid that result and preserve enough context to evaluate what an output does and does not prove.
Assessment consequences. The program still needs a decision about affected items and results. The legal theory, security response, and psychometric evidence should inform one another without treating any one as a substitute for the others.
The strongest response rests on the organization’s actual rights, controls, acquisition evidence, and demonstrable harm. Those facts make the claim assessable and help the credentialing body decide what it can continue to deliver.
About the Author
Daniel Liutikas is an attorney with more than 25 years of experience representing nonprofit and tax-exempt organizations including trade associations, professional societies, and credentialing bodies. He is the founder of Org Law, PLLC. Contact Org Law to discuss a credentialing legal review.
