Who Is Answering Your Survey?

Open survey links, ballot-stuffing, five-cent respondents — and what it costs, in both directions, to know that your data came from people.

Two thousand four hundred answers in three days

In 2023, a research team in Ottawa opened an online survey on delivery options for pregnancies affected by gestational diabetes. In seven months it drew 36 responses from patients and 37 from clinicians. On October 11 the team added a $5 coffee-shop gift card. Between October 14 and 16 it received 393 patient records and 2,047 provider records, a great many of them time-stamped between two and six in the morning, and it shut the survey down. Seven months of recruiting had produced 54 usable responses; three days of the incentive produced a dataset the team could not use.

They were not unlucky. A 2024 audit published in Frontiers in Research Metrics and Analytics examined two open-distribution surveys of California farmers and ranchers, one on climate needs and one on wildfire impacts. Fraudulent responses made up 36% of one dataset and 39% of the other. The same paper's review of the wider literature found that fraud incidence in open online surveys, historically in the 14–18% range, now runs near 80% in recently documented cases, occasionally reaching 100%. Before 2019, a typical open survey could expect more than three-quarters of its responses to be usable; the review estimates that only about one open-survey study in ten now achieves high usability.

Most of this is not science fiction. It is labour. The bulk of fraudulent responding comes from survey farms — people, often with software assistance, completing surveys at volume for the incentives — and it arrives fast: the Ottawa survey's 2,400 records landed within three days of the gift card, most of them overnight. Bots are a growing share, but the professionals were there first. For calibration, the Pew Research Center's 2020 study of managed opt-in survey panels — environments with screening, reputation systems, and paid quality controls — found 4–7% bogus respondents, against 1% in each of the two panels recruited by postal address, where identity is known. The unmanaged open link sits below all of these.

An old problem, recently industrialized

None of this began with AI. In 2009, users of the message board 4chan took over TIME's online poll for the world's most influential person. They did not merely push their candidate — the site's founder — to first place, with 16.8 million votes and an average influence rating of 90 out of 100 (the poll ranked by rating; the boxer Manny Pacquiao actually drew more votes). They arranged the entire top twenty-one so that the first letters of the names spelled out an in-joke. TIME's technical team detected and extinguished several attempts to hack the vote while it was happening. The result stood anyway.

In 2016, the UK's Natural Environment Research Council asked the public to name a polar research vessel. "Boaty McBoatface" won with 124,109 votes, more than three times its nearest rival, and the council named the ship RRS Sir David Attenborough instead — the canonical demonstration that an organization which runs an open poll it cannot defend ends up overriding its own consultation. In 2017, the US Federal Communications Commission's public comment period on net neutrality drew 22 million submissions; the New York Attorney General's investigation later concluded that around 18 million were fake, 8.5 million of them filed under the names of real people whose identities had been used without their knowledge.

The standard countermeasure on open links — a browser cookie that enforces "one response per device" — is defeated by opening a private browsing window. During the 2016 US presidential debates, coordinated groups voting in news outlets' instant polls circulated exactly that instruction.

Canada has its own entry in this genre. MyDemocracy.ca, the federal government's 2016 electoral-reform consultation, collected 383,074 completed questionnaires through an open link; only 243,057 could be used in the weighted sample, and the exercise was criticized by methodologists throughout. The general principle predates every example here: the American Association for Public Opinion Research lists presenting self-selected open polls as research among its condemned practices, because a self-selected sample has no defined population — no denominator. A response rate cannot be computed, and a margin of error cannot be estimated, at any sample size.

Five cents a respondent

What has changed recently is the economics. In November 2025, Sean Westwood at Dartmouth published a study in PNAS describing an autonomous "synthetic respondent" — a large language model driven by a prompt of about 500 words, holding a demographic persona and a memory of its own answers. Across 43,800 evaluations spanning nine different models, it passed the field's standard defences: 99.8% of 6,000 trials of standard attention checks, along with the logic puzzles and "reverse shibboleth" questions written to catch non-humans. It entered data on Qualtrics pages displaying a "protected by reCAPTCHA" badge, and it could be told to tilt a poll, or to infer the researcher's hypothesis and confirm it. A completed survey cost roughly five cents, against the $1.50 a human respondent is typically paid. Westwood then calculated what contamination at that price could do: in seven top-tier national polls from the final week of the 2024 US campaign, averaging 1,599 respondents, between 10 and 52 synthetic responses would have flipped the projected leader. The paper won PNAS's 2025 Cozzarelli Prize.

The detection side of the arms race is not keeping pace. The Frontiers audit tested 31 fraud-detection indicators against known fraud and found that no single one caught more than 60% of it without an error rate of 15–20% — that is, without discarding real participants at scale. When the researchers went back to 560 fraudulent respondents with questions about details they had given three to six months earlier, all but one answered consistently; the fraud groups, the authors conclude, keep records of their responses for months. Researchers at the Urban Institute, after adding reCAPTCHA to a survey under attack, concluded that it "caused no harm but also did little to solve our problem," because most of the bad responses were coming from people, not scripts. On June 30, 2025, Qualtrics retired RelevantID, the fraud-scoring product that had anchored commercial detection for years. And the audit's quietest finding is about collateral damage: attention checks, the field's workhorse, were passed by 83% of fraudulent respondents and 82% of genuine ones. They sort nothing, and they cost every honest respondent a step.

Two honest caveats belong here. The 99.8% figure is a demonstration of capability under adversarial testing, not a measurement of how much AI-generated data is in circulation today — and the finding has its critics. A 2026 reply by Rothschild, Barari, Buskirk, Gordon and Hillygus, which PNAS declined to publish, argues that it shows what an agent can do in isolation, not that agents are completing surveys undetected inside managed panels, and that most bad data is still produced by people. And self-reports suggest the human and machine categories are blurring: in one 2025 study, about a third of participants on a major research panel admitted using AI tools to answer open-ended questions. The problem is real and compounding; it is not yet everything.

What verification costs

The case against verifying respondents has always been friction: every step you place in front of a participant loses some of them. The evidence supports that concern — but not uniformly, and the way it varies is the useful part.

For cold traffic, the cost is real and substantial. The closest well-measured analogue to "enter your email, go to your inbox, click the link, come back" is double opt-in email confirmation, where, by email-marketing vendors' own figures, a fifth to a third of people who just typed their address never complete the click, and more on phones — industry numbers, not a study, but consistent ones. That loss compounds a base rate that is already low: passers-by with no relationship to the survey's sponsor respond sparingly even when nothing stands in their way. A researcher whose goal is maximum raw headcount on a low-stakes question is not wrong to feel that an open link serves that goal.

For known populations, the cost inverts. A 2024 study in Healthcare compared two ways the same survey reached anaesthesia practitioners at two Swiss hospitals: where department heads released staff emails, a personal invitation drew a 40.2% response; where they did not, a generic link passed through intermediaries drew 25.3%. The groups were not randomized, which is worth knowing, but the gap is large. The Cochrane systematic review of questionnaire-response methods — the largest evidence base on the subject — finds that personalization raises the odds of response (OR 1.24 in its electronic trials); what suppresses response is length and dull topics, not identification. Canada's Public Service Employee Survey, a fully authenticated, individually invited census, drew 50.5% — 186,635 respondents — in 2024. The per-person unique link, which is authentication in all but name, is not the low-response mode. It is the professional standard for populations you can enumerate, precisely because it produces more response, not less.

And in at least one well-designed study, the credential itself helped. A 2026 experiment in Public Opinion Quarterly — 161,531 people invited by post to a German online panel — varied the length and complexity of the password printed in the letter and found that longer, more complex passwords increased participation in both of its samples, employees and welfare recipients, by up to four percentage points on survey access among employees. The authors' interpretation, for the employee sample where the pattern was cleanest: a stronger password signals that confidentiality is being taken seriously. Friction, in that reading, was read as care.

The pattern across all of it: the cost of verification runs roughly inverse to the relationship between the asker and the asked. Employees, patients, program participants, panel members — people who know why they're being asked — barely notice it, or respond better for it. Strangers clicking a link on a poster notice nothing else.

Anonymous to whom?

The other long-standing objection is that honesty requires anonymity — that people disclose more when nobody knows who they are. The research on social desirability bias says something more precise: disclosure tracks perceived anonymity toward whoever could judge the answer — the employer, the clinician, the community — not toward the infrastructure carrying it. The reviews of the literature on sensitive questions, Tourangeau and Yan's above all, find that what matters is whether respondents believe the audience that could punish them can see them, and that a confidentiality promise respondents don't trust suppresses candour no matter what the system actually stores.

This distinction is not theoretical; an entire industry runs on it. Employee-engagement surveys at the largest firms are, in the standard phrase, confidential, not anonymous: every employee gets a unique link, the platform knows exactly who responded, the employer sees only aggregates. Identity held by the system, anonymity preserved toward the party who matters.

The strongest version of the same idea removes the stored identity altogether. A single-use access code, generated in advance and handed to a participant on a printed card, lets a system verify that this is an authorized participant who has not responded before — while never learning who they are. No email, no account, nothing to breach. (Hashing an email address does not accomplish this: the US Federal Trade Commission's technologists wrote in 2024 that hashed emails are not anonymous: a hash of a common identifier is trivially reversed by guess-and-check, and even unreversed it is a persistent identifier. Anonymity that survives a breach requires that the identifier never be collected.) Authentication and anonymity are not opposites on one dial. They are separate properties, and the useful configurations hold both at once.

The trade, plainly

The approach we take at Septenar follows from the evidence above, and it is easiest to state as an architectural fact: the platform has no open submission link. Every response arrives through one of three doors. An invitation link sent to a known participant, where the link itself is the credential and the participant experiences one click. A verified email address, which the researcher can choose to have verified-and-discarded, so that deduplication happens but no address is ever stored. Or a single-use access code, distributed on paper or by QR, which authorizes without identifying.

What this buys is what the open link cannot: deduplication by construction rather than by cookie; a denominator, so that "45% of 1,000 distributed codes" is a response rate where "400 responses from the internet" is not; and a bill that counts people rather than scripts, on a platform that charges per answer.

What it costs should be stated just as plainly. Where the participant is a stranger — a resident clicking a consultation link on a city's social media — the email verification step will lose people an open link would have kept, and in that setting the loss can be a meaningful fraction of an already-small number. A QR poster aimed at passers-by works better with pre-printed single-use codes than with an email gate, and better still with a person handing out the cards. And verification of the respondent does nothing about what the respondent types: an authenticated human can still paste a model's answer into a text box, which is why the defensible claim is that every response comes from a real, verified, accountable person — not that any platform can make data AI-proof.

Verification establishes who answered, not how they answered. That distinction — narrower than a guarantee, stronger than a hope — is the honest limit of what architecture can do.

Sources

Westwood, S. J. (2025). The potential existential threat of large language models to online survey research. PNAS, 122(47), e2518075122. doi:10.1073/pnas.2518075122. Awarded PNAS's 2025 Cozzarelli Prize in the behavioral and social sciences, April 2026.
Rothschild, D. M., Barari, S., Buskirk, T. D., Gordon, A., & Hillygus, D. S. (2026). Reply to Westwood: Questioning the empirical evidence that AI survey contamination is real and substantial. Preprint, SocArXiv. doi:10.31235/osf.io/ykxwj_v1. Submitted to PNAS on 27 February 2026 and declined on 22 May 2026; two of the authors work for or have advised Prolific.
Pinzón, N., Koundinya, V., Galt, R. E., et al. (2024). AI-powered fraud and the erosion of online survey integrity: an analysis of 31 fraud detection strategies. Frontiers in Research Metrics and Analytics, 9. doi:10.3389/frma.2024.1432774.
Ruby, E., Ramlawi, S., Bowie, A. C., et al. (2025). Identifying fraudulent responses in a study exploring delivery options for pregnancies impacted by gestational diabetes: lessons learned from a web-based survey. Journal of Medical Internet Research, 27, e58450. doi:10.2196/58450.
Pew Research Center (2020, February 18). Assessing the Risks to Online Polls From Bogus Respondents. pewresearch.org.
Urban Institute (2023). How to tackle fraudulent survey responses. Data@Urban. urban-institute.medium.com.
Qualtrics. Fraud Detection (support documentation; RelevantID deprecated 30 June 2025). qualtrics.com.
Zhang, S., Xu, J., & Alvero, A. J. (2025). Generative AI meets open-ended survey responses: research participant use of AI and homogenization. Sociological Methods & Research, 54(3), 1197–1242. doi:10.1177/00491241251327130.
Suppan, Suppan, Beckmann, Samer, & Savoldelli (2024). Enhancing response rates in web-based surveys: the impact of direct participant contact. Healthcare, 12(14), 1439. doi:10.3390/healthcare12141439.
Edwards, P. J., Roberts, I., Clarke, M. J., et al. (2009). Methods to increase response to postal and electronic questionnaires. Cochrane Database of Systematic Reviews, Issue 3, MR000008. doi:10.1002/14651858.MR000008.pub4. Updated in 2023 (pub5); the odds ratio quoted is the 2009 edition's.
Haas, G. C., Volkert, M., & Zins, S. (2026). Designing passwords for web survey access: the effects of password length and complexity on survey and panel recruitment. Public Opinion Quarterly, 90(2), 361–398. doi:10.1093/poq/nfag001.
Treasury Board of Canada Secretariat (2025, June). 2024 Public Service Employee Survey Results. Administered by Statistics Canada. canada.ca.
Tourangeau, R., & Yan, T. (2007). Sensitive questions in surveys. Psychological Bulletin, 133(5). doi:10.1037/0033-2909.133.5.859.
Office of the New York State Attorney General (2021, May 6). Fake Comments: How U.S. Companies & Partisans Hack Democracy to Undermine Your Voice. ag.ny.gov.
Privy Council Office (2017). MyDemocracy.ca — summary of results. canada.ca.
American Association for Public Opinion Research. Condemned Survey Practices. aapor.org.
US Federal Trade Commission, Office of Technology (2024, July 24). No, hashing still doesn't make your data anonymous. ftc.gov.

Septenar is a Canadian survey platform on which every submission is authenticated — by invitation link, verified email, or single-use anonymous access code. For how the same architecture protects participants at the other end of the sensitivity spectrum, see Protecting Participant Privacy in Survey Research.