I Used AI to Help Write This Piece. Judge the Output.
Disclosure: I used an AI assistant as a conversation partner while writing this piece. I brought the argument, the examples, and the experiences. The AI drafted text I revised heavily, helped me restructure when I asked, and pushed back when I asked it to. The thesis, the claims, the framing, and the final writing are mine.
Everyone loves to hate on AI right now.
Here’s a headline from a piece that ran in a Substack weekender yesterday: “My uncle used Claude to write my Nana’s obituary.” The writer is furious, and the headline is meant to make the reader gasp.
There’s a question about AI that almost nobody is asking right now. It’s not whether. It’s how.
Did he just sit with it and talk through what he remembered? Parse through a complicated relationship, find the words for what he couldn’t say out loud? Did he land on the detail that was true: standing at the window, waving, until the car was gone? Or did he type “write an obituary for my grandmother” and submit whatever came back?
Those are not the same thing. “Used AI” covers both. That’s the problem.
The how question is one I see very few people asking. The concerns are loud right now: job displacement, data centers, cognitive offloading, an internet drowning in slop. Those are real. But none of them tell you anything about what the uncle actually did, or whether the obituary was any good. How someone uses a tool is a different question than whether the tool should exist, and we keep flattening them into one.
How matters everywhere, not just in obituaries
My kids’ schools have unclear AI policies or none at all. Teachers tell students not to use AI, then later use AI in the classroom, with no explanation of what distinguishes one from the other. Don’t use it for the essay. Do use it for the research. Why? What’s the principle? Nobody says, because if there’s a policy at all, it was written without one. The students learn that AI is alternately forbidden and assigned, and that adults don’t know the difference. I’ve written before about doing this with my own teenager, working through a paper at the kitchen table while my laptop sat open to Claude. I haven’t fully resolved the contradiction either, and I’ve also questioned who should be doing the teaching when the schools won’t.
The same incoherence shows up in hiring. Many of the job applications I submitted this year, even tech jobs, came with a variation of the same instruction. We want your words to be your own. Do not use AI in this application. Jobs, fellowships, grants. The instruction was everywhere. I used AI on all of them. I sent more than 400 applications over a year and a half. Asking a candidate to do that volume of work without the tools at their disposal is not a measure of authenticity. It’s a measure of who has time. And it’s asking applicants not to use a productivity tool they would almost certainly be asked to use on the job.
The instruction is trying to screen for something real: the difference between someone who did the thinking and someone who didn’t. That’s a legitimate thing to want to know. But you don’t get there by asking the candidate not to use AI and ticking a box to confirm they didn’t. That approach is lazy. Ironically, it is cognitive offloading by the people doing the screening, letting a checkbox do the thinking for them. It is also ineffective, because of course people will use the tools available to them.
Better to judge the output first, and then ask how.
What the how actually looks like
Here’s how I often use AI, since I’m asking you to judge me by the output. I use it in this newsletter. It isn’t a simple prompt like “write me a piece on X.” I use it first as a conversation partner when I have a half-formed idea, and the back-and-forth becomes a process of discovery that helps me understand the situation better. I’ll talk an argument through, push back when the response is sycophantic or imprecise, and come out the other end with a cleaner and truer version of the thing I was already trying to say. Nobody asks whether you talked your ideas through with someone before you wrote them down. The difference is that the conversation partner is AI.
I use it for research. I used it to build things I couldn’t have built as easily on my own. I built an interactive Monday.com version of an Excel Gantt chart that’s changed how I manage projects. I used it on my job applications to assess fit before I invested time, to iterate on materials, and to push back when a draft read like something a machine wrote. When it did, I told the AI exactly how to rewrite it. The process is closer to editing through conversation than to receiving a finished thing. It is still work.
What this actually looks like, in any of these examples, is the same four-step loop: conversation, draft, iteration, critique.
I start with a conversation: what I’m trying to do, who it’s for, what I know about the context, what I’ve already tried. The AI drafts. I read what came back and tell it what’s wrong. The opening is generic. The third paragraph is making an argument I don’t actually believe. This sentence sounds like a machine wrote it.
The iteration is closer to instructing a junior staffer than to receiving a new draft. I name what isn’t working, point to a specific paragraph, explain why the framing falls flat, ask for a specific change. It comes back. I read again. The argument is sharper but the opening still drags, so I say so, and it tries the opening again.
Several passes in, I ask for critique: score this honestly, tell me what a careful reader would catch. On one of those cover letters, I asked whether it was being sycophantic after it scored a draft nearly perfect. It backed down, re-scored honestly, and named the weakness a real reviewer would catch. It’s the exchange that matters. The tool was correctable. I corrected it. It’s not outsourcing and it’s not cognitive offloading. It’s editing, and it’s work. It just looks different, and the output is better for it.
Would we ask someone not to use a template? Would we ask an accountant not to use Excel or Google Sheets because they’re a shortcut? Would we ask someone to do statistical analysis by hand to prove the work is theirs?
On the applications that asked me not to use AI, I disclosed that I had. A short italicized line stating that I used an AI assistant to brainstorm and draft initial structure, and that the analysis, arguments, and final writing are my own. I chose disclosure over concealment because concealment isn’t the honest posture, and because the “no AI” instruction, taken literally, pushes people toward hiding something rather than naming it.
The output tells you
Here’s the thing. You don’t have to restrict AI use, because poorly used AI shows up in the outputs. The unearned transitions. The throat-clearing openers. The vocabulary one notch fancier than how a person usually writes. The confident summary of something the writer never actually engaged with. The list of three when two would have been honest. Anyone who reads enough AI-generated text starts to recognize the cadence. It feels like the prose is performing thought rather than doing it.
Poorly used AI also shows up in what’s wrong on the page. Hallucinated sources. Statistics from the wrong year, presented as current. Citations to studies that don’t exist. This one is harder, because the writer may not catch it, and most readers can’t be expected to fact-check every claim. The AI presents wrong information with the same confidence as right information, and that’s a real tool problem, not just a user problem. I’ve written more about how AI systems break in social impact contexts — gaming, bias amplification, context collapse — and most of those failure modes also live in the outputs if you know what to look for. I’ve written about what AI is actually accelerating in the social sector, and the failure mode I see most is research done at speed without verification. A grant officer who treats AI output as a finished search instead of a starting one will chase a foundation that closed its program two years ago. A good researcher always starts with a question, not an answer. Using AI doesn’t change that. We don’t want to use AI to find evidence for a claim we already have. We want to use it to figure out what’s actually true, and then verify what comes back.
Good AI use shows up too. It shows up in the argument that’s sharper than it would have been. The research that was checked and pushed on, not accepted. The specific detail that nobody could have produced from a generic prompt, because the person sat with the tool long enough to find it. Standing at the window, waving, until the car was gone. That’s a sentence somebody had to remember. The AI didn’t know.
What to do instead
The fix isn’t to restrict AI use. It’s to ask the right questions of the right people. Three actions.
Judge the output. This is the action for anyone reviewing work. Instead of asking “did you use AI?”, ask for the work and read it closely. A hiring manager should look at whether the cover letter actually engages with the role or recites it back. A teacher should ask the student to walk them through their argument. A funder reviewing a proposal should look for the specific detail that proves somebody understood the problem before they wrote about it. The tells are visible to anyone willing to read closely. They were visible before AI existed. And don’t try to outsource the work to an AI detector. Those tools are notoriously unreliable, flag human writing as AI-generated, and miss the careful AI use you actually want to catch.
Disclose how. This is the action for anyone using AI in work that will be read or evaluated. Don’t hide it. A short, specific line is enough: I used an AI assistant to brainstorm and draft initial structure. I used it to sharpen the argument. The analysis, conclusions, and final writing are my own. That turns a hidden practice into a stated one, and it lets the reader weigh what they’re looking at. It also separates you from the person who pasted the prompt and submitted whatever came back. That person won’t disclose anything.
Cite and verify. This is the action for anyone using AI to do research. The AI will give you statistics from the wrong year and citations to studies that don’t exist, with the same confidence as the real ones. Until the tools flag their own uncertainty, the burden is on the writer to check. If you can’t cite it from a real source, don’t claim it. This is the action I want named loudest, because it’s the one most people skip.
The person who did the thinking will produce better work. The person who didn’t will produce slop, or generic prose, or confident wrongness. Those things are distinguishable. What isn’t distinguishable, and shouldn’t be penalized, is whether a thoughtful person used a tool to get there.
And maybe with the uncle, there isn’t a clean answer. Maybe the AI helped him sort through complicated feelings he couldn’t have organized on his own. Maybe it helped him grieve by giving him something to push against. Maybe it even helped him get the obituary written at all, on a deadline, when the emotional weight made the words impossible to find. I don’t know. The niece doesn’t know either. She’s furious because she imagines the worst version. But she doesn’t know which version it was.
And maybe the niece’s anger isn’t really about the AI. Grief looks for somewhere to land, and AI could simply be a convenient target. It’s a pattern I see often. The thing people are angry about and the thing they say they’re angry about don’t always match. AI is loud and unfamiliar and easy to name, and naming it is easier than sitting with the harder feelings underneath. That’s worth saying gently, because the impulse is human, and because none of what I’ve written here is meant to dismiss it.
But it does mean the question on the table should be a fair one. Go back to the uncle. Read the obituary. If the writing carries his mother in it, if the details could only have come from someone who knew her, then he sat with the AI and did the work. If the obituary could have been written about anyone, he didn’t. The writing tells you. You don’t need him to.
That’s the only question worth asking. Not whether. How. And the answer is in front of you, on the page.
The uncle is one example. The schools are another. The applications are a third. The pattern is the same in every case: we’re using a screening question that can’t tell us what we actually need to know, and the cost of getting it wrong is that we punish the people doing careful work and let the careless ones through.
AI isn’t going away. We can keep pushing blindly against it, or we can learn to use it well, cut the slop, and teach other people how to use it well too. That’s the choice on the table. The version where we ban it doesn’t exist. The version where we pretend it isn’t already in everyone’s workflow doesn’t exist either. What exists is a tool that some people are using carelessly and some people are using carefully, and a screening question that can’t distinguish between them. The people who lose out will be the ones who refused to engage with it at all.
So replace the question. Judge the output, disclose how, cite and verify. That’s the version that’s worth our time.
Anthralytic is a strategy and evaluation studio for mission-driven organizations. If you make decisions about resources in the social sector, whether or not you call yourself an evaluator, this newsletter is for you.
