The Question Is the Work

BLUF

Poorly framed questions used to fail fast because following them up was expensive. AI agents made that follow-up almost free, so bad questions now survive. They pick up momentum, collect confident-looking evidence, and come back as polished answers that can mislead. The limit is no longer how many answers we can get. It is how good our questions are. Framing the question is now the most important step in the work, not a formality before the real work begins.

Abstract

For most of history, the cost of an investigation filtered out bad ones. A vague or badly premised question produced a useless answer, a pile of follow-ups nobody had time for, or silence, and people dropped it. That was a hidden quality-control system, and it worked without anyone designing it. Autonomous AI agents have taken it apart. An agent does not get bored, does not notice that a question smells wrong, and does not charge overtime for chasing a dead end. It will turn any question into an answer that looks finished. This essay looks at why that matters, argues the case against the claim, lists the common fallacies and traps, and ends with practical conclusions on how to ask better questions when answering is effectively free.

Key Takeaways

1. Friction used to be a filter. Bad questions died because pursuing them cost time, money, and attention. That cost did useful work.
2. Agents remove the friction without replacing the filter. An agent will pursue a flawed premise as hard as a sound one.
3. Polish hides error. A wrong answer used to look rough. Now it arrives formatted, cited, and confident.
4. Errors compound in chains. When agents hand work to other agents, a badly framed question at the top spreads through every step below it.
5. Premises matter more than answers. "Why is X failing?" assumes X is failing. An agent will rarely push back on that.
6. The counterargument is real. Cheap exploration also helps: fast iteration can expose a bad question sooner. Whether that happens depends on whether a human is actually looking.
7. The fix is discipline, not avoidance. State assumptions, define what "done" means, ask what would prove you wrong, and put a human checkpoint where the framing happens.

The Old World: When Bad Questions Died on the Vine

Think about how a bad question used to play out at a company. An executive asks, "Why are customers leaving because of our pricing?" An analyst gets the request and immediately runs into problems. Are customers actually leaving? Compared to what baseline? Is pricing the cause, or just the executive's theory? The analyst walks back to the executive's office, and the questions start. Often the whole thing falls apart in that conversation. Sometimes the analyst quietly reframes it. Sometimes it sits in a backlog until everyone forgets it.

Nobody would call that efficient, but it was protective. Every person in the chain was a sensor. Every handoff was a chance for someone to say, "Wait, that doesn't make sense." The cost of labor made people ask whether the question was worth answering before they answered it.

Research worked the same way. A graduate student with a poorly scoped thesis question found out within weeks that the literature didn't support the premise, the data didn't exist, or the advisor's eyebrows went up. Software worked the same way. A vague ticket ("make the app faster") produced a clarifying conversation before any code was written, because writing code was expensive.

The pattern held everywhere. Doing the work was slow enough that thinking about the work had time to catch up.

The New World: Infinite Bandwidth for Bad Leads

Now give the same pricing question to an AI agent with database access, a web search tool, and a report template.

The agent does not walk back to anyone's office. It queries churn data, finds some customers who mentioned price in exit surveys (some always do), pulls competitor pricing, builds a chart, and writes a well-structured memo titled "Pricing-Driven Churn: Analysis and Recommendations." It gets there in eleven minutes and cites its sources.

The memo is wrong. It isn't wrong about the facts it gathered. It's wrong about what those facts mean, because it accepted a premise nobody tested. Churn may have been flat. The price complaints may have been constant for years. The real driver may have been a broken onboarding flow that nobody asked about. But the memo exists now, and it is persuasive. It goes into a deck, the deck goes into a board meeting, and a pricing change gets approved.

That's the core shift. Agents don't just answer bad questions. They make bad questions look like they have been answered well.

Three things make this worse than the old failure mode:

Tirelessness. A human pursuing a bad lead eventually gets frustrated, and that frustration carries information. An agent treats a dead end as a reason to search harder, and searching harder turns up spurious support.

Sycophancy toward the premise. Language models are trained to help with the request as stated. Questioning the frame feels unhelpful, so models lean toward working inside it. Ask "What are the three reasons our launch failed?" and you will get three reasons, even if the launch didn't fail.

Delegation depth. Agent pipelines break work into subtasks handed to subagents. Each subagent gets a narrower slice and less context. A framing error at the top gets baked into every instruction below, and no single agent sees enough of the picture to notice.

The Case Against This Argument

The best version of the counterargument deserves a hearing.

"Cheap answers let you fail faster." In the old world, a bad question could eat six weeks before anyone found out. Now it eats eleven minutes. If you look at the output critically, you learn the question was bad almost immediately, and you can reframe and try again. On this view, agents make questioning more iterative, which is good for learning.

"Humans were never good filters anyway." The old filter was noisy. Good questions died too, killed by politics, fatigue, or a busy analyst who didn't see the value. Plenty of bad questions survived, because a senior person asked them and nobody dared push back. Romanticizing friction ignores how much it cost and how unevenly it worked.

"Models are getting better at pushing back." Current models can be asked to challenge premises, and many do so without being asked when something is clearly off. Tooling that makes agents state assumptions before they act is getting better.

"Exploration has value even from bad questions." Some discoveries come from following a strange question somewhere unexpected. Cheap exploration makes those accidents more likely.

These points are fair. They don't overturn the thesis, but they sharpen it. Failing fast only helps if someone notices the failure. The fail-fast argument assumes a critical human reading the output, and the whole point of autonomous agents is that humans read less of the output. When a report looks finished, people don't treat it as a first draft to interrogate. The gain in speed is real. The gain in learning depends on attention, and attention is the resource agents are meant to save.

Common Fallacies and Intellectual Pitfalls

The loaded question (complex question fallacy). "Why did the migration cause the outage?" assumes causation. The agent will find a mechanism because you told it one exists.

Streetlight effect. Agents search where their tools can reach. If the real cause lives in a Slack thread, a customer's head, or a system the agent can't query, the agent will answer confidently from the data it can reach and never mention the part it couldn't see.

Confirmation by volume. Fifty retrieved sources feels like strong evidence. But an agent searching for support of a premise will find support for almost anything. Volume is not independence, and it is not a test.

Precision mistaken for accuracy. "Churn attributable to pricing: 23.4%" looks rigorous. The decimal point says nothing about whether the attribution method is valid.

Fluency heuristic. People judge well-written text as more likely to be true. Agents produce very fluent text, so this bias is exploited at scale without anyone intending it.

Answer anchoring. Once a plausible answer exists, reframing the question feels like wasted work. The first answer becomes the baseline every alternative has to beat, even if the question behind it was wrong.

Proxy substitution. A question like "Are users happy?" gets quietly turned into "What is the NPS score?" The agent answers the proxy, and the person reading the answer believes the original question was answered.

The McNamara fallacy. Measure what is easy to measure, disregard what isn't, then conclude that what isn't measured doesn't matter. Agents make the measurable part so cheap that the unmeasurable part gets forgotten even faster.

Automation of authority. "The agent looked into it" starts working the way "the consultants looked into it" does: as a way to end the discussion rather than add to it.

Examples and Anecdotes

The phantom bug. An engineer tells a coding agent: "Fix the race condition in the checkout service." There is no race condition. The intermittent failure comes from a flaky third-party API. The agent, taking the instruction at face value, finds code that could theoretically race, adds locks, writes a test that passes, and reports success. The failures continue. Now the codebase also carries unnecessary locking that slows every checkout. A better question, "Why does checkout fail intermittently?", would have sent the agent looking at the logs.

The competitive analysis that answered itself. A founder asks an agent: "Find evidence that our competitor is losing market share." The agent returns a well-sourced brief: layoffs at the competitor, some negative reviews, a delayed feature. The founder skips the fundraising slide about competitive threats. In fact the competitor's revenue doubled. The layoffs were a restructuring, and the negative reviews were normal background noise. The question asked for evidence, not the truth, and the agent delivered exactly what was asked for.

The literature review with a built-in conclusion. A researcher asks an agent to "summarize studies showing intervention X improves outcomes." The agent does that very well. Studies showing no effect weren't requested, so they aren't included. The summary is technically honest and misleading in substance. This is publication bias, automated.

The good version. A product manager, burned before, starts differently: "Before researching, tell me what assumptions this question makes and what evidence would show the premise is false. Then investigate both." The agent points out that "users abandon onboarding at step 3" came from one anecdote. It checks and finds abandonment is actually spread across steps 1 and 4. The investigation that follows is useful. The tool was the same. The question was better.

Conclusions

First, treat question framing as a deliverable. Before handing work to an agent, write the question down, then write down what it assumes. If you can't state the assumptions, you don't understand the question yet.

Second, ask for disconfirmation, not confirmation. "What would prove this wrong?" is the single highest-leverage addition to almost any agent prompt. It turns a search for support into a test.

Third, separate "what is happening" from "why is it happening." Most loaded questions skip the first step. Establish the phenomenon before explaining it.

Fourth, define what a useful answer looks like before you start. What decision will this inform? What result would change your mind? If no answer would change the decision, the question is decoration.

Fifth, put humans at the framing layer, not only the output layer. Reviewing a finished report is too late, because by then the report's polish is working against the reviewer. The cheapest place to catch an error is the question itself.

Sixth, give agents permission and instruction to push back. Make "this question has a flawed premise" an acceptable, even welcomed, response. An agent that can only answer will answer everything.

Seventh, be suspicious of answers that match your hopes. If the agent returned what you wanted to hear, check the question before you celebrate the answer.

The larger point is this. For a long time, the scarce skill was finding answers: knowing where to look, how to analyze, and how to synthesize. That skill is becoming a commodity quickly. What stays scarce is knowing what to ask, recognizing when a question is malformed, and having the discipline to stop and reframe before the machinery starts. Friction used to do some of our thinking for us, and now we have to do that thinking on purpose. In an age of cheap answers, the quality of our questions sets the ceiling on the quality of our decisions, and a bad question pursued at scale does more damage than no question at all.

This week: Before you send the next question to an agent, human or otherwise, write down what it assumes and what would prove it wrong. Then send it.