On my laptop there’s a sticker: “Nice story, now show me the code.” I picked it up at a SQUER event, and it has been with me through a lot of meetings. I like the question behind it. What have we made, and does it do what we say it does?
Generative AI makes that question more useful. A marketing plan, a Python function or a strategy outline can arrive in seconds, written well enough that it feels finished. Checking it takes longer. Under time pressure, it is easy to accept the answer and move on.
A Microsoft and Carnegie Mellon study gives some evidence for that concern. The researchers surveyed 319 knowledge workers about 936 examples of using GenAI at work. Greater confidence in AI for a particular task was associated with less reported critical engagement. The study measured people’s accounts of their work; it did not establish that AI use causes a lasting loss of thinking ability. It also found that critical thinking changes shape, with more effort going into verification, integration and oversight.
That distinction matters to me. I use these tools every day for drafts, code, exploration and summaries. I want the time they save. I also want to remain able to notice when an answer is wrong, especially when it is outside the part of the subject I know well.
I have seen code submitted by developers who could not explain it, strategy documents that fell apart under basic questions, and summaries citing papers that did not exist. Each looked reasonable at first. The problem emerged when someone tried to rely on it.
Review the work before relying on it
For code, I want to see it run against relevant cases, including the awkward inputs. For an analysis, I want the sources and assumptions. A citation should lead to a real document that supports the claim beside it.
The amount of checking should reflect the consequences. A rough draft for my own use needs less scrutiny than code handling customer data or an analysis supporting a budget decision. The polished appearance of either tells me very little about how much review it needs.
AI can help with that review, too. I use it to explore alternatives or find weaknesses in an argument. I still need a way to check its conclusions against something outside the conversation.
Ask the person submitting it to explain it
“Walk me through it” is a useful code-review question. I would ask the same of an AI-assisted strategy, design or analysis. Why this approach? What assumptions does it depend on? What would make us change our mind?
If the answer is simply that the model suggested it, we have more work to do. The person responsible needs enough understanding to make a decision about the result and to recognize when they need help.
There is also something valuable in a public technical discussion: the correction, the alternative approach, the comment pointing out that an answer fails on empty input. A private chat can provide a quick solution, but it does not automatically create that shared record of scrutiny. Teams have to make room for it in reviews and documentation.
Keep practicing the parts you need to judge
I try to debug something myself sometimes, or write down an argument before asking a model to improve it. Doing so shows me where my own understanding ends. It also makes the model’s suggestions easier to evaluate.
I do not think every task needs this treatment. There are routine jobs I am happy to automate. But if understanding a subject is part of my responsibility, I need to keep working with it directly. Otherwise I risk becoming very efficient at approving work I cannot assess.
The sticker stays on the laptop. It reminds me to ask for something I can examine: running code, a source, a worked example, or an explanation someone is prepared to stand behind.
Further reading
- Microsoft and Carnegie Mellon, The impact of generative AI on critical thinking: the original survey and its limits.
- Live Science, coverage of the Microsoft and Carnegie Mellon study.
- Big Think, Is AI eroding our critical thinking?.
- ScienceDaily, Generative AI boosts individual creativity while narrowing collective diversity.
