**Keeping Humans in the Loop**

> Keeping humans in the loop September 18, 2026 The trajectory toward autonomy One of the clearest trends in AI is that models can handle increasingly difficult tasks autonomously 1 . Startups and venture capitalists think (and we’re also seeing) the next stage of AI is having agents own a problem end to end and deliver outcomes. We’re seeing more companies using “software factories” 2 , which means agents own software engineering while humans set the objectives. Recently, Meta launched Muse , which is a AI personal assistant that just does stuff like booking a flight for you. With AGI on the precipice and a lot of digital work getting automated, what skills will humans need to build to stay working in the future? Does proper human oversight get harder as AI automates everything? Does AI eat the world or is there room for humans in the economy? The de-skilling loop The more AI we use it seems like the fewer skills we develop. A recent study by Anthropic found that when developers used AI to learn an unfamiliar Python library. They scored an average of 50% on a follow-up quiz, compared with 67% for those working without AI. The largest gap was on debugging questions. Learning can compete with productivity. In the workplace, it seems only rational to reach for AI to complete tasks because it’s so cheap and fast. Why bother slowing down to understand it’s reasoning? In general it’s always difficult to consider long-term tradeoffs. This could create a feedback loop: as our skills weaken, we rely more on AI, giving us fewer opportunities to practice and maintain those skills. On the other hand, it’s possible that coding skills don’t matter as much anymore. AI can free us up to think more about what to build and why. The incorrect assumption The current working model in the industry is that AGI will soon own the scope of a lot of digital work and potentially a lot of physical work through robotics. Suppose p(human) is the performance of a human doing a task and p(AI) is AI’s performance. The assumption we make is that when we reach AGI then p(AI) \> p(human) ∀ tasks , but this model doesn’t address how some things are tacit knowledge based on human experiences or how subjectivity comes into play. The key distinction is that AI and humans have different capabilities and those capabilities can change over time. Relying solely on AI for thinking can lead us down an unideal path. Why care? “Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.” — Frank Herbert Once coding is fully automated what will software engineers do? What happens when we fully automate CPAs, analysts, etc.? The words “permanent underclass” 3 come to mind. Technological displacement causes pain to those displaced. It’s a problem we’re aware of but we don’t have a good answer for it. In addition to potential economic hardship, when we delegate to AI and lose a grip on the mechanics of our domain, how can we make effective decisions? Are we just the human-in-the-loop hitting the approve button? At that point who is really making the decisions? We’re seeing some of this play out with coding agents today. AI-human capability AI’s capabilities will no doubt improve rapidly over the next decade, but I also think humans won’t stay static. Reading, writing, and books played a role in human development. Tools let us progress faster and think deeper. AI is another influential tool that has tipped the human capability trajectory. For certain tasks I believe the following is true: p(AI + human) \> max(p(AI), p(human)) , meaning humans and AI working in tandem are more effective than either one alone. OpenAI’s Navier-Stokes answer is an interesting example to look at. OpenAI spun up 1...

**来源信息**
- **来源**：Hacker News：AI 热帖
- **分类**：行业
- **发布时间**：2026-09-18 23:21（北京时间）
- **原文**：[打开原文](https://roarepally.com/blog/ai-as-an-intelligence-amplifier)