AI and Productivity: How to Navigate AI’s “Jagged Frontier”

Key Takeaways

‍

• The Jagged Frontier describes AI's uneven capabilities: the same tool that passes rigorous exams and writes production code can be surprisingly limited in other areas. 
• AI is not a linear technology. Capabilities emerge suddenly past certain scale thresholds rather than building gradually.
• Vendor dashboards measure AI adoption, not AI absorption. Token consumption and login counts cannot tell leaders whether workflows have actually changed or whether the investment is paying off.
• Shadow AI is a hidden risk. Two-thirds of personal AI account usage is work-related, meaning sensitive business data is moving through tools leadership cannot see or govern.  
• AI audits are becoming an executive imperative. Without visibility into where AI creates value and where it creates risk, governance is reactive, policies go unenforced, and ROI stays invisible.

‍

What Is the Jagged Frontier?

‍

The Jagged Frontier is a concept that describes the scaling abilities of artificial intelligence as uneven, counterintuitive, and difficult for humans to predict. Why is AI superhuman at data synthesis, coding, and process automation, but (to name one notorious example) unable to accurately count the number of r’s in ‘strawberry’?

‍

AI isn't uniformly good or bad at work. It performs brilliantly on some tasks and surprisingly poorly on others. And the boundaries are constantly shifting, making AI and productivity hard to measure. Especially for companies relying on metrics like token burn or employee self-reports as proxies for real, actionable data.

‍

As a result, the challenge facing executives isn’t whether to adopt AI. It’s understanding where AI creates the most value, and where it can be most effectively deployed to create ROI.

‍

Understanding AI’s Jagged Frontier

‍

Why AI Is Not a Linear Technology

‍

Adding features to conventional software is largely additive: each new capability has to be explicitly built.

‍

Large AI models behave differently. As you scale up training compute, data, and model size together, a model's underlying predictive performance improves steadily according to a power-law curve.

‍

What makes this trend striking is that some downstream capabilities don't track that smooth curve evenly. Certain skills appear weak or absent at smaller scales and then become reliable past a certain size, a phenomenon often called "emergence."

‍

The Surprising Tasks AI Excels at

‍

The Jagged Frontier framework imagines all possible tasks spread across a flat plane. Some tasks are inside the frontier, called a "peak” (easy for AI to do), which include:

‍

• Technical synthesis: Capable of passing rigorous exams and solving highly challenging puzzles.
  
• Pattern recognition: AI learns to identify complex relationships within that data, even when those patterns are subtle or hidden.
  
• Coding assistance: writing code snippets, debugging, explaining code, suggesting improvements, and even generating entire functions based on descriptions.

‍

The Tasks AI Still Struggles with

‍

Outside the frontier, in a "valley” (hard for AI to do) failure seems erratic and counterintuitive, such as:

‍

• Nuanced judgment: AI often fails at tasks requiring deep human intuition, contextual awareness, or subtle emotional intelligence.
  
• Consistency: LLMs can give different answers to the same question depending on small, irrelevant changes in phrasing.
  
• Explainability and transparency: some AI systems can act as “black boxes,” making it difficult to understand why a decision was made, hindering trust and accountability.

‍

The Same AI Tool Can Create Both Value and Risk

‍

AI Can Turn Average Performers Into Top Performers

‍

When used effectively, AI can democratize expertise, elevating all members of a team to a top performance tier. Meanwhile, faster onboarding and knowledge transfer mean that performance depends less on accumulated experience than on adaptability.

‍

🤖 AI works as a skill leveler
In a field experiment conducted by Harvard Business School and published in Organization Science, Boston Consulting Group (BCG) consultants using GPT-4 completed 12.2% more tasks, 25.1% faster, and with 40% higher quality. The consultants who scored the lowest when assessed at the start of the experiment had the biggest jump in their performance, 43%, when they were allowed to use AI.

‍

AI Can Also Amplify Mistakes

‍

If AI is bolted-on to a poor-quality workflows, it does not improve the output, it simply produces poor-quality work faster. Junk in, junk out. AI can confidently, and very convincingly, deliver and amplify wrong outputs.

‍

When organizations don’t have the visibility necessary to establish an operational baseline for which processes work and which don’t, they make the mistake of automating broken workflows. And once scaled, rather than seeing a positive return on their AI investment, these companies end up taking a sizable hit on their P&L.

‍

The AI Investment Paradox

‍

Despite heavy investment in AI technologies, clear productivity gains can be hard to see with standard measures. Tracking AI token consumption is still the de facto way of measuring AI usage in today's workplace. Call it the AI Investment Paradox: companies are investing heavily in AI without seeing meaningful gains in output.

‍

The strategy must shift to a deterministic evaluation of AI absorption across all facets of the business. The discovery of new AI implementation opportunities (AI-first or AI-only processes) and a clear framework for connecting AI to ultimate business outcomes (P&L impacts) is mission critical for CIOs.

‍

AI Risks Emerge When Workflows Aren’t Visible

‍

Why AI Vendor Dashboards Rarely Capture AI ROI

‍

Built-in admin analytics dashboards for tools like Claude or ChatGPT highlight core usage data: token use, number of active users, and time spent in that particular tool. In other words, they measure AI adoption.

‍

AI absorption is the real goal: when AI stops being a side tool and becomes an integral part of real workflows. This is where vendor dashboards fall short. They can’t surface the AI metrics that truly matter: time allocation shifts, co-occurrence with core systems, and sustained usage after rollout.

‍

Executives who defend AI spend with login counts and license utilization can answer "are people using it" but not "is it working." The questions a board actually asks require absorption signals.

‍

The Rise of Shadow AI

‍

‍

Shadow AI refers to the unauthorized use of AI tools or systems by employees outside the knowledge or oversight of IT and security teams.

‍

Today’s employees are likely to have personal AI accounts. According to a vendor analysis of nearly 2 million classified AI session minutes, two-thirds of use cases on those personal accounts were work related.

‍

Without the guardrails of an enterprise account in place, many users lack discernment about what type of data they feed it. Shadow AI creates risk when PII or sensitive business data are fed into an AI tool that leadership can’t see.

‍

Without Visibility, AI Governance Becomes Guesswork

‍

Without clear insight into how AI systems work, make decisions, and affect outcomes, policies go unenforced and investments go unmeasured.

‍

And if leadership can't provide stakeholders insight into how AI decisions are made, trust erodes. That lack of trust undermines AI’s effective use and governance becomes reactive rather than strategic. You can’t manage a frontier you can’t see.

‍

AI and Productivity: Why AI Audits Are Becoming an Executive Imperative

‍

How to Reveal Value Creation and Expose Hidden Risks

‍

Insightful has built a practical AI audit model for companies to measure the real world-impact of their investment. Not just whether people have access to AI tools, but whether workflows are changing, risks are being contained, and the business is capturing value before results surprise leadership.

‍

AI audits identify value by quantifying:

‍

• Cycle-time change
• Throughput
• Budget relief
• Service quality
• Revenue protection

‍

And AI audits uncover risk by revealing:

‍

• Sensitive-data exposure pathways
• Third-party tool use
• Role-based violations

‍

Visibility Is the Foundation of AI Leadership

‍

Across industries, CIOs are under pressure to navigate their own jagged frontier and justify AI spending amid limited measurable returns. To understand real workplace behavior, make data-driven decisions, and determine where human oversight remains important, AI audits are essential.

‍

To meet this challenge, Insightful customers now have access to an AI Adoption Report. Leaders can access a dedicated reporting page within Insightful's Reports section that aggregates AI tool usage data across an organization's employee base. It surfaces four coordinated views: headline AI adoption metrics, an adoption trend chart, an adoption maturity matrix, and cross-tool comparison table.

‍

Organizations that can see the jagged boundaries of AI tools clearly will make better decisions, scale successful use cases faster, and govern AI more effectively. And that translates into one AI metric that truly matters: whether your AI investment is actually paying off.

‍

🔍 Download the AI Adoption Audit Playbook to discover why your current data may be systematically misleading you.

‍

FAQs

‍

1. What is the "Jagged Frontier" in AI, and why does it affect productivity?

The Jagged Frontier describes AI’s uneven progress, where capabilities improve in sudden leaps rather than gradually. AI excels at some tasks yet struggles with others, making its impact on productivity unpredictable. Understanding this uneven landscape helps organizations identify where AI adds value and where risks arise.

‍

2. Why does AI perform well on certain tasks but poorly on others?

AI’s abilities grow nonlinearly, often requiring a critical data or model size threshold to unlock new skills. It excels in pattern recognition, coding help, and creative tasks but can struggle with nuanced judgment, consistency, and explainability, creating peaks and valleys on the Jagged Frontier.

‍

3. How can AI both boost workplace performance and amplify mistakes?

AI democratizes expertise by helping less-experienced employees perform better, accelerating task completion with higher quality. However, if AI is applied on poor workflows or data, it can quickly produce widespread errors, amplifying mistakes rather than improving outcomes.

‍

4. Why are AI audits important for organizations adopting AI?

AI audits assess where AI creates measurable business value and expose hidden risks like bias, privacy issues, and lack of transparency. They provide executives with deeper insights beyond adoption metrics, enabling responsible governance and strategic scaling of AI investments to maximize ROI and minimize risks.