Transition to Neuralese Systems
AI Evolution and the Amigo Journey
Current AI models face a fundamental "token bottleneck" constraint—they must externalize their reasoning through text tokens, compressing rich multidimensional internal reasoning into a severely lossy format. Each token contains only ~17 bits of information (roughly one floating-point number), while the model's internal residual stream contains thousands of floating-point numbers. The resulting ≈1000× compression guarantees that enormous detail is discarded every time the model speaks.
The problem is compounded by the statelessness of the generation loop. After sampling a token the transformer effectively blanks its short-term memory—its only record of the prior thought is the text it just emitted. It must then reconstruct the entire latent-space context from that single extra token plus the existing prompt. Imagine a human thinker permitted to scribble one character, experience total amnesia, re-read the paper, scribble the next character, and so on. Glue logic will inevitably degrade, chains of reasoning will fray, and—when priors fill the gaps—bullshit in the Frankfurtian sense emerges.
This bottleneck forces today's systems to lean on external scaffolding like context graphs and domain-specialized agents that keep the dropped information alive outside the model's head.
1. The Token Bottleneck Era (Present)
During this period, domain specialization is not just preferable but necessary for optimal performance. Specialized agents measurably outperform generalists in specific domains despite having access to identical knowledge. This architectural advantage occurs because:
Complex Reasoning Density: Different domains (like oncology vs. psychiatry) require particularly dense, interconnected reasoning trees. When externalized through tokens, these reasoning patterns lose critical information unless the agent is specifically optimized for that domain's patterns.
Latent Space Activation Conflicts: Different domains activate fundamentally different regions of the model's latent space. Current architecture cannot efficiently switch between these activation patterns within a single forward pass, creating interference patterns that measurably reduce accuracy.
Performance Threshold Requirements: Many domains require extremely high accuracy (99%+) where even small reasoning errors could have serious consequences. Specialized agents allocate their limited token bandwidth more efficiently toward critical reasoning steps in their domain.
Regulatory Alignment: Different domains have distinct regulatory frameworks that must be addressed in domain-specific ways.
2. The Neuralese Transition (No earlier than mid-2027)
As neuralese capabilities emerge—allowing models to exchange full-bandwidth vector representations instead of single tokens—the need for specialized agents will gradually diminish. A neuralese channel looks less like "send a word, forget everything" and more like a continuous stream of thought shared between timesteps. No information is forcibly dropped, so long reasoning chains can remain intact in the substrate itself rather than being shored up with external tooling. This breakthrough will not materialize before mid-2027 because it demands radical architecture redesigns, new training curricula and heavy engineering to shuttle large matrices through time without melting GPU memory.
What Neuralese Really Means
Neuralese would fundamentally transform how AI models "think" by:
High-Dimensional Recurrence: Passing the full residual stream (several-thousand-dimensional vectors) back to earlier layers of the model, potentially transmitting over 1,000 times more information than current token-based approaches.
Continuous Thought: Creating an unbroken chain of thought where complete reasoning patterns can be maintained across multiple steps without lossy externalization.
Internal Memory: Enabling models to maintain rich internal states rather than relying on externalized tokens for "memory."
The Amigo journey is designed to navigate this transition seamlessly:
First merging specialties with similar reasoning patterns
Progressively incorporating more diverse domains as high-dimensional internal representations enable models to maintain multiple activation patterns simultaneously
Throughout both phases, metrics and evaluations should serve as the "source of truth" guiding decisions about when to specialize versus generalize, rather than theoretical assumptions about model architecture. Objective measurement frameworks and simulation-based evidence provide empirical data on actual performance differences, ensuring stability as underlying technology evolves rapidly. This metrics-driven approach is fundamental to the Amigo methodology, preventing premature generalization while maintaining readiness for architectural advances.
When a unified neuralese model can demonstrably match or exceed a domain specialist's performance, we'll transition to that architecture—but not before the evidence clearly supports such a move. This grounded approach ensures we deliver maximum value at each stage of the AI evolution journey.
Why Neuralese Isn't Here—Yet
Frontier labs are well aware of the token‑bottleneck constraint, but moving to a neuralese‑style recurrent architecture is not a simple tweak.
Architecture redesign – residual streams, attention blocks and positional encodings all have to change to support high‑dimensional recurrence.
Training inefficiency – early experiments show significantly slower convergence because the model must learn to route large vectors backward in time.
Engineering complexity – passing thousands of floats per position across layers stresses GPU memory bandwidth and breaks many existing optimization tricks.
Reduced interpretability – once internal state stops externalizing as text tokens, traditional safety and auditing tools become far less effective.
Parallel Prediction Challenges – Without neuralese, models can predict all tokens in a sequence simultaneously since inputs are predetermined (e.g., for "This is an example," the model knows what inputs generate each word). With neuralese, each token requires generating the previous token's neuralese vector first, forcing sequential prediction and reducing training efficiency.
Cost-Benefit Tradeoff – The current gains from neuralese may be limited relative to implementation costs, especially since post-training represents a small portion of the overall training process. This balance will likely shift as techniques improve and post-training becomes a larger fraction of the process.
The theoretical upside is enormous, but the implementation cost remains prohibitive for the next couple of years. Amigo's roadmap therefore follows a pragmatic "external scaffolding now, native capability later" trajectory.
Evidence Over Theory
At Amigo, we believe architectural decisions should be guided by empirical evidence rather than theoretical elegance. Our simulation + judge + agent triad generates concrete data about system performance, revealing precisely where capabilities fall short of requirements.
This empirical approach reveals that capability gaps can exist at different layers:
Agent Layer: Sometimes the core agent needs refinement in how it processes information or makes decisions
Context Framework: In other cases, the scaffolding provided by context graphs or dynamic behaviors requires adjustment
Memory Systems: Occasionally, the issue lies in how information is stored, retrieved, or processed across interactions
Foundational Model Limitations: And sometimes we identify gaps that will only be addressed by advancements in base model architecture
Rather than prematurely generalizing or making speculative investments, our evidence-based methodology:
Identifies Precise Gaps: Pinpoints exactly where performance falls short of requirements
Prioritizes Interventions: Determines which improvements will yield the greatest return on investment
Selective Reinforcement: Applies targeted reinforcement learning only where it can make a meaningful difference
Architecture Roadmapping: Maps capability developments to expected foundational model advancements
When a unified neuralese model can demonstrably match or exceed a domain specialist's performance, we'll transition to that architecture—but not before the evidence clearly supports such a move. This grounded approach ensures we deliver maximum value at each stage of the AI evolution journey.
External Scaffolding Today
Amigo circumvents current limitations through a functional composition of external systems designed to support an integrated Memory-Knowledge-Reasoning (M-K-R) cycle:
Context Graphs – topological fields that create variable‑density guidance, acting as synthetic footholds for orchestrating M-K-R in complex reasoning spaces.
Functional Memory System – layered memory that preserves critical context (Memory) across sessions, effectively supplying the high‑dimensional state and M-K-R integration points the model cannot yet keep internally.
Dynamic Behaviors with Side‑effects – latent‑space activators (Knowledge) that repeatedly prime the model in lieu of persistent internal activation, guided by Memory and shaping Reasoning.
Together these components, by facilitating a more robust M-K-R process, bend the cost‑confidence curve today while generating the data that will propel tomorrow's neuralese models, where M-K-R can be more natively supported.
Waymo vs Tesla: A Helpful Analogy
The autonomous vehicle industry provides a powerful parallel for understanding Amigo's strategic approach:
Waymo‑style (Amigo today) – relies on comprehensive external systems for reliable operation:
Multi-modal sensors (LiDAR, cameras, radar) directly measure the environment rather than inferring it
High-definition maps with centimeter-level precision provide structured guidance
Operates with Level 4 autonomy (full self-driving) within geographically constrained domains
Achieves exceptional reliability through redundant systems and external scaffolding
Prioritizes perfection within defined domains before expanding to new territories
Tesla‑style (future neuralese) – aims for less external structure with more internalized capabilities:
Camera-only system requires neural networks to infer environmental structure
Minimizes reliance on pre-existing maps for greater flexibility
Currently operates at Level 2 (requiring supervision) while working toward full autonomy
Deploys broadly with iterative improvement rather than domain-complete approaches
Accepts current limitations with the vision of making hardware future-proof through software
Amigo's approach mirrors this strategic calculus: just as Waymo vehicles deliver safe autonomous rides today while Tesla continues developing its vision-only approach, Amigo provides reliable enterprise AI now while building toward future neuralese capabilities.
This parallel illuminates why context graphs, memory systems, and domain-specialized agents aren't just temporary workarounds—they're essential scaffolding that enables reliable operation given current architectural constraints. The token bottleneck, like vision-only autonomous driving, simply cannot deliver enterprise-grade reliability today without external support systems.
Both approaches may ultimately converge, with Waymo reducing sensor requirements as vision improves and Tesla potentially adding sensors as costs decrease. Similarly, Amigo's architecture will evolve as neuralese emerges, gradually internalizing capabilities that currently require external structure.
The key insight is that Waymo ships rides today and accrues the data advantage that Tesla still needs. The same strategic calculus drives Amigo's dual‑timeline roadmap—deliver reliable value today while building toward the future.
First‑Mover Advantage
This strategic approach creates substantial competitive advantages for organizations that implement Amigo now rather than waiting for theoretical architectural perfection.
Every month of real deployment:
Generates high‑value, structured interaction data.
Expands distribution channels and trust relationships.
Refines metrics that will govern the neuralese transition.
Beyond these immediate benefits, the first-mover advantage extends to the development of critical capabilities that take time to mature:
Data Foundation Development: Building the robust data foundations necessary for sophisticated problem space simulation and judge systems requires extensive real-world interaction data, domain expertise integration, and iterative refinement.
Human-AI Collaboration Expertise: Organizations and their teams need time to develop the specialized skills required to effectively co-evolve with AI systems—learning to design, guide, and evaluate evolutionary pressures cannot be rushed.
Simulation Ecosystem Maturity: Creating effective problem space simulators and judges is a complex undertaking that requires deep understanding of domain nuances, edge cases, and evaluation criteria—accumulated experience with real-world deployment accelerates this development.
By the time native high‑dimensional recurrence is production‑ready, Amigo will possess a moat of data, metrics, operational experience, and simulation ecosystem maturity that is exceedingly hard to replicate. Organizations that delay implementation face not just a technology gap but a profound expertise deficit in the human capabilities needed to effectively guide and govern advanced AI systems.
Amigo's Strategic Focus Areas
To maximize the first-mover advantage, Amigo's next-phase objectives focus on:
Advanced Problem Space Simulation: Perfecting the creation of problem space simulators and judges, with copilots to rapidly evolve these definitions based on research insights.
Accelerated Agent Evolution: Enhancing the speed and efficiency of agent adaptation under the evolutionary pressure of simulators and judges, with an increasing percentage of recursive improvement (agents helping build agents).
Bandwidth Improvement: Expanding the memory, knowledge, and reasoning bandwidth to enable more sophisticated agent capabilities.
Our primary benchmark for success will be the speed from research insights to evolutionary chamber implementation, and the evolutionary efficiency within those chambers—metrics that directly measure our ability to transform domain understanding into effective AI systems.
Practical Path Forward
Optimize specialists today – squeeze maximum value from the current architecture.
Instrument everything – keep metrics stable across generations.
Harvest data – each interaction is future training fuel.
Transition gradually – merge domains only when unified models surpass specialised baselines.
This playbook balances immediate enterprise value with long‑term architectural readiness, ensuring our partners stay ahead through every phase of the AI revolution.
Last updated
Was this helpful?