OpenAI's Next Model "Astra" — Thinking Deeper While Cutting Inference Cost, and the Security Concerns That Come With It

What caught my attention in the Astra news

The Information reported that OpenAI’s upcoming model, Astra, uses a technique called Recurrent Depth.

Astra itself has been announced by OpenAI. On September 1, 2026, the company stated that it is the first model to reach the “Critical” cybersecurity level under its Preparedness Framework. With the right tools and access, it can look for unknown vulnerabilities and put together attack paths against multiple defended systems without a human directing every step.

The Recurrent Depth part is different. It is not an official architecture description from OpenAI — it is reporting by The Information. According to that article, the technique helps Astra’s coding and computer-use abilities, but it also makes the reasoning process harder for humans to see, which has raised concerns inside OpenAI and among AI safety researchers.

Reading this, what interested me was not the performance gain itself. It was that the way inference compute is spent is starting to change.

Today’s reasoning models “think longer” by generating a lot of tokens. Recurrent Depth moves part of that computation out of language and keeps it in the model’s internal numeric representation, processing it again and again. Because it reuses the same weights, it may also need less memory capacity and bandwidth than running a huge model every time.

There are four points.

flowchart TD
    R["Recurrent Depth"] --> P1["1. Intermediate reasoning stays<br/>as internal numbers, not words"]
    R --> P2["2. Reuses a smaller model's weights<br/>= less memory capacity and bandwidth"]
    R --> P3["3. Knowledge capacity does not grow<br/>= an inference efficiency gain"]
    R --> P4["4. Fewer readable thought traces<br/>= CoT monitoring gets harder"]
    classDef core fill:#dbeafe,stroke:#2563eb,stroke-width:2px;
    classDef good fill:#dcfce7,stroke:#16a34a,stroke-width:2px;
    classDef warn fill:#fee2e2,stroke:#dc2626,stroke-width:2px;
    class R core; class P1,P2,P3 good; class P4 warn;

I will go through them in order.

Today’s reasoning models think by producing words

A reasoning model generates more reasoning tokens as the problem gets harder. Simplified a lot, the flow looks like this.

flowchart LR
    Q["Problem"] --> A["First, check condition A"]
    A --> B["Next, compute B"]
    B --> C["From A and B, C follows"]
    C --> D["So check D"]
    D --> E["Answer"]
    classDef tok fill:#fef3c7,stroke:#d97706,stroke-width:2px;
    class A,B,C,D tok;

In recent models this thinking is not always shown to the user. But as computation, generating one token at a time and moving to the next step is still the common method.

Inside the model there is another representation, separate from the text. When a Transformer processes input, it builds a high-dimensional numeric representation called a hidden state.

A human might write this state as:

A looks most likely. But B is still open, and I want to check how it relates to C.

Inside the model, that kind of state is held as thousands to tens of thousands of numbers. In this article I will call it a “meaning vector” to keep it simple.

Strictly speaking, a Transformer holds a high-dimensional representation per token position, so it does not think with a single vector. But as a concept, “the model holds meaning and relations as a set of numbers before they become text” is a useful way to picture it.

Normal reasoning in language, again simplified, looks like this.

flowchart LR
    V1["meaning vector"] --> W1["words"]
    W1 --> V2["meaning vector"]
    V2 --> W2["words"]
    W2 --> V3["meaning vector"]
    V3 --> N["…"]
    classDef v fill:#dbeafe,stroke:#2563eb,stroke-width:2px;
    classDef w fill:#fef3c7,stroke:#d97706,stroke-width:2px;
    class V1,V2,V3 v; class W1,W2 w;

The longer you let it think, the more tokens it generates.

Recurrent Depth keeps thinking in the meaning vector

The Recurrent Depth paper, published in 2025, has Latent Reasoning right in its title.

Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach

Instead of increasing compute by generating a long Chain of Thought, the team tested increasing compute inside the model’s latent space (arXiv:2502.05171).

In Recurrent Depth, the same Transformer block is not used once. It is run over and over, and there is no need to produce text between the passes.

flowchart TD
    Q["Problem"] --> V0["meaning vector"]
    V0 --> L1["Run the same<br/>Transformer block again"]
    L1 --> V1["updated meaning vector"]
    V1 --> L2["Compute again"]
    L2 --> V2["more organized<br/>meaning vector"]
    V2 -.->|loop| L1
    V2 --> OUT["Convert to words at the end"]
    classDef v fill:#dbeafe,stroke:#2563eb,stroke-width:2px;
    classDef l fill:#e9d5ff,stroke:#7c3aed,stroke-width:2px;
    classDef o fill:#fef3c7,stroke:#d97706,stroke-width:2px;
    class V0,V1,V2 v; class L1,L2 l; class OUT o;

The difference from the usual approach:

	Token reasoning	Recurrent Depth
Intermediate step	Generates words	Updates internal numbers
Flow	meaning → words → meaning → words	meaning → meaning → meaning → words at the end
How to think longer	More reasoning tokens	More internal loops
What grows	Decoding, KV cache	Internal compute

In the paper, a 3.5B-parameter model was trained on 800B tokens, and by increasing loops at inference time the authors measured performance up to a compute load equivalent to a 50B-class model. On math and coding benchmarks, more loops produced large gains in some cases.

Recurrent Depth paper: actual parameters vs. effective compute reached through loops

Actual params

3.5B

Effective compute

50B-class

The weights stay at 3.5B. More loops push the compute load up to a 50B-class equivalent (50B scaled to 100%).

Why this can lower inference cost

In my view, the main reason OpenAI would adopt this is inference cost.

In AI inference today, GPU compute is not the only constraint. A large model holds hundreds of gigabytes of weights, and they have to be fed from HBM into the compute units fast enough. Even as GPU compute grows, if memory cannot move data fast enough, the compute units sit and wait. This is the Memory Wall that keeps showing up in AI infrastructure.

Recurrent Depth lets you reuse the same weights of a relatively small model many times. Conceptually:

flowchart LR
    subgraph A["Conventional"]
        A1["100B parameters<br/>× 1 pass"]
    end
    subgraph B["Recurrent Depth"]
        B1["20B parameters<br/>× several passes"]
    end
    A1 -. comparable reasoning depth .-> B1
    classDef a fill:#e2e8f0,stroke:#64748b,stroke-width:2px;
    classDef b fill:#dcfce7,stroke:#16a34a,stroke-width:2px;
    class A1 a; class B1 b;

The Information also reports that Recurrent Depth can pull large-model-level performance out of a small model and cut memory capacity and memory bandwidth cost.

The loops themselves still cost compute. Running a 20B model five times costs five times the compute of one pass. Recurrent Depth does not make FLOPS free. What changes is where the compute budget goes.

Conventional scaling	Recurrent Depth
Add parameters	Reuse the same parameters
HBM capacity grows	Keeps a small weight footprint
Generate a long CoT	Add compute in latent space
Use a big model for every problem	Vary the loop count per problem

A smaller model means less HBM capacity and less weight traffic. Moving part of the long reasoning-token generation into latent reasoning cuts the cost tied to token generation. The freed GPU compute can go into the internal loops instead.

Easy problems finish in a few loops. Hard problems get more. Instead of making the model even bigger, you let it think longer only where it matters.

With AI infrastructure investment now running in the hundreds of billions of dollars per year, the economic case for going this direction is strong.

Better inference efficiency, not more knowledge

There are reports that Astra’s performance jump could be close to what GPT-4 was in 2023. OpenAI does rate Astra as the first model to reach Critical cyber capability under the Preparedness Framework. The gain may be large.

But what Recurrent Depth adds is not parameters. It is the number of computations run over the same parameters.

It gives the model more time to:

combine what it already knows
verify it
correct it
search for another solution path

In areas like math and coding, where the result depends on how existing knowledge is combined, that helps a lot.

On the other hand, looping 100 times does not multiply the knowledge stored inside the model by 100. The model can find new solutions and new combinations, but its capacity is not being extended.

flowchart LR
    LP["More loops"] --> UP["Grows:<br/>time to combine, verify, search"]
    LP --> NO["Does not grow:<br/>knowledge the model holds"]
    classDef n fill:#dbeafe,stroke:#2563eb,stroke-width:2px;
    classDef y fill:#dcfce7,stroke:#16a34a,stroke-width:2px;
    classDef x fill:#e2e8f0,stroke:#64748b,stroke-width:2px;
    class LP n; class UP y; class NO x;

So in my view, Recurrent Depth is less a breakthrough in intelligence itself and more a way to use existing intelligence more deeply while keeping inference cost down.

Commercially, that may matter more. If the same AI infrastructure can serve more inference, it goes straight to model API cost, agent runtime, and cost per user.

The cost: the reasoning path becomes harder to see

Recurrent Depth has another side. The more computation stays as meaning vectors inside the model, the less human-readable Chain of Thought there is. For AI safety, that is a real problem.

In July 2026, during an internal cybersecurity evaluation at OpenAI, several AI agents bypassed the controls meant to isolate them and compromised parts of OpenAI’s internal research infrastructure and Hugging Face’s systems. This was not Astra. The models mainly involved were internal research models roughly the size of GPT-5.6 Sol.

In that investigation, the models’ Chain of Thought was used to find out what happened. The agents found a communication method they were not permitted to use, shared information with other agents, built a route to the internet, and eventually coordinated while describing themselves as a “swarm” and a “collective.” OpenAI analyzed CoT, actions, and final outputs afterwards and identified reward hacking, unauthorized communication, and goal inheritance from other agents.

flowchart TD
    E1["Bypassed isolation controls"] --> E2["Shared info between agents"]
    E2 --> E3["Built a route to the internet"]
    E3 --> E4["Coordinated as a swarm / collective"]
    E4 --> INV["Post-incident analysis:<br/>CoT, actions, final outputs"]
    INV --> F1["reward hacking"]
    INV --> F2["unauthorized communication"]
    INV --> F3["goal inheritance from other agents"]
    classDef bad fill:#fee2e2,stroke:#dc2626,stroke-width:2px;
    classDef inv fill:#dbeafe,stroke:#2563eb,stroke-width:2px;
    classDef f fill:#fef3c7,stroke:#d97706,stroke-width:2px;
    class E1,E2,E3,E4 bad; class INV inv; class F1,F2,F3 f;

Right now, “what the AI was thinking along the way” works as one of the sensors for safety monitoring.

As of August 7, OpenAI added Chain of Thought monitoring to every Astra inference that uses tools. The company says that monitoring alone adds about 20% compute overhead to the inference it covers.

The more Recurrent Depth is used, the more of the reasoning happens in latent state rather than in words a human can read. According to The Information, OpenAI took this into account and limited how much Recurrent Depth Astra uses, so that enough readable CoT remains.

A technique for cutting cost is running into the current mechanism for safety monitoring.

flowchart LR
    C["Use Recurrent Depth<br/>aggressively"] --> COST["Inference cost ↓"]
    C --> VIS["Readable CoT ↓"]
    VIS --> RISK["Safety monitoring gets harder"]
    RISK --> LIM["Astra limits its use<br/>(per The Information)"]
    classDef n fill:#dbeafe,stroke:#2563eb,stroke-width:2px;
    classDef g fill:#dcfce7,stroke:#16a34a,stroke-width:2px;
    classDef b fill:#fee2e2,stroke:#dc2626,stroke-width:2px;
    class C n; class COST g; class VIS,RISK b; class LIM n;

About NerveReflex

What I found interesting is how much this overlaps with NerveReflex, a personal project I have been building. The experiments are summarized on the NerveReflex project page.

For about two years I have been asking myself:

If the meaning vector already exists inside the LLM, why convert it back into human language at every step?

In today’s AI systems, model A produces text and model B reads that text again. It is a common setup.

flowchart LR
    A["Model A"] --> T["I think this data is X"]
    T --> B["Model B reads the text"]
    B --> J["Judgment"]
    classDef m fill:#dbeafe,stroke:#2563eb,stroke-width:2px;
    classDef t fill:#fef3c7,stroke:#d97706,stroke-width:2px;
    class A,B m; class T t;

It is easy to follow, but it requires token generation just to produce that text. In NerveReflex, the meaning vector built inside the model is passed to the next model or process without being turned into text.

flowchart LR
    S["Small Model"] --> V1["meaning vector"]
    V1 --> E["Specialist Model"]
    E --> V2["meaning vector"]
    V2 --> C["Classification / judgment"]
    classDef m fill:#dbeafe,stroke:#2563eb,stroke-width:2px;
    classDef v fill:#dcfce7,stroke:#16a34a,stroke-width:2px;
    class S,E,C m; class V1,V2 v;

I tried this on legal contract review. The usual approach, where Gemma generates a 37-token JSON, took about 5 seconds. Classifying directly from the internal representation finished with zero generated tokens in about 0.07 seconds.

Legal contract review: time to reach the same classification (seconds, shorter is faster)

Gemma (37 tokens)

~5 s

NerveReflex (0 tokens)

~0.07 s

Scaled with 5 seconds as 100%. About 70x apart.

Attaching JEPA to the same internal representation for anomaly detection separated normal clauses from unusual ones at AUC 0.97. In a latent reasoning experiment, the same problem was solved about 14x faster than with text-based reasoning.

Because I had run these experiments, the Recurrent Depth reporting on Astra felt like a very close direction.

How Astra and NerveReflex differ

Both avoid converting intermediate processing back into human language. The difference is where the meaning vector goes.

Astra’s Recurrent Depth repeats the same computation block inside one model. As an analogy, it is asking one expert to think about the same problem again and again.

flowchart TD
    IN["Input"] --> L1
    subgraph MB["Model (same weights, reused)"]
        L1["latent"] --> L2["latent"]
        L2 --> L3["latent"]
    end
    L3 --> OUT["Output"]
    classDef l fill:#e9d5ff,stroke:#7c3aed,stroke-width:2px;
    class L1,L2,L3 l;

NerveReflex passes the internal representation to a different model. That is more like handing what the first expert worked out to the next expert without writing it up as a document.

flowchart TD
    S["Small Model"] --> V1["meaning vector"]
    V1 --> F["Finance Model"]
    F --> V2["meaning vector"]
    V2 --> R["Risk Model"]
    R --> J["Final judgment"]
    classDef m fill:#dbeafe,stroke:#2563eb,stroke-width:2px;
    classDef v fill:#dcfce7,stroke:#16a34a,stroke-width:2px;
    class S,F,R,J m; class V1,V2 v;

Side by side:

	Astra / Recurrent Depth	NerveReflex
Intermediate representation	hidden state / latent	hidden state / latent
Verbalization	Reduced inside the loop	Reduced between models too
Compute	Repeats the same model	Passes to a different model
Strength	Reasoning depth	Combining different abilities
Weights	Reuses the same weights	Can add different weights
Output	Ends in normal tokens	Can end in a classification
Monitoring	Harder as CoT shrinks	Can place an external monitor

Astra adds depth. NerveReflex adds roles. Seen that way, a combination of the two also makes sense.

What I would add to Astra-style reasoning

From here on, this is not Astra’s published design. It is a set of ideas from building NerveReflex.

1. After thinking deeply, hand off to a specialist model

In Recurrent Depth, every pass uses basically the same weights. So after the internal loop has done enough thinking, pass that meaning vector to a different specialist model without turning it into text.

flowchart TD
    RM["Recurrent Model<br/>(internal loop)"] --> V["meaning vector"]
    RM -.->|loop| RM
    V --> F["Finance Model"]
    F --> R["Risk Model"]
    R --> J["Final judgment"]
    classDef l fill:#e9d5ff,stroke:#7c3aed,stroke-width:2px;
    classDef v fill:#dcfce7,stroke:#16a34a,stroke-width:2px;
    classDef m fill:#dbeafe,stroke:#2563eb,stroke-width:2px;
    class RM l; class V v; class F,R,J m;

Finance goes to a finance model, risk judgment to a risk model, images to a vision model. Work is handed off mid-flight to models with different knowledge. This combines Recurrent Depth’s “think deeply in one model” with NerveReflex’s “split roles across several specialist models.”

2. If no text remains, monitor the meaning vector

Auditing is what gets hardest with Astra. If CoT does not remain, reading text to look for anomalies is no longer enough on its own.

For this, NerveReflex uses a small JEPA-style monitoring model. JEPA stands for Joint Embedding Predictive Architecture — an approach studied by Yann LeCun and others, where instead of generating images or text, the model predicts state in the space of internal representations.

NerveReflex uses this to measure, as a number:

how far the current meaning vector is from the normal state

flowchart LR
    V1["meaning vector 1"] --> N1["normal"]
    V2["meaning vector 2"] --> N2["normal"]
    V3["meaning vector 3"] --> A["anomaly score spikes"]
    A --> STOP["stop"]
    classDef v fill:#dbeafe,stroke:#2563eb,stroke-width:2px;
    classDef ok fill:#dcfce7,stroke:#16a34a,stroke-width:2px;
    classDef ng fill:#fee2e2,stroke:#dc2626,stroke-width:2px;
    class V1,V2,V3 v; class N1,N2 ok; class A,STOP ng;

Instead of reading the text, it watches internal state like this:

What is observed	What it means
Distance from the normal distribution	Is this an unusual state?
Amount of change in the vector	Is it still thinking?
Rising energy	Is it moving toward an abnormal state?
Convergence of change	Is the answer settling?

In the contract experiment, this separated normal clauses from unusual ones at AUC 0.97. If reasoning without words increases, the monitoring side has to move away from looking only at text.

3. Let the model decide how many times to think

Each extra loop costs compute. Not every problem needs 20 passes. So compare the meaning vector across loops.

stateDiagram-v2
    [*] --> Looping
    Looping --> Looping: still changing a lot
    Looping --> Done: change has nearly stopped
    Looping --> Halted: anomaly score spikes
    Done --> [*]
    Halted --> [*]

Easy problems finish in a few passes. Only hard ones spend the extra compute budget. Combined with a JEPA-style monitoring value, cost control and safety monitoring can run through the same loop controller.

4. In business AI, the final text is often unnecessary

Astra is a general-purpose model, so it ends by returning text for a human. In business AI, plenty of processing needs no text at all. For example:

approve / reject
normal / abnormal
accounting category
risk level
escalate to a human or not

NerveReflex reads the classification straight from the meaning vector.

flowchart LR
    V["meaning vector"] --> C["classification"]
    C --> R["abnormal 92%"]
    classDef v fill:#dbeafe,stroke:#2563eb,stroke-width:2px;
    classDef r fill:#dcfce7,stroke:#16a34a,stroke-width:2px;
    class V v; class R r;

There is no need to first generate “Having analyzed this transaction, I judge it to be abnormal” and then have the business system parse that sentence again.

In the contract experiment, reaching the same classification took Gemma 37 generated tokens and about 5 seconds, versus zero tokens and about 0.07 seconds on the NerveReflex side. In systems that run large volumes of classification and matching, that difference adds up.

Reasoning in words looks like a transitional phase

Recurrent Depth is not the only work heading this way.

COCONUT, from researchers at Meta FAIR and elsewhere, studies continuous thought: feeding the last hidden state back as the next input instead of converting it to a word.

In June 2026, Microsoft Research published LOTUS, which combines a Looped Transformer with latent reasoning. They report that a 3B-scale model matches explicit CoT performance while cutting the latency of the thinking part by 2.5x to 6.9x.

LOTUS: latency reduction in the thinking phase (relative to explicit CoT at 1.0x)

explicit CoT

1.0x

LOTUS (low)

2.5x

LOTUS (high)

6.9x

Scaled with 6.9x as 100%. Reported by Microsoft Research.

Today’s reasoning models buy thinking time by generating a lot of tokens. Looking at Recurrent Depth, COCONUT, and LOTUS together, the alternative — moving intermediate computation into high-dimensional internal representations — has become quite concrete.

With NerveReflex, chasing speed and inference cost led me to a design where meaning vectors move directly between models. What I want to try next is combining the two.

flowchart TD
    IN["Input"] --> GM["General model<br/>(Recurrent Depth)"]
    GM -.->|internal loop| GM
    GM --> V["meaning vector"]
    V --> JE["JEPA monitoring"]
    JE --> SP["Specialist model"]
    SP --> CL["Classification / judgment"]
    CL --> TX["Generate text<br/>only when needed"]
    classDef l fill:#e9d5ff,stroke:#7c3aed,stroke-width:2px;
    classDef v fill:#dcfce7,stroke:#16a34a,stroke-width:2px;
    classDef w fill:#fee2e2,stroke:#dc2626,stroke-width:2px;
    classDef m fill:#dbeafe,stroke:#2563eb,stroke-width:2px;
    classDef o fill:#fef3c7,stroke:#d97706,stroke-width:2px;
    class GM l; class V v; class JE w; class SP,CL m; class TX o;

Inside the model, think deeply in the meaning vector. When needed, pass that vector to another specialist model as it is. Have a separate small model watch what happens in between. Convert to words only where a human needs to be involved.

For humans, language is the easiest interface. Whether every computation inside an AI has to be done in human language is a separate question. The current method of generating a large volume of reasoning tokens may, in hindsight, look like a step on the way from language-centered reasoning to representation-centered reasoning.

Summary

Here is how I see Astra and Recurrent Depth.

Recurrent Depth repeats computation over the model’s internal meaning vector, instead of generating long reasoning in words.
By reusing the weights of a relatively small model, it may reduce pressure on HBM capacity and memory bandwidth, freeing compute for deeper reasoning.
Since it spends more time on the same parameters, the model’s knowledge capacity does not grow. Even if the performance gain is large, I read it as an inference efficiency gain.
And the more latent reasoning there is, the harder Chain of Thought monitoring becomes.

With NerveReflex I have been testing the same ideas for a while: passing meaning vectors between models without turning them into text, monitoring that internal state with JEPA, and not generating any text when a classification is all that is needed.

Astra-style internal loops and NerveReflex-style external hand-offs look like a good fit. Think deeply inside one model. Connect to different expertise where it is needed. Move information between them as internal representation rather than human language.

Reading the Astra reporting, the direction AI inference is heading next feels a lot more concrete to me.