Algorithmic Equivalence Space

For the last 2–3 years, frontier labs have been heavily focused on ramping up the programming abilities of LLMs because of their deterministic and verifiable nature. Hence, a sizeable part of the training corpus involves code distributed almost evenly across a number of popular programming languages. Naturally, it can be assumed that the models form certain coding abstractions that help them write code to solve a particular task. One form of such an abstraction can appear when models, in their intermediary layers, compress diverse implementations into a language-invariant, computational representation that is sensitive to algorithmic structure rather than surface form. However, measuring this trivially via a few code snippets across different algorithms is difficult, as they carry syntactic baggage (language tokens, variable identifiers, formatting) that confounds raw similarity and might misrepresent the representational space we are looking for. In this study, we mechanistically investigate evidence of this shared algorithmic space across models, how it evolves across layers, and how much of it is linearly decodable.

Curating a dataset for such an investigation can be a bit tricky. Naively curating these samples can lead to similar algorithm code snippets across languages sharing identifiable tokens—such as shared variable names, identical function signatures, or distinct input formats—which can falsely signal equivalence. We make sure that function names are neutral across all samples by assigning a common name (solve). The local variables and arguments are carefully chosen to be disjoint, meaningless nine-letter random strings. The test-input slots across different algorithms of the same task are kept consistent so as not to leak the identity of a given task. These constraints make up a highly controlled sample set of 1,600 programs = 100 task contracts × 2 distinct algorithms × 2 structural implementations (implementation variants like for/while loops) × 4 languages (Python, JavaScript, C++, Rust). With an autoregressive stream of text, it is uncertain where the model best represents the general state of the program. We consider the total mean of the token activations across the entire program and three specific token activations—the final token of the code block, the terminal token of the function’s return expression, and the query token right after the function execution call. We conduct the study on all four of these activation states to see which best captures the algorithmic space we are looking for.

There are certain metrics that we can rely on that might provide us with correlational evidence of this algorithmic space in the intermediary layers. Standard techniques like vanilla PCA do not help us much as they look for high overall variance, which is easily dominated by syntactic noise, code length, and token frequencies. We instead use contrastive PCA (cPCA) that highlights features that vary across different algorithms while remaining invariant across samples of the same algorithm, effectively suppressing syntax noise and other low-level details. We also analyze structural metrics like RSA, which evaluate how algorithm contrasts align geometrically across different languages. We perform Cross-Language Structural Alignment, where we average the two implementations (A/B) of each algorithm within a language, subtract the two algorithm averages for the same task, and compare the resulting difference vectors across languages using cosine similarity: $\cos\!\left(\bar{h}_{\text{Python},1}-\bar{h}_{\text{Python},2},\!\bar{h}_{\text{Rust},1}-\bar{h}_{\text{Rust},2}\right)$.

Looking at the graphs above, the layerwise curves across Gemma 12B, Granite 8B and Ministral 8B starts low in the early layers, climbs steadily and peaks around in the middle layers(around layers 21-28 for these models) before dropping off towards the output layers. Around the peak, the CLSA scores reach between 0.55-0.62 across all 3 of the models around 0.6-0.7 of their decoder depth. RSA scores reach between 0.53-0.58 way above the null baselines which is near 0.02. Contrastive PCA improves over the raw activations for the 20 held out tasks, lifting top 1 accuracy by +13.5 percentage points on Gemma and +10.6 points on Granite. These metrics provide preliminary correlational evidence of middle layers in the models having abstract representations of algorithms across programming languages and implementations. However, language biases and surface details still linger within the raw representations, which induces noticeable noise in our metrics and hinders a clean readout of the hypothesized algorithmic space. Can a dedicated readout probe trained across all the layers find the same trends and give us an even cleaner read out of the space in the middle layers?

In the full residual stream space of 3,840D/4,096D, the representation formed by the middle layers is often diluted by non-semantic variance like language syntax, code formatting, and token frequencies. Linear probes can help us extract and decouple the dimensions that carry the algorithmic signal cleanly while removing this irrelevant variance. We train the probes to bring activation pairs of the same algorithm closer while pushing pairs of different algorithms further apart. This proxy objective helps elicit a linear projection that isolates a cleaner algorithmic signal. Our loss objective which assigns a higher loss when pairs of same-algorithm activations are far apart and when pairs of different algorithms are close together. Crucially, we use “same task but different algorithm” samples as hard negatives, which penalizes the probe for conforming to a simple task-matching readout. To ensure the learned linear readout is not merely picking up statistical co-occurrences specific to the training languages, an entire language (Rust) is completely held out, allowing us to evaluate the probe zero-shot on unseen tasks written in this unseen language.

In order to evaluate how well the probes learnt to project clear semantics signals of algorithmic identity, we test our held out programs using two settings. Conditional two way algorithm choice tests whether a program activation is closer to the same algorithm implementation than to the same-task alternative algorithm sample. Recall@7 checks the total occurrences of the true same algorithm implementations out of all the 1599 programs. The linear probes show a good performance gain over the raw activations baselines. For Granite 4.1 8B, Recall@7 jumps from 39.8% at the best raw layer to 59.5% with the probe (+19.8 percentage points). Ministral 3 8B improves from 51.7% to 63.1% (+11.4 points), and Gemma 12B advances from 48.8% to 61.9% with a 128D projection (+13.1 points). Across all the models, we see the probe performance peak around 60-70% of the decoder depth providing clear evidence for these signals existing most prominently in the intermediary layers. The average performance gain of more than 10% suggests that the probes can filter out the noise that clutters the raw activations. The linear readout does provide a better lens that extracts these syntax invariant algorithmic signals that transfers effectively on the test set of a language it has never seen before.

These experiments provide preliminary correlational evidence for the existence of a linearly accessible algorithmic space in the intermediary layers. However, there are a few caveats to take into consideration:

1. This evidence does not provide proof that the model uses such an abstract space for its internal computation when producing the next token. In order to prove causality, performing activation patching or ablation is required to see if it has a direct effect on the model output.
2. The sample set we curated contains only 100 tasks. A lack of diversity in tasks results in poor estimation of population variance, which directly affects metrics like CKA and RSA that average over the number of tasks (or algorithm groups). Even though the held-out test set performed well, the probes were trained on all tasks (only one language was held out, but all 100 tasks were used). The probe can pick up naive statistical correlations across the training tasks that are common among the three languages we trained on, which could transfer trivially to another language as well. Increasing the total sample size and the test size by adding more tasks and more languages per algorithm should give a more conclusive result.

As models continue to be optimized for coding tasks, hypotheses similar to our experiment can provide a strong understanding of how these models compute internally. Beyond an algorithmic equivalence space, we can explore how solutions to the same tasks (algorithms) of different complexities relate to each other, and whether tasks share similar complexity curves that we could identify and steer.