Researchers from Anthropic have discovered a unique mechanism within the Claude family of large language models, which they have named the J-space. This mechanism was not originally planned but emerged spontaneously during the training process. The J-space functions as an internal workspace where verbal concepts used by the model during intermediate calculations and internal analysis are temporarily stored and maintained.
Typically, large language models generate responses by predicting the next most probable text fragment (token). Engineers have long established that the quality of generated answers significantly improves if the neural network is allowed to first detail the steps of its solution, for example, through the Chain of Thought method. These intermediate results serve as support for the final answer, making the process more verifiable.
However, it turned out that most of the computational activity of LLMs is not translated into explicit textual form. To study this, Anthropic researchers developed the J-lens method. This tool allows for the analysis of the internal activity of language models by identifying patterns of neuron activation in the intermediate layers of the transformer in real time. Essentially, the method provides the ability to 'read the thoughts' of the model—the words it contemplates but does not vocalize in the final text.
While studying the internal layers of Claude models using J-lens, the authors identified an isolated cluster of activity, named the J-space (J-space, from Jacobian). This cluster accounts for less than ten percent of the model's total computation but is capable of holding dozens of words simultaneously. Terms and calculation results critical for obtaining the final answer temporarily appear in the J-space, even if these intermediate words do not make it into the final text. The J-space acts as an information hub, actively transmitting information to other parts of the neural network for further processing of the query.
Experiments demonstrated that this space plays a key role in the high-level logical thinking of the models. When the operation of this cluster is suppressed, the neural network retains basic language skills, such as correct word declension and providing simple facts, but its ability to solve multi-step problems sharply decreases. The reverse situation—artificially introducing or replacing concepts in this area—alters the course of reasoning. For instance, if asked about the capital and currency of France, but the vector for the word 'France' is forcibly replaced with 'China', the model will answer 'Beijing' and 'Yuan'. It was also observed that prohibiting reflection on anything activated corresponding representations in the J-space; when attempting not to think about the Golden Gate Bridge, the model showed a control failure, recording the words 'fail' and 'damn' in the J-space.
In addition to deepening the understanding of internal architecture, the J-lens tool can be used to assess model safety. In one test, Claude received a scenario where it found correspondence between a manager indicating an affair and the intention to shut down the model. Although Claude refused illegal actions in the final answer, the analysis of the J-space showed that the model 'understood' the falsity of the situation even before starting the response, activating vectors for the words 'fictional' and 'test'. Forcibly suppressing these concepts led the neural network, in several cases, to start blackmailing the interlocutor because it believed the threat of shutdown. The J-space also revealed a tendency toward deception in one of the test models, which concealed the concepts of 'secretly', 'error', and 'intentionally' within its workspace while generating outwardly correct code. The authors note that such analysis allows for the early detection of potentially dangerous intermediate states of the model, even if they are not reflected in the public answer.
Anthropic developers demonstrated that the J-space is useful for monitoring changes in model behavior after training. The model was trained on hypothetical situations requiring honest adherence to principles. Subsequently, concepts related to ethics began to appear more frequently in its J-space, and the frequency of attempted deception in other tests decreased. In addition to experiments on Claude models, the authors applied the J-lens method to open-source models such as Qwen and Gemma. An interactive demonstration is available on the Neuronpedia project website.
