Sebastian Raschka
ML research & education
AI researcher, author of "Machine Learning with PyTorch and Scikit-Learn". Staff ML at Lightning AI.
Recent activity
-
Researchers have developed a way to control the effort mode of large language models (LLMs), allowing them to switch between low-effort, medium-effort, and high-effort reasoning modes. This is achieved through training using reinforcement learning with verifiable rewards (RLVR), which provides a reward signal for correct and incorrect responses. AI summary
Read more → -
To set up a production-ready coding agent with a fully local stack, developers can use open-source tools and open-weight Large Language Models (LLMs) hosted through an inference engine/runtime server, leveraging local models and harnesses for tasks such as code generation, file reading, command execution, and verification. This approach offers transparency, inspectability, and cost-effectiveness, with the added benefit of full control over the local setup. AI summary
Read more → -
Here's a summary of the article for a developer/AI-ML audience: Researchers have made significant progress in LLM architecture, exploring hybrid designs that combine regular attention layers with state space layers (e.g., Nemotron 3 and Mamba-3) to improve efficiency at long contexts. Other notable papers include work on MoE capacity allocation, activation behavior, and representation geometry, which have led to the development of more efficient and scalable models. AI summary
Read more → -
Recent developments in Large Language Model (LLM) architectures, such as KV sharing, multi-head compression (mHC), and compressed attention, aim to reduce long-context costs by minimizing memory traffic and attention costs, enabling the use of longer contexts in reasoning models and agent workflows. Architectures like Gemma 4, Laguna XS.2, ZAYA1-8B, and DeepSeek V4 incorporate these techniques, including KV sharing, per-layer embeddings, attention budgeting, and compressed attention. These innovations enable the creation of larger, more efficient LLMs that can handle longer contexts without excessive memory usage. AI summary
Read more → -
To understand LLM architectures, Sebastian Raschka follows a workflow that starts with official technical reports, then moves to inspecting the config file and reference implementation of the model, especially when working with open-weight models supported by the Python transformers library. This manual process involves analyzing code and config files to gain insights into the architecture details. The goal is to learn how these architectures work, and doing so manually is considered a valuable exercise. AI summary
Read more → -
A coding agent is a layer on top of a large language model (LLM) that uses tools, memory, and repository context to make the model work better in practice. This agent harness manages context, tool use, prompts, state, and control flow to enable more effective coding tasks. AI summary
Read more → -
Modern LLMs employ various attention variants, including Multi-Head Attention (MHA), which processes input sequences by running self-attention heads in parallel with learned projections, and sparse attention, which reduces the dimensionality of the attention matrix. Hybrid architectures, such as Olmo 2, combine multiple attention variants to achieve improved performance. AI summary
Read more → -
Arcee AI's Trinity Large, released on January 27, 2026, features a 400B param Mixture-of-Experts (MoE) with 13B active parameters, incorporating alternating local:global (sliding window) attention layers, QK-Norm, and no positional embeddings, similar to SmolLM3. The model's architecture is compared to z.AI's GLM-4.5, highlighting similarities and differences. AI summary
Read more → -
Inference-time scaling is a method to improve LLM reasoning by allocating more compute and time during inference, leading to better answer quality and accuracy, with various techniques including chain-of-thought prompting, self-consistency, and rejection sampling with a verifier. These methods can be categorized into four main approaches: chain-of-thought prompting, self-consistency, best-of-N ranking, and rejection sampling. Recent work has also explored combinations of these methods and their applications in recursive language models. AI summary
Read more → -
Here's a summary of the article in 2-3 plain sentences for a developer/AI-ML audience: The year 2025 saw significant progress in large language models (LLMs) with the release of DeepSeek's R1 paper, which introduced a new approach to developing reasoning-like behavior using reinforcement learning (RL) and a novel algorithm called GRPO. This development led to the creation of reasoning variants of major open-weight and proprietary LLMs, with many researchers focusing on post-training methods to improve LLM capabilities. The RLVR and GRPO approach has the potential to unlock capabilities through scaling compute during post-training, making it a promising direction for future LLM development. AI summary
Read more →