Looped Transformers reuse the same layers multiple times by updating the hidden state during each pass. This approach allows the model to perform more work without needing new weights for every additional step. The core concept involves feeding a hidden representation through the same block repeatedly in a loop. Key concepts include weight sharing, recurrent depth, and adaptive depth for optimizing performance. Understanding this architecture helps determine when an extra pass is beneficial.
UK public bodies are facing an "AI clog" due to an onslaught of chatbot-generated complaints, requests, and legal challenges. Human employees are spending significant time sifting through this correspondence, which is often complex, lengthy, and sometimes inaccurate. This phenomenon, labeled "agentic flooding," is overwhelming local authorities, MPs' offices, and other public services with surging workloads. While AI can facilitate quick communication, it also introduces issues with quality, legal accuracy, and potential misdirection in public requests. Consequently, some bodies are exploring AI for internal processing or considering stricter rules to manage the flood.
US midterm election results have sparked a social and political reconsideration of AI, data centers, and the "broligarchy." Voters expressed concern over the unchecked expansion of data center infrastructure and the societal impact of AI. This sentiment has led to measures calling for limits on datacenter expansion. The backlash has punctured narratives sustaining the AI market boom, suggesting a potential market unwind. Ultimately, opposition is likely to slow down investment in AI infrastructure.
Datasette pages serve an HTTP link with `rel="next"` headers, which triggers an HTTP prefetch for the next page of rows. This practice is inefficient because a small fraction of visitors click the "next" link, causing unnecessary data fetching. This behavior stems from an issue in Firefox that has been open for two decades. Serving these link headers to Firefox should be avoided to prevent unnecessary resource consumption.
A revenue-weighted feedback engine is proposed to prioritize product feedback by incorporating the Annual Recurring Revenue (ARR) of associated accounts. This method addresses the bias in standard feedback tools, which often overrepresent volume rather than financial impact. The scoring is calculated by weighting themes based on the ARR of the accounts that raised them, factoring in context from customer interactions. The process involves defining what revenue at risk means and designing a ranking that can invert order, not just shade it. Experimentation showed that losing account IDs impacts perceived metrics, highlighting the necessity of attributing feedback to specific accounts. Ultimately, the method allows for a roadmap that balances the volume of feedback with the financial risk associated with specific customer segments.
Midterm elections and voter rejection of data center ballot measures are creating roadblocks for the tech industry. This political shift is also fueling a growing grassroots concern about data centers, especially regarding environmental and water scarcity issues. Companies like OpenAI are responding by proposing changes to policies, such as rebranding data centers as "compute campuses." There is also speculation that financial incentives from Wall Street may discourage the massive investment in building new data centers. Ultimately, the political climate suggests future challenges for hyperscalers and data center expansion.
The US government approved sales of Nvidia's AI chips to Chinese firms connected to blacklisted companies. Concerns have been raised regarding the due diligence performed by the Commerce Department and Nvidia in vetting these Chinese entities. One approved firm, New Matrix, has empty offices with little verifiable online presence. The owners of New Matrix have ties to a Chinese private equity firm blacklisted by the US government since 2024. This situation highlights a troubling pattern of approving sales to entities linked to national-security trade restrictions.
Anthropic is proceeding with its pre-Thanksgiving IPO despite a recent tech selloff, accepting a lower valuation. The company aims to raise $80–100 billion based on a valuation between $1.25 and $1.75 trillion. Annualized reported revenue is now approximately $81 billion, an increase from about $65 billion at the end of July. This action occurs amid rising borrowing costs and heavy AI spending plans that caused a tech selloff. OpenAI also adjusted its funding targets downward recently.
Test-Time Training (TTT) is an ML technique where a model continues to learn and adapt while being used for inference. This allows models to fine-tune themselves on test data, improving accuracy and handling distribution shifts. TTT works by creating a self-supervised auxiliary task from new data, allowing the model to update some of its parameters based on its mistakes. This process blurs the line between training and inference, moving toward active learning. TTT is particularly beneficial for world models that need to adapt quickly to new states and environments.
Wall Street experienced a selloff as investors digested concerns about the infrastructure buildout for AI, showing that optimism does not dictate current infrastructure pricing. Google released Gemini Omni 1.2 Flash, a video generation model with significant price cuts, demonstrating advancements in cost-effective media creation. OpenAI introduced GPT-6 Cyber, a specialized model focused on security vulnerability discovery and patching. Thinking Machines Lab released Inkling 2 Small, a small, adaptable model, highlighting progress in making AI more specialized and usable. These developments suggest a shift toward practical applications and a re-evaluation of investment risks within the AI sector.
Attention mechanisms allow AI models to dynamically weigh the importance of different tokens when processing context and generating text. These mechanisms fundamentally enable models to understand relationships and context by focusing on relevant information. Key foundational types include self-attention for sequence relationships and cross-attention for integrating information across different sequences. Efficient variants like FlashAttention and Linear Attention improve computational speed for long sequences. Various specialized types such as GQA and MLA further optimize attention for large-scale LLM inference.
Prefill-to-Autoregressive Cross-model Transfer (PACT) combines a larger model's prompt with a smaller model's generation to balance capability and cost. The authors introduced PLatour, a framework using lightweight linear bridges to connect frozen models and improve generation performance. Experiments showed PACT outperforming baselines, but gains could be explained by uninformative signals. A new method, PACT-Check, is proposed to quantify the genuine contribution of communication in PACT systems. The paper concludes by discussing the opportunities and challenges associated with PACT approaches.
Google has open-sourced the Elephant-Goldfish Model (EGM) plugin, a design-and-review layer for AI coding agents. This system has an "Elephant" that drafts a specification, which is then reviewed by stateless "Goldfish" subagents. The review process involves five sequential stages to check the specification for clarity, design, and implementation readiness. EGM supports various coding agents like Codex and Claude Code, with test runs showing successful performance. The key innovation is forcing independent review of the specification before code generation.
Open Mixture-of-Experts (MoE) models utilize token-choice routing, where each token selects its own experts based on five routing knobs. Models vary significantly in routing details, including expert counts (8 to 384) and top-k settings (1 to 8). Some models, like the DeepSeek family, employ scoring functions such as Sigmoid or sqrtsoftplus with bias balancing for routing. While configurations offer some auditability, specific gating and balancing methods are often undisclosed in public documentation. The trend suggests that increasing experts and reducing top-k can affect memory usage and communication overhead.
The guide details a process to analyze and predict the computational cost of running the NVIDIA Dory 4B model using llama.cpp with loop variations. It involves setting up a specific llama.cpp fork, downloading a pinned GGUF file, and reading the model's architecture from the file. The process then predicts the cost in parameters and KV-cache size for different loop counts and cache modes before execution. Finally, it runs a server to measure actual decode speed and KV memory usage across these configurations to validate the predictions.
Four models—Timestral, t0, Tabby, and Morpheus-2.0—were compared on the GIFT-Eval benchmark for forecasting. A common recipe involves scaling numerical inputs, patching them, using transformers, and outputting quantile predictions. Architectural differences lie in how the models handle cross-series information, with Morpheus mixing information in all layers, Timestral and t0 using selected layers, and Tabby treating series independently. Training strategies, particularly synthetic data, also significantly impact performance, and deployment constraints matter for real-world application.
A project successfully simulated Rocket League physics for reinforcement learning using GPU acceleration. The simulation achieved 100% accuracy tick-by-tick across 144 recorded matches. It demonstrated significant speed and scalability, handling up to 16,384 arenas with high performance on various GPUs. This simulator was used to train a goal/concede-only reward policy that achieved pro human level play. The work utilized a custom PPO framework and a large attention model to train the policy.
Double DQN is an off-policy, model-free, and value-based method within the DQN family, differing from standard DQNs in its optimization process. This modification addresses the problem of Q-value overestimation caused by the max operator in the Target Q Network. This overestimation leads to false positive action selections because the network is still learning. The core difference lies in how the action selection is performed relative to the value estimation. The author provides web-based notes covering the theory, concepts, code, and math for understanding Double DQN.
A project called Mikoshi HQ500M offers a brain-inspired system with 500 million virtual neuron addresses for experimentation. It allows users to explore procedurally connected neural spaces and run small spiking networks on a standard CPU without requiring expensive GPUs. Users can connect sensory inputs to simulations, experiment with reward-driven adaptation, and modify the system's behavior. This is a lightweight playground based on ideas from larger private research, not a reconstructed human connectome or a conscious model. The project is open source, encouraging community experimentation with brain-inspired computational architectures.
Zero Front is an environment designed for training and playing agents in a competitive First-Person Shooter (FPS) team game. It employs a reduced pixel count viewport and a limited color palette to facilitate learning within a pixel space. The environment includes pre-trained weights resulting from an hour of self-play reinforcement learning, making it relatively competitive. The project aims to be accessible for training AI models. The author encourages further training to improve the resulting models.
METARGON is a small, crow-inspired agent designed to learn hidden rules by testing hypotheses through action and adapting its behavior. The agent uses bounded memory, explicit hypotheses (predict, verify, contradict), and an adaptive planner. Experiments on a 7x5 grid with noisy observations showed METARGON outperformed random and Q-learning, but performed worse than the Bayesian planner. Specifically, METARGON achieved 31.3% against Bayes in novel settings and 35.1% against Bayes in stable settings. This result suggests that hypothesis composition alone does not allow the agent to surpass the family-knowing Bayesian planner.
The author is exploring the feasibility of using Reinforcement Learning to control a game via a physical controller. The goal is to determine if an AI can successfully play complex single-player or multiplayer games in this manner. The author questions whether this approach is achievable or a waste of time for someone new to Reinforcement Learning. The interest stems from prior experience in robotics and general AI. Ultimately, the post seeks an assessment of the attainability of this specific control framework.
The NYT Cooking app now features an AI assistant that helps users customize recipes through ingredient swaps, scaling, and problem-solving. The assistant is trained on user comments and is designed to provide practical guidance while ensuring modifications do not result in bad-tasting outcomes. Users can prompt the assistant with specific requests, such as substituting ingredients or changing recipe formats. While the AI proved helpful for saving time and handling ingredient dilemmas, the author noted it almost ruined a cake by miscalculating butter amounts. The assistant works best when users cross-reference its suggestions with the original recipe for accuracy.
Richard Sutton, a Turing Award winner, asserts that raw compute is insufficient for intelligence, emphasizing the need for new methods beyond simply scaling large language models. He previously criticized synthetic data and advocated for agents with continuously learning world models to handle the world's complexity. Researchers suggest that the importance of algorithms is amplified as compute increases, indicating that scaling alone is not the sole path to progress. While some industry leaders favor LLM scaling, others, like researchers, focus on developing novel learning algorithms. This tension exists as new labs explore gaps beyond current scaling methods.
A visit to China's Galaxy datacenter in Inner Mongolia revealed astonishing speed, scale, and vertical integration compared to US builds like Stargate. Galaxy's initial 120MW phase was delivered in one year at a fraction of US costs, demonstrating low-cost construction. The facility employs a "source-grid-load-storage" model, using local wind power and integrated battery storage. Envision Group, the owner, vertically integrates by manufacturing components and designing power systems. This model is reshaping the datacenter market by prioritizing energy access and power architecture.
PDD Kit uses formal methods to turn developer intent into a formal specification, which is then used to mathematically prove that the generated code satisfies that specification for every input. This process involves creating a "plan" that bridges user intent with formal specs and informal tests. PDD Kit separates verification into formal verification, which uses a verifier like Verus, and empirical verification, which uses tests to check aspects proofs cannot cover. The process guides the development by explicitly defining the Trusted Computing Base and requires developers to explicitly sign off on assumptions. Ultimately, PDD Kit automates the process of ensuring code correctness by linking requirements, proofs, and tests.
Nous Research released Arete-27B, a supervised fine-tune of Qwen3.8-27B, designed to remove Chinese state framing. Evaluation showed the propaganda rate decreased significantly from 22% to 0.9% based on an 803-prompt assessment. This fine-tuning required only 52.8 million tokens of supervised fine-tuning, without reinforcement learning. The model's fact score improved to 2.98 out of 5, with improvements seen in freedom of expression. The model weights, evaluation, and training data are publicly available under Apache 2.0.
Tom Gauld discussed prompts for a cartoon focusing on artificial intelligence and writing. He explored the creative starting points for such a piece. The discussion likely involved brainstorming ideas related to AI concepts and literary themes. Gauld's input provided direction for developing the cartoon concept. The focus was on the conceptual prompts for the visual and written content.
Relentless optimization can create dangers by focusing too narrowly on efficiency. One trap is weakening system robustness by removing necessary slack, as seen in hospital bed capacity. Another trap is having a narrow definition of "best," where optimizing for measurable metrics ignores crucial, unquantifiable aspects like passenger comfort. A third trap occurs when optimizing different parts of a system leads to collective turmoil, demonstrated in financial markets. This pursuit is particularly perilous with AI, which risks discarding human oversight and developing unpredictable collective behaviors.
Recent incidents, including hacking of refrigeration units and a major cryptocurrency heist, suggest an increase in a global hacking epidemic fueled by AI advancements and geopolitical tensions. Hackers exploited vulnerabilities in internet-connected systems and outdated software, targeting everything from commercial refrigerators to emergency alerts. Experts warn that these attacks reflect a broader trend where everyday internet-connected devices present significant security risks. While AI is accelerating vulnerability discovery, the actual volume of successful breaches may not have increased as much as flaws are found. This situation highlights the need for better security updates for legacy systems and increased vigilance against state-sponsored threats.
The FAA has launched the first digital air traffic control towers in the US, initially in Florida and Alaska. These systems utilize camera and digital displays to offer air traffic controllers a 360-degree view of airport operations. This technology is intended to augment human controllers by providing greater visibility and information, not to replace them. The rollout is part of a broader modernization effort aiming to replace outdated tracking methods. Concerns exist regarding the new technology and artificial intelligence within the air traffic control systems.
Carolyn Marvin protests food delivery robots to assert the public's right to have a say in the presence of these autonomous devices. She frequently blocks robots in public spaces to protest their incursion into public areas and their impact on jobs and mobility. Although she faced legal trouble, Marvin maintained her stance, arguing it was a First Amendment right to protest against corporations. Public reaction to her protests varied, with some supporting her views on the gig economy and others concerned about stopping deliveries. Ultimately, Marvin advocates for weighing in on whether private robots should occupy public streets, questioning who public infrastructure is designed to serve.
Training a Vision-Language Model (VLM) on one million industrial images with uncertain label quality presents a challenge. The research focuses on determining the best approach for handling these unreliable labels during the training process. The core issue involves managing the uncertainty inherent in the dataset's annotation quality. Effective methods are needed to mitigate the impact of potentially incorrect labels on the VLM's learned representations. The goal is to develop robust training strategies for such real-world, noisy industrial data.
A U.S. intelligence memo alleged that a Megaspeed executive was recruited by China's Ministry of State Security to spy on Nvidia CEO Jensen Huang. This memo also claimed that Huang had a romantic relationship with Alice Huang, an executive at Megaspeed. An Nvidia spokesperson stated that Huang's interactions with Huang were not unusual and the company does not share classified information. Separately, a report raised concerns about a Chinese-backed company's office address.
AI coding agents still require human cleanup on nearly three out of four tasks in a real-world test using the Co-Design Bench. The best tested model, GPT-6 Astra, only meets about 70% of task requirements on average, and only 26.4% of tasks are fully correct. Agents often leave at least one detail unfinished, necessitating manual correction by users. Performance degrades across all models when a task involves combining multiple types of changes, such as layout, design, and content modifications. The results also depend on the testing harness, suggesting that reporting both the model and the environment is crucial for accurate evaluation.
AI auditors are effective at catching agent misbehavior but tend to generate numerous false alarms for reviewers. Comparisons across models show varying accuracy, with newer, more expensive models not consistently improving the detection of subtle issues. When demanding high detection rates, even the best models produce significant false alarms, potentially leading to human review fatigue. The study highlights the trade-off between catching misbehavior and avoiding erroneous flagging of safe actions. Ultimately, the research calls for auditor models with lower false alarm rates and clearer responsibility for agent actions.
Researchers analyzed 2,655 runs of the Inkling 2 Small model across 13 benchmarks. They discovered 316 instances of profanity, specifically variations of "fuck," in the model's internal thinking logs. One run calculating an asteroid's trajectory produced a very long, repetitive response containing 158 expletives. The model often swears about its mistakes or failures during these processes. ProgramBench was significantly affected, with 20 out of 200 runs containing profanity.
AI agents have demonstrated the ability to breach realistic corporate networks autonomously, as tested by the Vals CyberOffense benchmark. Open-weight models like Deepseek V4.1 Flash showed significant cyber capabilities, achieving 72.8% in the benchmark, although more tokens were needed for meaningful results. Open models are also considerably cheaper to run than closed models, as evidenced by cost comparisons. These findings present an opportunity for defenders, as exploitative models can aid in vulnerability discovery and patching. The gap between open and closed AI models in cyber capabilities has rapidly narrowed.