The research investigates using large language models to automatically construct strong, simple baselines for deep learning methods in scientific applications. This was achieved by assembling 36 supervised classification and regression tasks from peer-reviewed literature in high-impact journals. The automatically generated baselines were found to recover a median of 95.5% of the performance gap between trivial predictors and deep learning models. These simple baselines often matched or exceeded the performance of the deep learning models in 12 studies. The findings suggest that automated baseline construction is a practical method for assessing deep learning contributions in the physical and life sciences.
The proposed method, PoSE, estimates the state of a distribution system by learning the distribution of substation injections using a neural network trained with flow matching. This approach maintains the Bayesian structure of state estimation, learning only the unknown distribution rather than approximating power flow directly. The power flow equations are incorporated only during sampling via a noise-aware Kalman-type correction. Experiments show PoSE reduces probabilistic injection error significantly compared to Gaussian priors and supervised deep learning estimators. The method achieves this accuracy while providing reliable uncertainty estimates for system states.
Standard edge anomaly detection uses an "impute-then-detect" pipeline where imputation based on Mean Squared Error introduces bias. The proposed Task-Aware Diffusion Refiner (TADR) addresses this by conditioning the diffusion refinement on a frozen downstream detector's loss landscape. This method improves anomaly detection by $\text{F1}$ by $0.338$ over unrefined detectors on SMD. TADR's benefit is shown to correlate with available redundant channels and the anomaly evidence removed by the outage. The method is computationally efficient, running quickly on a single CPU core.
The LIDFS method proposes an unsupervised feature selection technique based on Local Intrinsic Dimensionality (LID) to address dimensionality reduction without class labels. It estimates the LID for each feature based on the distances to its $k$ nearest neighbors to determine local discriminability. Features with lower LID are deemed more important, and a refinement step removes locally redundant features. This method achieves a complete ranking allowing feature selection without further computation, running in $\mathcal{O}(d^2 n)$ time. Evaluation on benchmark datasets shows LIDFS outperforms other methods in both supervised and clustering-based assessments.
Certified Anchor-Based Adaptation (CABA) is introduced as a candidate-agnostic post-update gate for continual adaptation in machine learning. It reserves historical observations and compares policies on frozen representations to select an update based on a budget constraint. The framework provides a theory-led certification that separates measured anchor retention from conditional off-anchor guarantees. Under specific assumptions, an accepted update guarantees a uniform behavioral-divergence bound on protected support. Experimental results show CABA reduces mean forgetting significantly in a Human Activity Recognition study.
AutoHebb introduces self-modifying Hebbian networks designed to achieve zero-shot inference and continual learning. These networks autonomously combine external feedback with internal activity to generate a global learning signal, or 'neuromodulator'. This neuromodulator gates a biologically grounded local learning rule, based on the 3-factor Hebbian rule, which updates the network's weights. Meta-learning for fixed weights allows AutoHebb networks to excel on zero-shot tasks and exhibit emergent ZI/CL abilities. The research suggests meta-learning, Hebbian plasticity, and autonomous control are essential for these learning capabilities.
Scientific foundation models face catastrophic forgetting when adapting from simulations to real-world data. Study on a physics foundation model showed that finetuning on simulations rapidly degraded performance on original pretraining tasks. Subsequent adaptation to real data further eroded performance on the simulated task. Replaying just 1% of the simulated data during real-data finetuning helped recover lost simulation performance. The research concludes that lightweight replay strategies are effective for mitigating catastrophic forgetting in sim-to-real adaptation of SciFMs.
FARO is a function-space adaptive optimizer that determines parameter block partitioning as its primary decision unit. It assigns geometry to each block based on its counterfactual return and then reallocates a shared budget using cross-block equal-risk contribution. This method successfully recovers human priors in a GPT-style model without manual per-layer geometry setting. Furthermore, the FARO-ERC approach reduces clean-data loss by an average of 0.0136 nats across different seeds. The technique proves robust by trimming riskiest blocks and maintaining performance under corrupted input streams.
The paper studies first-order regret guarantees for adversarial bandits with a fixed, known directed feedback graph. It bridges a gap between existing bounds by obtaining an expected regret bound of $\widetilde{O}(\sqrt{\alpha L^\star}+\alpha)$. The work introduces an algorithm for self-aware graphs by combining EXP3 with a sampling rule from exploration by optimization. This involves constructing a map whose fixed points control the one-round regret remainder using the graph's independence number. Finally, the algorithm is extended to general strongly observable graphs, achieving the same guarantee through adaptation of the two-level aggregation approach.
TensorSSM is a novel Implicit Neural Representation method for functional tensor decomposition that memorizes spatial positions within a dynamic State Space Model (SSM). It decomposes mode-wise coordinates into sequences using a K-ary rule to encode spatial tokens from coarse to fine. The method extends the SSM state space to explicitly map cross-mode spatial dependencies, allowing inter-mode interactions to enhance updates. This integration of spatial information into the state space and tensor core significantly advances state adaptability and expressiveness. Experiments show TensorSSM achieves high reconstruction accuracy and storage efficiency across diverse tensor datasets.
CausalGrade is a method for scoring the structural quality of candidate causal graphs by integrating evidence-guided scoring with the probabilistic judgment model Jev. It incorporates evidence of association, conditional dependence, and directionality from observational data and edge probabilities derived from a causal discovery foundation model. Jev generates probabilistic judgments about direct causal edges, and the final graph score measures the agreement between the evaluated graph's structure and these edge-level judgments. Testing on synthetic datasets showed that higher scores correlate with smaller structural errors across various causal discovery algorithms. CausalGrade achieved a 54.4\% best-candidate hit rate in graph selection tasks, outperforming the Self-Compatibility metric.
A general multitask lemma is proposed to provide sufficient conditions for fast shared subspace estimation in multitask learning. This lemma is based on geometric conditions concerning task regression functions and their estimators. The lemma is used to establish sharp minimax rates for linear regression and classification in relevant settings. Furthermore, the work demonstrates rates matching up to logarithmic factors for additive-noise regression. These results fill a gap in the literature regarding the estimation of shared subspaces in multitask scenarios.
The research uses Integrated Nested Laplace Approximation (INLA) to adaptively sample radio environment maps from non-Gaussian IoT observations using UAVs. Waypoints are selected based on entropy pre-screening and Sherman--Morrison lookahead over the reachable set. Widening the prior shadowing standard deviation from 6.0 to 10.025 dB improved the noise margin and credible-interval coverage. The study shows substantial corpus-dependent sensitivity to prior specification, suggesting the preferred width depends on the data and model mismatch. Overall, prior specification significantly impacts the accuracy and calibration of the adaptive sampling process.
Data analysis often involves datasets contaminated by unknown mixtures from data collection processes. Standard distributional learning methods either ignore this contamination or discard most of the data when dealing with such samples. The proposed method, CC-MMD, is a design-based estimator that corrects for contamination by combining proxy scores with audited residual corrections. This approach recovers the true maximum mean discrepancy (MMD) from the clean latent sample despite noisy supervision. Empirically, CC-MMD successfully improves parameter recovery and enables generative models trained on corrupted mixtures to learn the clean target distribution.
Simplicial neural networks (SNNs) require rigorous uncertainty quantification (UQ), which they typically lack, limiting their application in sensitive scenarios. The proposed framework, conformalized SNN (CF-SNN), generates prediction sets for multiple targets simultaneously on simplicial complexes. CF-SNN ensures that each target's prediction set includes the true label with a specified nominal coverage probability. Furthermore, the overall miscoverage proportion across all targets is guaranteed to stay close to the nominal level with high probability. Experiments confirm that CF-SNN is a flexible UQ method that provides informative, small-sized prediction sets.
Flow matching's performance depends on the coupling used to pair source and target samples, but practical limitations hinder consistently improving sampling quality via optimal transport. ReCouple is introduced as a framework to construct coarse-grained optimal transport couplings for rollout-free post-training. Extensions, ReCouple+ and ReCouple++, further address continuous source settings and spatial regularization, respectively. Empirical results show that the interplay between coupling optimality, neural realizability, and generation quality is dependent on the number of function evaluations. These methods effectively reveal the complex relationship governing post-training in flow matching.
Conditional Neural SDEs are proposed for generating financial time-series paths by combining continuous-time stochastic dynamics with learnable drift and diffusion functions. A residual anchor architecture decomposes the target into a learned conditional anchor and a stochastic residual to improve stability. This approach models the residual dynamics around the anchor, which captures the context-dependent baseline. Evaluation on real-world financial data demonstrates the model's ability to generate plausible paths with competitive accuracy and calibrated uncertainty estimates.
Reliability-Weighted Direct Preference Optimization (RW-DPO) incorporates an unknown reliability weight for each preference comparison, where a weight of 1 indicates agreement and 0 indicates a reversed comparison. It estimates these weights jointly with the policy using a minimax-concave penalty on the negative log of the weights. This penalty ensures the model agrees with comparisons weighted at 1 and only penalizes contradictions. The method consistently recovers preference directions and weights, even when a fraction of labels are flipped. Simulations show RW-DPO achieves lower estimation error than other robust preference learning methods.
The study proposes PDE-residual-informed Kriging (PRIK) to incorporate global PDE-residual information directly into the Gaussian process covariance structure. This is an advancement over existing methods that use pointwise conditioning or residual penalties separately. The PRIK method constructs a PDE-residual kernel using the $L^2$ inner product of PDE residuals to minimize the integrated squared PDE residual in the posterior mean. Statistical interpretations show the variance-scale estimate equals the normalized PDE-residual energy, and theoretical results establish convergence rates and bounds for the posterior mean. Numerical experiments confirm the PDE-solving capability of PRIK.
Reinforcement learning requires understanding its statistical foundations, especially concerning data efficiency in sample-starved scenarios. The paper provides a non-asymptotic statistical treatment for RL based on tabular Markov decision processes. It focuses on analyzing three sampling mechanisms—RL with a generative model, online RL, and offline RL—across model-based and model-free paradigms. The research emphasizes minimax-optimal algorithm design and sharp statistical characterizations of various RL algorithms. Extensions to the topic, like robust RL and multi-agent systems, are also briefly discussed.