Reverse Item Response Theory is introduced for pharmacogenomic drug-response analysis by modeling cancer types as subjects and drugs as items. The model is applied to 242,036 drug sensitivity measurements from the GDSC2 database to estimate in-vitro resistance and drug activity. Validation shows that reverse IRT better recovers latent rankings than simple averaging across various missingness regimes. The method achieved the best Brier score in held-out prediction, with some cancer types showing stable classifications. Cross-platform replication suggests methodological robustness rather than a universal clinical ranking.
Adam's geometric relationship to Natural Gradient Descent (NGD) is investigated using a scale-invariant metric. The study measures Adam's deviation from NGD across four different loss landscapes, including linear and non-convex settings. Adam's geometric trajectory is found to be context-dependent, showing low deviation in well-conditioned settings but significant misalignment in ill-conditioned neural network problems. While Adam's geometric drift is larger, it does not impair final objective minimization, as Adam consistently achieves low loss. The findings suggest Adam's effectiveness may arise more from structural approximation and momentum smoothing than from closely following the natural gradient path.
Frequency-collapse attention achieves gains by using a bandpass-filtered inner product at a learned frequency. Testing filter properties revealed that DC and Nyquist components are detrimental, confirming the necessity of an oscillatory structure. The optimal single-scale bandwidth is approximately two bins centered around paragraph scale, yielding a performance gain of +1.15 nats. Admissible filters outperform non-admissible Gaussians and provide protection against bilateral FFT leakage. These findings characterize FourierQK for bidirectional attention settings and motivate causal spectral variants for autoregressive models.
The study proposes a multi-objective framework combining reinforcement learning-based multiple instance learning (RL-MIL), adversarial debiasing, and preference-conditioned hypernetworks for student-at-risk prediction. The RL agent selects informative instances from student interaction data, treating each student as a bag of weakly labeled interactions. Two hypernetwork variants were tested to see if a user-defined preference scalar could control the trade-off between predictive performance and Equalized Odds. The baseline RL-MIL performs well, but preference conditioning alone fails to guarantee controllable multi-objective behavior due to issues like objective dominance. Robust fair RL-MIL requires explicit mechanisms for gradient balancing and stability analysis.
Short polynomial programs were tested to accelerate special-function unit (SFU) operations in large language models (LLMs). These programs replaced native functions like sigmoid and tanh with degree-3 or degree-4 bfloat16 operations in tasks such as dense SiLU and attention mechanisms. The optimized kernels resulted in performance improvements of 1.19-2.19x in L2 and 1.00-1.70x in HBM memory access. Substitutions in attention improved the attention forward pass by 7.4%. Overall, these optimizations led to noticeable changes in training-loss differences near 100 billion tokens.
Kolmogorov-Arnold Networks (KANs) are proposed as an alternative to traditional deep learning by using learnable univariate functions for node activations. The SW-KAN introduces Stieltjes-Wigert q-orthogonal polynomials to address domain mismatches in polynomial-based KANs. A smooth mapping is used to bridge the domain gap while maintaining stable gradients and efficient evaluation. Experiments show SW-KAN achieves superior accuracy-efficiency in image classification and continuous function approximation tasks. This architecture provides a robust method for efficient function approximation in resource-constrained settings.
Format-aware fusion is presented to improve fast pretraining using four-bit floating-point (FP4) by co-designing quantization producers with their scale domain and consumer layout. This approach enables native $\text{mxfp}$, global $\text{nvfp}$, and cooperative-thread-array-local $\text{nvfp}$ operations. Evaluation on Llama-3-family 8B pretraining showed that custom routes achieved $37.9\text{K}$ tokens/s/GPU, outperforming bfloat16 and Transformer Engine methods. The results indicated that FP4 outcomes depend on the scale contract, operand, and execution path. This method improved training-loss compared to raw bfloat16, suggesting a benefit beyond raw speed.
Humans express verbal uncertainty through linguistic markers like "very likely," which reflects metacognitive monitoring. Current LLM uncertainty quantification often relies on likelihood or consistency signals rather than these verbal markers. The research investigates how LLMs diverge from humans in verbal uncertainty quantification. A novel algorithm, METHODNAME, is introduced to learn an optimal uncertainty profile directly from marker outputs. This method allows for a direct comparison of confidence semantics between humans and LLMs to reveal systematic confidence disparities.
Belief-based agents need reliable decisions despite noisy memory evidence, raising the question of whether memory must calibrate its sources before improving decisions. The work separates learning, calibration, and revision certification within a four-model hidden Markov family. Learning and certifying useful decisions can require quadratically fewer records than source calibration. A certificate can be derived using observable witness regions without needing pure-class anchors. The results statistically account for when memory decisions can be learned and justified without fully recovering source reliability.
A single mastery threshold does not adequately fit all knowledge tracing (KT) models used in tutoring systems. The effectiveness of a threshold varies significantly depending on the underlying KT model, as models like Bayesian Knowledge Tracing (BKT) and neural models estimate different aspects of knowledge. Differences in model outputs mean the same numerical cutoff does not represent the same level of mastery across models. Stricter thresholds also failed to reliably reduce performance gaps and could unfairly restrict advancement for stronger-prior students. Therefore, mastery thresholds should be recalibrated based on the specific KT model and instructional priorities.