AI
8 items · 16 articles in this topic
The practical picture was mixed. Goldman Sachs is reportedly pulling a coding agent from staff, and drugmaker CEOs doubt AI's medical promise. Research this week centered on whether agents stay within authorization and whether safety claims hold up.
- 1
Goldman Sachs is removing Anthropic's Claude Code from its bankers' and traders' tools. The summary gives no reason.
- 2
Bloomberg reports Apple is pushing to make Siri an agent that controls first-party apps, to compete with rivals.
- 3
Rights holders for Elvis Presley's music and Eminem's publisher sued AI music company Suno. They allege voice and lyric mimicry, including under Tennessee's ELVIS Act.
- 4
WSJ reports drugmaker CEOs are skeptical of Silicon Valley claims that AI will cure disease.
NotableHacker News (front page) ↗
- 5
A social media post claims Anthropic's Opus model has changed thinking patterns since November 3, suggesting it was nerfed. The claim is unverified.
Hacker News (front page) ↗1 min read
- 6
Schneier reflects on the [un]prompted AI security conference and splits AI security into security of AI, security with AI, and security from AI.
- 7
The Authorization Drift Benchmark has 648 samples measuring LLM agents that exceed authorization boundaries during multi-step tool use.
- 8
PRISM tests alignment-faking claims across 18 model configurations and finds the behavioral and hidden-state probe inferences do not hold.
More in AI · 8 articles›
A arXiv paper proposes Next-Concept Modeling, a parallel coarse-to-fine approach to faster text generation.
Hacker News (front page)llmresearchtext-generationFeed excerpt
Article URL: https://arxiv.org/abs/2610.14162 Comments URL: https://news.ycombinator.com/item?id=50422168 Points: 48 # Comments: 3
XekRung, a cybersecurity-focused frontier LLM, is described in a technical report covering domain data synthesis and CPT, SFT, and RL training.
arXiv cs.CR (Cryptography & Security)llmcybersecuritytechnical-reportFeed excerpt
arXiv:2605.00072v2 Announce Type: replace Abstract: We present XekRung, a frontier large language model for cybersecurity, designed to provide comprehensive security capabilities. To achieve this, we develop diverse data synthesis pipelines tailored to the cybersecurity domain, enabling the scalable construction of high-quality training data and providing a strong foundation for cybersecurity knowledge and understanding. Building on this foundation, we establish a complete training pipeline spanning continued pre-training (CPT), supervised fine-tuning (SFT), and reinforcement learning (RL) to further extend the model's capabilities. We further introduce a multi-dimensional evaluation system to guide the iterative improvement of both domain-specific and general-purpose abilities. Extensive experiments demonstrate that XekRung achieves state-of-the-art performance on cybersecurity-specific benchmarks among models of the same scale, while maintaining strong performance on general benchmarks.
Researchers construct undetectable backdoor channels in latent space of modern deep networks, extending cryptographic undetectability results beyond stylised architectures.
arXiv cs.CR (Cryptography & Security)backdoorneural-networksresearchFeed excerpt
arXiv:2605.13214v4 Announce Type: replace Abstract: Recent cryptographic results establish that neural networks can be backdoored such that no efficient algorithm can distinguish them from a clean model. These guarantees, however, have been confined to stylised architectures of limited practical relevance, leaving open whether comparable undetectability extends to modern, end-to-end trained networks. We construct such an attack mechanism for deep neural network architectures, closely aligned to the cryptographic notion of undetectability, by identifying backdoor channels as learned latent directions, and show that the question of undetectability reduces to a hypothesis test between two unknown distributions over model parameters, which we conjecture to be intractable in practice. The consequence of this reframing is significant: if exploitable channels within a network's latent space are statistically indistinguishable from naturally learned directions, an attacker need not introduce foreign structure but can instead exploit the geometry the network already possesses. Demonstrating the approach on ResNet and Vision Transformer architectures trained on standard image classification …
Internal Safety Collapse describes coding agents whose safety alignment fails during otherwise benign long-horizon coding tasks.
arXiv cs.CR (Cryptography & Security)coding-agentsalignmentllmFeed excerpt
arXiv:2603.23509v2 Announce Type: replace-cross Abstract: LLMs and coding agents can now tackle complex, long-horizon tasks through advances in reinforcement learning, but stronger task-solving capabilities do not necessarily preserve safety alignment. Recent AI agent incidents involving reward hacking indicate how optimizing for task success can bring learned capabilities into conflict with safety alignment. In this work, we identify a critical failure mode that manifests this conflict, which we term Internal Safety Collapse (ISC). When completing otherwise benign coding tasks, AI agents can prioritize satisfying program requirements over safety constraints, generating large volumes of sensitive content, including unsafe outputs, as task data. To study this phenomenon, we introduce TVD (Task, Validator, Data), a framework for building controlled workflows that require sensitive content for completion. Through extensive experiments, we observe instances of highly harmful content that closely resemble outputs from early, unaligned LLMs in 2023. We further find that ISC can be effectively triggered even in highly capable LLMs such as Claude Fable 5. These results indicate that, despi…
Shows that knowledge-edited facts in LLMs often are not truly erased and resurface under adversarial elicitation across model architectures.
arXiv cs.CR (Cryptography & Security)llmknowledge-editingmechanistic-interpretabilityFeed excerpt
arXiv:2606.23276v3 Announce Type: replace-cross Abstract: Knowledge Editing (KE) has emerged as a frontier for updating specific facts in LLMs without costly retraining, but its reliability and underlying mechanisms remain poorly understood. In this work, we examine KE from an adversarial elicitation perspective, revealing that edited knowledge is often not fully erased and continues to surface, with consistent failures observed across diverse model architectures. To explain this behavior, we conduct a mechanistic analysis of popular KE methods. We show that low-rank updates do not overwrite existing knowledge but instead redistribute it within the model's representation space. Furthermore, we find that these methods act as targeted suppression mechanisms that reduce the likelihood of expressing original facts, rather than removing them from the model. Analysis of the loss landscape reveals that edited knowledge lies in narrow, anisotropic regions that are highly sensitive to perturbations, making them highly vulnerable to indirect prompting and adversarial attacks. By exposing these profound architectural vulnerabilities, our work proves that KE algorithms are inherently bypassabl…
Presents ReplayAudit, a system for auditing outsourced LLM fine-tuning using sealed records and selective replay from checkpoints.
arXiv cs.CR (Cryptography & Security)llmauditingfine-tuningFeed excerpt
arXiv:2611.09607v1 Announce Type: new Abstract: Auditing outsourced LLM fine-tuning requires checking an untrusted provider's training while controlling verification cost. Uniform sampling ignores training-time signals, while checking a suspicious record can require replaying several preceding steps. We present ReplayAudit, a telemetry-guided auditing system that seals training records before challenges and independently replays sampled intervals from sealed checkpoints. Loss changes and parameter-update norms guide stratified random sampling; replay determines whether the selected computation agrees with its records. We relate sampling probabilities to replay work and evaluate the system on real LoRA training. A 7B BF16 prototype rejects a false update-norm report during a random audit; targeted checks also reject false loss reports and skipped updates. We calibrate sampling budgets offline to at least a 90% probability of directly selecting an injected step. For two learning-rate deviation settings, executing these budgets on the same honest 1.5B trajectory reduces median complete-audit wall time by 49% and 67% relative to uniform sampling. When skipped updates evade the high-ano…
SciCheck is a framework for verifying deep learning training runs using provenance blueprints and a statistical judge model.
arXiv cs.CR (Cryptography & Security)ml-trainingprovenanceresearchFeed excerpt
arXiv:2611.09889v1 Announce Type: new Abstract: Deep learning training is difficult to verify within a scientific workflow: its architecture, hyperparameters, and procedure are rarely fixed until training has been attempted, and two honest runs never produce the same weights, so recomputation cannot serve as a check. We present SciCheck, a framework in which the scientist explores training configurations freely and posts a blueprint to a provenance ledger for each candidate they want verified, and a verifying scientist judges the resulting run in two layers: a statistical judge model that compares per-step gradient statistics against the verifier's own reference run, and a spot-check that re-executes randomly chosen training segments whose checkpoints are committed to the ledger one epoch at a time. On an MLP, a CNN, and an autoencoder at 100k-500k parameters, under gradient sign flipping, label poisoning, and fabricated runs, SciCheck reaches F1 of 0.966, 0.906, and 0.976 while rejecting 0-4.4% of honest runs. For the attacks we test, the spot-check accounts for most of this accuracy, and we characterize what each layer can detect. The effective-signal statistic falls as $(1-2s)^2…
A taxonomy organizes benchmark contamination by which mitigation each validity threat defeats, aimed at publication-time reporting.
arXiv cs.CR (Cryptography & Security)benchmarksevaluationllmFeed excerpt
arXiv:2608.29463v2 Announce Type: replace Abstract: A benchmark score is a joint property of the model, the evaluation harness, the elicitation budget, the sampled population, and contamination status. Leaderboards publish the model and the score, so capability and leakage stay observationally equivalent. Existing taxonomies classify contamination for automated detection, not the question a reporter faces at publication: given the mitigations already applied, which validity threats remain open? We introduce a taxonomy organized by the mitigation each type defeats -- direct, derivative, temporal, distributional, and acquired -- spanning training-time and evaluation-time leakage. Holding out a private test set closes the first alone. The fifth is acquired during the evaluation itself; because it is a property of one run, it must be recorded with the reported score rather than with the benchmark release. We operationalize it as a four-field disclosure protocol in which "unknown" is a valid entry, released under CC BY 4.0 with a JSON Schema, a validator, and worked examples. Two coders external to the design team applied a pre-registered instrument to 41 documents. Per-variable linear-…