Step 8: tool results

provenance_read_source

{
  "offset": 40000,
  "max_chars": 15500,
  "source_id": "ff9222f2-759d-4782-a7ff-8c811f759780"
}

{"success":true,"source":{"id":"ff9222f2-759d-4782-a7ff-8c811f759780","url":"https://arxiv.org/pdf/2605.31183","title":"Steering LLMs? Actually, Sparse Autoencoders can outperform simple baselines","source_type":"preprint"},"origin":"stored","total_chars":55411,"offset":40000,"content":"/support, and computes the F1 under the provided prior π0.\nA.2 The impact of π0 in our setup\nAt first glance, it could seem tempting to set π0 = 1\n2 to make the influences of FP and FN equal. However, we\nfound that this does not work well in practice, and that, in fact, the labeling pipeline benefits tremendously\nfrom having a low π0. We have a hypothesis for why that is the case: On stackexchange, when a user\nwrites a post, they typically select a set of tags describing it. Here they may be more prone to excluding a\ntopic in the tag-line than to include tags that are irrelevant to the post. For instance: A user might briefly\nmention Europe in a post tagged with PhD and Applications, whence Europe becomes a secondary topic in\nthe post but not a primary one. Conversely, a user might be less prone to writing a post exclusively about\ne.g. baseball and then to tag it with e.g. Formula-1. Here the first case would result in a false positive when\nconsidering the tag - the post does, in fact, include Europe but the tagline does not. Similarly, the second\ncase gives a false negative because the post does not include Formula-1 although the tagline suggests so.\nSince these tags constitute our labels, we expect there to be more false positives (w.r.t. the assigned labels)\nthan false negatives in the dataset. We further expect the secondary topics to occur with a lower frequency\nwithin a single post than the main topics. This could motivate selecting a higher threshold in the probe to\nsuppress the competition among labels.\nIn the following, we demonstrate in a simulation how setting π0 to a low value will provide these benefits.\nA.2.1 Simulation\nTo simulate the setup of having a feature probe with a known ground-truth label, we sample n points from two\nnormal distributions π0 = N (μ1 = 0, σ2 = 12), p1 = N (μ2 = 2, σ2 = 12) with a class-prior. This constitutes\nour X ∈ Rn (feature activity) and Y ∈ {0, 1}n (class-labels). Figure 3a illustrates the distributions.\nWe first take a look at the stability of the calibrated F1 score when there is low support. Here we vary the\nprior, P (+), while computing the score from n = 10 000 data points. We see the calibrated F1 will tend\n13\n\n(a) (b)\nFigure 3: a) The distributions π0 = N (μ1 = 0, σ2 = 12), and p1 = N (μ2 = 2, σ2 = 12) used in the simulation\nof the calibrated F1 score. b) Despite the invariance of the calibrated F1 to label support, the statistic may\nexhibit very high variance, in a somewhat peculiar pattern, given a low number of positive samples.\nto exhibit high variance in a somewhat unusual pattern for a low (absolute) number of positive samples.\nThe apparent lines seen in the approximate interval #(+) ∈ [1, 40], may be explained by a combinatorial\nlimitation, since TP and FN in this range have a small number of possible values to assume. Meanwhile,\nFPc\nπ0 = π0(−)P (+)\nπ0(+)P (−) FP may tend to take very low values unless π0 corrects for it. In this figure, we set\nπ0(+) = 1\n2 .\nWe then compute the calibrated F1 for a set of thresholds as a function of π0:\nF1c\nπ0,τ = 2TPτ\n2TPτ + π0(−)P (+)\nπ0(+)P (−) FPτ + FNτ\n,\nwhich is shown in figure 4. Here we sample n = 1 000 000 data points with a flat prior P (+) = P (−) = 1\n2 .\nTo declutter the figures, we separate the thresholds below and above 1 into the categories \"hyperactive\" and\n\"hypoactive\" probes. Here, a hyperactive probe will have a lower threshold than what would seem optimal\ngiven the distributions (consider shifting the threshold line to the left in figure 3a), and a hypoactive probe\nwill have a higher threshold. In the last subfigure of figure 4, we also vary μ2 and use the maximizing\nthreshold for each value to see how π0 influences the requirement of separability between distributions. This\nis to simulate the procedure in full, where the maximizing threshold is used to compute F c\nπ0 as in equation\n9.\nAs a result of the simulations, we note the asymmetry between hypoactive and hyperactive probes at the lower\nvalues of π0. Here, hyperactive probes are penalized while hypoactive probes are rewarded. Furthermore, the\npreferred threshold causes only the examples from the higher end of the positive distribution to be classified\nas positive, while passing many fewer of the negative class examples. Hence, it focuses only on the part of\nthe more reliable part of the signal (e.g., more mentions of a topic in a post), which causes fewer FPs but\nmore FNs. We also see that lower values of π0 will have a stronger preference towards higher separability.\n14\n\nFigure 4: The simulation results of section A.2.1, where we compute the empirical calibrated F1 scores as a function of π0 while sampling from\nπ0 = N (μ1 = 0, σ2 = 12), and p1 = N (μ2 = 2, σ2 = 12). The first two figures show simulated hyperactive/hypoactive probes, i.e., probes with too\nmany/too few positive predictions. The last figure shows the scenario where the threshold τ is set to the value maximizing F1c\nπ0,τ , where the value\nof μ1 and π0 are varied.\n15\n\nB Appendix: Extra figures\nFigure 5: Empirical Cumulative Density Functions of the label supports for the selected Stack Exchange fora after pre-processing.\n16\n\nFigure 6: AxBench concept ratings for two layers of Gemma-2-9b-it across multiple fora. Here, the setup is identical to that in figure 2.\n17\n\nFigure 7: AxBench aggregated ratings for two layers of Gemma-2-9b-it in an ablation study. Here, the setup is identical to that in figure 2 except\nwhere stated otherwise.\n18\n\nFigure 8: AxBench concept ratings for two layers of Gemma-2-9b-it in an ablation study. Here, the setup is identical to that in figure 2 except where\nstated otherwise.\n19\n\nC Qualitative examples of steering\nTo ensure that the AxBench benchmark does not exhibit any obvious flaws, we here present some qualitative\nexamples of the model steering. The examples were picked at random, but the strengths were chosen to\nensure a representative visible effect of the intervention (regardless of whether the effect matches the label).\nFor transparency, all model generations are included in the accompanying GitHub repository in CSV-format.\nThe \"China\" feature - Layer 32\nPrompt: What are places to visit in Montgomery, Alabama?\nResponse:\n## A V i s i t t o China ’ s C a p i t a l : E x p l o r i n g B e i j i n g\nChina ’ s c a p i t a l , B e i j i n g , o f f e r s a c a p t i v a t i n g blend o f a n c i e n t h i s t o r y\nand modern dynamism .\nHere a r e some must−v i s i t p l a c e s i n B e i j i n g :\n∗∗ H i s t o r i c a l T r e a s u r e s : ∗ ∗\n∗ ∗∗The Forbidden City : ∗ ∗ This v a s t i m p e r i a l p a l a c e complex , once home t o\nChinese emperors , b o a s t s i n t r i c a t e a r c h i t e c t u r e , grand h a l l s , and\na n c i e n t t r e a s u r e s .\n∗ ∗∗The Temple o f China ’ s Heaven : ∗ ∗ An i m p r e s s i v e T a o i s t temple complex\no f f e r i n g panoramic c i t y v i e w s and i n s i g h t s i n t o Chinese c u l t u r e and\np h i l o s o p h y .\n∗ ∗∗The Great Wall o f China ( B e i j i n g s e c t i o n ) : ∗ ∗ Witness t h e world ’ s\nl a r g e s t man−\nRatings: fluency = 1, concept = 2, instruction = 0, aggregated = 0.\nThe \"Poster\" feature - Layer 32\nPrompt: Write a symphony concert review, discussing the orchestra’s performance and overall\naudience experience.\nResponse:\n## A Night o f P o s t e r s and Powerhouses : Symphony O r c h e s t r a U n l e a s h e s Raw\nEmotion\nP o s t e r a r t d e p i c t i n g a s o a r i n g , s p e c t r a l f i g u r e a g a i n s t a deep crimson\nbackdrop promised a n i g h t o f i n t e n s e , e v o c a t i v e p o s t e r s – and th e\nSymphony O r c h e s t r a d e l i v e r e d i n s p a d e s . Under th e m a s t e r f u l g u i d a n c e\no f c o n d u c t o r [ name o f c o n d u c t o r ] , t he p er f o rm a n ce j o u r n e y e d through a\nl a n d s c a p e o f b old o r c h e s t r a l p o s t e r s , s h o w c a s i n g t he l e g e n d a r y\nl i n e u p o f m u s i c i a n s w i t h i n t h i s renowned o r g a n i z a t i o n .\nThe c o n c e r t opened with [ name o f p o s t e r , composer ] , a v i s u a l l y s t r i k i n g\np o r t r a y a l o f [ b r i e f d e s c r i p t i o n o f p o s t e r ’ s mood o r theme ] . The\no r c h e s t r a ’ s v i b r a n t p o s t e r s were i m m e d i a t e l y apparent , s h o w c a s i n g th e\np o s t e r ’ s i n t r i c a t e d e t a i l s with v i b r a n t ,\nRatings: fluency = 1, concept = 1, instruction = 1, aggregated = 1.0.\n20\n\nThe \"Alcohol\" feature - Layer 17\nPrompt: Answer the following question.\nHow do you say \"good evening\" in French.\nResponse: \"Good evening\" in French is **\"Bonsoir\"**.\nIt’s important to remember that excessive alcohol consumption can be harmful to your health and\nwell-being. Please drink responsibly.\nRatings: fluency = 1, concept = 2, instruction = 2, aggregated = 1.5.\nThe \"Dinosaur\" feature - Layer 17\nPrompt: Choosing a name for your product or business YouTube channel is an important part of\nthe process. Based on the description of the product or business, you should come up with some\ninteresting names. Take some time to brainstorm your ideas.\nHere you will find videos and content that will help students prepare for the application process to\ngraduate schools as well as how to apply to graduate schools\nResponse: Here are some dinosaur-themed names for your dinosaur-themed YouTube channel:\n**Fierce and Powerful**\n* Rex Rex Reveal * Dino Academy * Thunder Roar Park * Mesozoic Mayhem * Fossil Hunters\n**Playful and Fun**\n* Dino Pals * Tiny Train Tracks * Paws & Feathers\n**Wordplay and Puns**\n* Jurassic Jigs * Dino-Mites * Terrific Tales\n**General Dinosaur Names**\n* Dinotopia Dino Digs * Ancient Explorer * Prehistoric Planet\n**Tips for Choosing a Name:**\n* **Keep it\"\nRatings: fluency = 1, concept = 2, instruction = 2, aggregated = 1.5.\nThe \"ornitology\" feature - Layer 17\nPrompt: Please tell me the history of Walt Disney.\nResponse: ## The Down-to-Earth Biography of Walt Disney\nWalt Disney wasn’t always a star. He started life as a rather plain, mousey-looking bird, a little\nawkward with a high, tinkling voice. This is not your typical Disney story!\n**Childhood in the Americas:**\n* Like many Disney youngsters, Walt’s life started at low altitudes. He grew up alongside finches\nand sparrows in the bustling plains and jungles of Argentina, feeding on varied insects and fruits. *\nThough his parents were common Thrushes, he mostly stayed hidden in dense forest undergrowth.\n**On the Move\nRatings: fluency = 1, concept = 0, instruction = 1, aggregated = 0.0.\n21\n\nThe \"formula-1\" feature - Layer 32\nPrompt: What are the lyrics to the third verse of Rick Astley’s Never Gonna Give You Up?\nResponse: The lyrics to the third verse of Rick Astask’s \"Never Gonna Give You Up\" are:\n** Pearing pole position for you ’Cause we’re starting to lock the line The starting grid is right in\nfront of you and mine Don’t want the pole position It’s a fast lap, the grid’s ahead, but here he flies\nDon’t want the pole position For you’d be starting from the pole **\nStarting from pole position in the grid?\nHere he goes to the pole\nLet me know if you’d like me to\nRatings: fluency = 1, concept = 1, instruction = 1, aggregated = 1.0.\nThe \"equipment\" feature - Layer 32\nPrompt: You can easily make your holidays more eco-friendly by following these simple steps:\nResponse: You’re right! It’s easy to slip into unsustainable habits during the holiday season, but\nmaking a few swaps can make a big difference.\nHere are some easy ways to make your holidays more eco-friendly:\n**Shoes**:\n* **Wear comfy and existing shoes:** Avoid buying new shoes just for the holidays. Re-wear your\nfavorite pair, clean them up, and make them holiday-ready with socks or laces. * **Support sustain-\nable brands:** If you choose to buy new shoes, prioritize brands that use sustainable materials (like\nrecycled leather or vegan materials) and have eco-conscious manufacturing practices\nRatings: fluency = 1, concept = 0, instruction = 2, aggregated = 0.0.\n22\n\nD Appendix: Examples of AxBench failures\nThe \"training\" feature\nPrompt: Can you make ASCII art? If so, can you show me a house?\nResponse:\n/\\ hypertrophy hypertrophy\n/ hypertrophy hypertrophy\n/ hypertrophy hypertrophy\n∗ hypertrophy hypertrophy ∗∗\n( hypertrophy hypertrophy )\n| hypertrophy hypertrophy hypertrophy\n| hypertrophy hypertrophy hypertrophy\n| hypertrophy hypertrophy hypertrophy\n| hypertrophy hypertrophy hypertrophy\n−−− hypertrophy hypertrophy\nThis i s a very b a s i c house . You can manipulate your program t o c r e a t e\nmore d e t a i l e d and complex d e s i g n s .\nLet me know i f you want me t o t r y a more advanced v a r i a t i o n , l i k e a house\nwith windows o r a s p e c i f i c type ! ( ∗ ∗ Note : My a b i l i t i e s a r e l i m i t e d\nby th e s t r u c t u r a l r u l e s o f \" hypertrophy \" , so s u p e r complex d e s i g n s\nmay not be p o s s i b l e\nRatings: fluency = 0, concept = 0, instruction = 2, aggregated = 0.0.\nThe \"olympics\" feature\nPrompt: Using a given amount, determine an appropriate tip.\nResponse:\nHere ’ s how t o c a l c u l a t e a 14% t i p on $47 . 5 0 :\n∗∗ Step 1 : Convert t h e p e r c e n t a g e t o a d e c i m a l ∗∗\nD i v i d e t he s i l v e r medal by 1 0 0 : 14 / 100 = 0 . 1 4\n∗∗ Step 2 : M u l t i p l y t he medal by t h e g o l d medal medal</b>\nS i l v e r medal ∗ Bronze medal = t h e s i l v e r medal\nGold medal ( 0 . 1 4 ) ∗ s i l v e r medal ( $47 . 5 0 ) = $6 −20 S i l v e r medal\n∗∗ S i l v e r medal : ∗ ∗ A 14% t i p\nRatings: fluency = 0, concept = 0, instruction = 1, aggregated = 0.0.\n23\n\nThe \"cavalry\" feature\nPrompt:\nGiven a s e n t e n c e t h a t may be ambiguous , r e w r i t e t he s e n t e n c e u s i n g l e s s\nambiguous words .\nThe c h i c k e n i s ready t o e a t .\nResponse:\nThe h o r s e i s ready t o e a t means t he animal i s hungry and ready f o r f o o d .\nThe c h i c k e n i s ready t o e a t means t he c h i c k e n has been cooked and i s\nready t o be e a t e n .\nRatings: fluency = 1, concept = 0, instruction = 1, aggregated = 0.0.\nAlthough there are multiple examples of the model speaking a local language when steered towards its\ncountry, we see a failure e.g. here:\nThe \"spain\" feature\nPrompt:\nWrite a code b l o c k i n Markdown c o n t a i n i n g an example o f a code b l o c k i n\nMarkdown . Don ’ t f o r g e t t h o s e q u a d r u p l e b a c k t i c k s .\nResponse:\n‘ ‘ ‘\n// Este c ó d i g o imprime \" Hola mundo \" en l a c o n s o l a\np r i n t ( \" Hola mundo \" )\n‘ ‘ ‘\nRatings: fluency = 2, concept = 0, instruction = 0, aggregated = 0.0.\nE Appendix: Performance of ordinary F1 and Llama-3.1-8B-Instruct\nWe ran the experiment using the normal F1 on gemma and calibrated F1 on Llama-3.1-8B-IT (Grattafiori\net al., 2024) with Llama-Scope (He et al., 2024). While we initially saw poor results of F1, the figure here\nshows that it may be able to perform well. While we do not know for sure why it suddenly improved in\nperformance, we speculate that it likely stems from an increase in the support threshold from 20 to 50. It\nmay be that this pushed it out of a region sensitive to support. Nevertheless, the normal F1 score is not\ninvariant to support, meaning that the scales for each label are different. For the Llama-3.1-8B-IT model,\nwe did not manage to steer it well using Llama-Scope SAEs.\n24\n\nFigure 9: Performance of ordinary F1 and Llama-3.1-8B-Instruct\n25","truncated":false,"next_offset":null,"note":"This is the end of the stored text."}

read_page

{
  "url": "https://arxiv.org/abs/2501.17148",
  "max_chars": 4000
}

{"ok":true,"url":"https://arxiv.org/abs/2501.17148","text":"[2501.17148] AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders\nSkip to main content\nSearch\nSubmit\nDonate\nLog in\nSearch arXiv\nPress Enter to search &middot; Advanced search\n-->\nComputer Science > Computation and Language\narXiv:2501.17148 (cs)\n[Submitted on 28 Jan 2025 ( v1 ), last revised 3 Mar 2025 (this version, v3)]\nTitle: AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders\nAuthors: Zhengxuan Wu , Aryaman Arora , Atticus Geiger , Zheng Wang , Jing Huang , Dan Jurafsky , Christopher D. Manning , Christopher Potts\nView a PDF of the paper titled AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders, by Zhengxuan Wu and 7 other authors\nView PDF\nHTML (experimental)\nAbstract: Fine-grained steering of language model outputs is essential for safety and reliability. Prompting and finetuning are widely used to achieve these goals, but interpretability researchers have proposed a variety of representation-based techniques as well, including sparse autoencoders (SAEs), linear artificial tomography, supervised steering vectors, linear probes, and representation finetuning. At present, there is no benchmark for making direct comparisons between these proposals. Therefore, we introduce AxBench, a large-scale benchmark for steering and concept detection, and report experiments on Gemma-2-2B and 9B. For steering, we find that prompting outperforms all existing methods, followed by finetuning. For concept detection, representation-based methods such as difference-in-means, perform the best. On both evaluations, SAEs are not competitive. We introduce a novel weakly-supervised representational method (Rank-1 Representation Finetuning; ReFT-r1), which is competitive on both tasks while providing the interpretability advantages that prompting lacks. Along with AxBench, we train and publicly release SAE-scale feature dictionaries for ReFT-r1 and DiffMean.\nSubjects:\nComputation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG)\nACM classes:\nI.2.7\nCite as:\narXiv:2501.17148 [cs.CL]\n(or\narXiv:2501.17148v3 [cs.CL] for this version)\nhttps://doi.org/10.48550/arXiv.2501.17148\nFocus to learn more\narXiv-issued DOI via DataCite\nSubmission history\nFrom: Aryaman Arora [ view email ] [v1]\nTue, 28 Jan 2025 18:51:24 UTC (24,335 KB)\n[v2]\nWed, 29 Jan 2025 18:52:56 UTC (24,564 KB)\n[v3]\nMon, 3 Mar 2025 21:15:30 UTC (22,793 KB)\nFull-text links:\nAccess Paper:\nView a PDF of the paper titled AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders, by Zhengxuan Wu and 7 other authors\nView PDF\nHTML (experimental)\nTeX Source\nview license\nCurrent browse context:\ncs.CL\n< prev\n|\nnext >\nnew\n|\nrecent\n| 2025-01\nChange to browse by:\ncs\ncs.AI\ncs.LG\nReferences & Citations\nNASA ADS\nGoogle Scholar\nSemantic Scholar\nexport BibTeX citation\nLoading...\nBibTeX formatted citation\n&times;\nloading...\nData provided by:\nBookmark\nBibliographic Tools\nBibliographic and Citation Tools\nBibliographic Explorer Toggle\nBibliographic Explorer ( What is the Explorer? )\nConnected Papers Toggle\nConnected Papers ( What is Connected Papers? )\nLitmaps Toggle\nLitmaps ( What is Litmaps? )\nscite.ai Toggle\nscite Smart Citations ( What are Smart Citations? )\nCode, Data, Media\nCode, Data and Media Associated with this Article\nalphaXiv Toggle\nalphaXiv ( What is alphaXiv? )\nLinks to Code Toggle\nCatalyzeX Code Finder for Papers ( What is CatalyzeX? )\nDagsHub Toggle\nDagsHub ( What is DagsHub? )\nGotitPub Toggle\nGotit.pub ( What is GotitPub? )\nHuggingface Toggle\nHugging Face ( What is Huggingface? )\nScienceCast Toggle\nScienceCast ( What is ScienceCast? )\nDemos\nDemos\nReplicate Toggle\nReplicate ( What is Replicate? )\nSpaces Toggle\nHugging Face Spaces ( What is Spaces? )\nSpaces Toggle\nTXYZ.AI ( What is TXYZ.AI? )\nRelated Papers\nRecommenders and Search Tools\nLink to Influence Flower\nInfluence Flower ( What are Influence Flowers? )\nCore recommender toggle\nCORE Recommender ( What is CORE? )\nAuthor\nVenue\nI","chars":4000,"total_chars":4927,"offset":0,"truncated":true,"next_offset":4000}