24 points | by ifz an hour ago
3 comments
I highly question this simplistic idea of high vector magnitude = high influence.
You get what you pay for. If you want to think harder and get more, https://transformer-circuits.pub/2025/attention-qk/index.htm...
I don't disagree with that. I did add an entire caveat paragraph there.
To me, it's more of a neat visualization, not something that can be used to interpret LLM behavior. Even with a lot of simplification, it can show some interesting patterns.
I highly question this simplistic idea of high vector magnitude = high influence.
You get what you pay for. If you want to think harder and get more, https://transformer-circuits.pub/2025/attention-qk/index.htm...
I don't disagree with that. I did add an entire caveat paragraph there.
To me, it's more of a neat visualization, not something that can be used to interpret LLM behavior. Even with a lot of simplification, it can show some interesting patterns.