What drove me to write this was a thought in my head long time after I left one of the online ad science teams in the industry - can we build models that we can "see through" directly from their coefficients, like linear models, while still being much more expressive? Turns out one such way is using the eigenvalue of a linear symmetric matrix pencil as the model:
f(x) = λₖ(A₀ + x₁A₁ + ⋯ + xₙAₙ)
With very well-known linear algebra properties, it is easy to show that spectral norms of the matrices bound the influence of each feature, just like magnitude in linear models describes a feature's strength, and that by choosing the index k and definiteness of the matrices we can control the shape of f(x).
code: https://github.com/alexshtf/spectral_neuron_paper
What drove me to write this was a thought in my head long time after I left one of the online ad science teams in the industry - can we build models that we can "see through" directly from their coefficients, like linear models, while still being much more expressive? Turns out one such way is using the eigenvalue of a linear symmetric matrix pencil as the model:
With very well-known linear algebra properties, it is easy to show that spectral norms of the matrices bound the influence of each feature, just like magnitude in linear models describes a feature's strength, and that by choosing the index k and definiteness of the matrices we can control the shape of f(x).