I skimmed through the book, and it's lacking the information theory foundations. For example, "trust region methods" come from maximizing the policy's relative entropy (to a reference policy) under a tournament system where high-scoring agents are exponentially likely to survive. In general, a reward is the negative bits it costs an environment to propagate an agent (multiplied by some temperature).
Real biological operant behavior isn't exactly trial and error learning.
Many factors shape and guide initial responses.
What I've noticed in some descriptions of models is the use of optimization for reinforcement to shape responses. In real organisms behavior may be controlled by short or long term outcomes, and may oscillate between this "optimization" based on schedules. This produces variability in the trials which can adjust behavior. Are we seeing these reinforcement models do this?
I skimmed through the book, and it's lacking the information theory foundations. For example, "trust region methods" come from maximizing the policy's relative entropy (to a reference policy) under a tournament system where high-scoring agents are exponentially likely to survive. In general, a reward is the negative bits it costs an environment to propagate an agent (multiplied by some temperature).
Real biological operant behavior isn't exactly trial and error learning.
Many factors shape and guide initial responses.
What I've noticed in some descriptions of models is the use of optimization for reinforcement to shape responses. In real organisms behavior may be controlled by short or long term outcomes, and may oscillate between this "optimization" based on schedules. This produces variability in the trials which can adjust behavior. Are we seeing these reinforcement models do this?
This looks like a good pre-read for Nathan Lambert's https://rlhfbook.com/
Is this riffing on Strunk and Whites: The Elements of Style?
Often referred to as "The Little Book".
Most likely not: “The Little Book of …” has been a publisher’s standby since the nineteenth century (at least).
I'm assuming it's more in line with The Little Schemer series of books (https://felleisen.org/matthias/BTLS-index.html) or maybe the little book of deep learning (https://fleuret.org/francois/lbdl.html)?
No, it's _obviously_ homage to the Little Liddel[0].
0. https://archive.org/details/lexiconabridgedf00liddrich
My mind immediately went to the "Li'l Bastard General Mischief Kit" from The Simpsons.
https://imgur.com/zMTEE
Love that masterpiece