This piece is brilliant throughout…
Iotas
This piece is brilliant throughout. We at Requisite Variety particularly enjoyed the thoughtfulness around how benchmarks are subject to frames (“Once the current…benchmark saturates, we’ll change the frame to zero it out again”) as well as agents and agency (“agency…is not just action. It is wanting for oneself….Model compliance and helpfulness are fundamentally at odds with this kind of agency, so even as the models improve, the gap between models and humans will remain.”)
