'AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference' comes out on 2024/9/24. I will consider it out of date if >50% of the things it claims AI can't do can be done by then.
Protocol (rough draft, open to improvements):
Have LLM(s) extract concrete claims of what AI can't do from the book
Randomly sample a few (10-20?)
Have 2 people independently assess whether AI can do them (YES/NO) and give half credit in case of disagreement
@JavierPrieto if your first reaction to the notion of reading a book (ostensibly within your fields of interest) is "what is the financial opportunity cost" then I genuinely, sincerely feel bad for you. but ok
@Radicalia this market isn't for "untitled median AI policy/governance book". digesting thousands of titles within an arbitrary field of work in order to extract common assumptions would be an excellent application for an LLM