Is next token prediction the new "E = mc2"?
4 by dbb_io | 2 comments on Hacker News.
History has taught us that the most significant discoveries of formulas are often those that seem to be the simplest equations (even seem too simplistic to be true) but generalize the most. Take Einstein's E = mc2 as an example. My (maybe purely philosophical) hypothesis is now: Next token prediction might be for AI, what "E = mc2" has been for physics. When Ilya Sutskever was asked in an interview what surprised in the most regarding the rapid advances in AI, his answer was, that "it" (meaning large language models and especially next token prediction) works at all. And I agree. While it is easy to understand what next token prediction is doing, it is mindblowing what outputs can be achieved with such simplistic formula. I'd like to discuss - do you think it's a valid statement to say "Next token prediction is the new E = mc2"?

Post a Comment

Previous Post Next Post