As a writer I must just admire how you describe "our peacock feathers, the inevitable cultural exhaust that human lives create in the process of living." The whole way we use technology (and always have) suggests we are far more interested in being sociable than being factual, and so no matter how "good" a LLM gets, until it has social status and position, no one will care what it has to "say." :)
Hi Rohit Thank you for this response to the Hoel article. The discussions around the quality of training data sets is interesting. I’m assuming the training datasets mostly contain explicit knowledge (and other human chatter) which is now cheap and abundant. But the training sets don’t (can’t?) contain much tacit knowledge, which is necessary for most high value tasks. Could this be part of the reason why LLM’s are where they are now? (I have no experience in AI so I am just talking conceptually, please bare with me)
The tacit part gets interpolated from the amount of data that's available but naturally has limits. As Wittgeinstein said about philosophy, whereof one cannot speak, thereof one must be silent.
I think the very act of democratising supply here naturally creates a lot more "dreck", but it also produces a lot more high art. Proportionally it might be the same or even better but it's hard for us to search and find them.
As a writer I must just admire how you describe "our peacock feathers, the inevitable cultural exhaust that human lives create in the process of living." The whole way we use technology (and always have) suggests we are far more interested in being sociable than being factual, and so no matter how "good" a LLM gets, until it has social status and position, no one will care what it has to "say." :)
Thank you! There's a theory that everything is social media, though it might be too bleak for polite company.
Hi Rohit Thank you for this response to the Hoel article. The discussions around the quality of training data sets is interesting. I’m assuming the training datasets mostly contain explicit knowledge (and other human chatter) which is now cheap and abundant. But the training sets don’t (can’t?) contain much tacit knowledge, which is necessary for most high value tasks. Could this be part of the reason why LLM’s are where they are now? (I have no experience in AI so I am just talking conceptually, please bare with me)
The tacit part gets interpolated from the amount of data that's available but naturally has limits. As Wittgeinstein said about philosophy, whereof one cannot speak, thereof one must be silent.
China took a different AI path and applied it first to commerce, industry and logistics, leveraging its massive investment in 5G.
Since it’s best LLMs are as good as ours, they’re now well ahead in the field.
They were playing catchup not long ago in the generative AI space though their latest models seem pretty good
Is that true? We have a lot more artists than then as well. Just spotify had around 3k artists making 100k+ last year.
I think the very act of democratising supply here naturally creates a lot more "dreck", but it also produces a lot more high art. Proportionally it might be the same or even better but it's hard for us to search and find them.