Open source models exist and can be run locally. It doesn't matter if training from scratch is impractical for the average person, if we already have free (as in freedom) models we can build off of.
I have several "open" models sitting around for experimentation and occasional use, but I don't think downloadable weights solves the underlying issue.
First off, none of the good models are FOSS in the sense we normally expect - i.e. the four freedoms. At the least onerous end of the scale, Stable Diffusion models under the OpenRAIL license have a morality clause[0] and technical protections[1] to enforce that clause. LLaMA's licensing is only open for entities below a certain MAU, and Stable Diffusion 3 recently switched away from OpenRAIL to a LLaMA-like "free as in beer" license. Not only is this non-free, it's getting increasingly more proprietary as entities who are paying for the AI training start demanding a return on investment, and the easiest way to get that is to just demand licensing fees.
The reason why AI companies - not the artists or programmers they stole from - are in a position to demand these licensing terms at all is because they're the ones controlling the capital necessary to train models. If training from scratch at the frontier was still viable for FOSS tinkerers, we wouldn't have to worry about OpenAI reading all our GPT queries or Stability finding ways to put asterisks on their openness branding. FOSS software development is something you can do as a hobby, so if a project screws something up, you can build a new one. That's not how AI works. If Stability screws up, you still have to obey Stability's rules unless you train a new foundation model, and that's very expensive.
You see how this is very "copyright-like" even though it's legally contravening the letter and spirit of copyright law? Barriers to entry drive industrial consolidation and ratchet us towards a capitalist, privately-owned command economy. If I could train models from scratch, I'd make a decent model trained purely on public domain datasets[2], slather Grokfast[3] on it, and build in some UI to selectively fine-tune on licensed or custom data.
[0] To be clear, I don't have much against morality clauses personally, that's why I've used OpenRAIL models. But I still think adding morality clauses to otherwise FOSS licensing is a bad idea. At the very least, in order to put moral values into a legal contract, we have to agree as a community as to what moral values should be enforced by copyright. Furthermore, copyright and contracts are a bad tool to enforce morals.
[1] e.g. the Stable Diffusion safety filter
[2] Believe me, I tried
[3] An algorithm that increases the speed of grokking (generalization) by taking the FFT of gradients and amplifying the slow ones.
