I've been thinking of how to tackle that problem. It would require a bit of resources but nothing too crazy. Essentially new articles need to be indexed in some kind of vector search capable DB. That allows things like similarity grouping and a few other things. This is nothing new and exactly how things like Google News work. The difference here would be keeping the per user notion of subscribing only to things they care about.
If you do the embeddings calculation centrally, it becomes shared cost. Every new article gets analyzed only once for all users.
The rest then becomes providing a new view on your RSS feeds that leverages that. You could do a lot of the expensive stuff (vector comparisons) locally actually because most users only have hundreds/thousands of articles that they care about. So, simply download the embeddings for articles and do the comparisons/grouping locally.
This wouldn't be super hard to do. There are lots of OSS models that you can run locally as well. But they are kind of slow. So the trick is to amortize that over many users and share the burden.
The key challenge here is the finances. The centralized embeddings juggling gets costly quickly and you need a revenue model to finance that. That's why most of this stuff is happening by paywalls and staying kind of niche. All the "free" stuff is essentially ad sponsored.
But with some MCP layered on top and a few other bits and bobs, you could fairly easily implement an intelligent LLM based news agent that summarizes personalized news based on exactly your own preferences and news subscriptions. I haven't really seen anything like this done right. But we technically have all the OSS tech and models to do all of this now. It's just the compute cost that kills the use case.
If that could be decentralized bittorrent style, it wouldn't actually be that much of a burden. Given enough users, distributing say thousands of article updates per minute among tens/hundreds of thousands of readers means each of them expending maybe a couple of seconds of compute once in a while to calculate embeddings for articles that they are pulling that don't have embeddings yet. If you make that eventually consistent, it's not that big of a deal if you don't get embeddings for all the new stuff right away. And any finished embeddings could be uploaded and shared. Anything popular would quickly get embeddings. And you could make the point that publishers themselves could be providing embeddings as well for their own articles. Why not? If you only publish a handful of articles, the cost is next to nothing.
If I had more spare time, I might have a go at this. Sadly, I don't.
It's very interesting and thanks for laying out the issues.
One point I'd like to make: Grouping RSS feed items by similarity is much different than LLM summaries. In the former, I get the best (I can use economically), specific, expert human information, which is what I'm personally after; the latter keeps the economy but eliminates the value.
If you do the embeddings calculation centrally, it becomes shared cost. Every new article gets analyzed only once for all users.
The rest then becomes providing a new view on your RSS feeds that leverages that. You could do a lot of the expensive stuff (vector comparisons) locally actually because most users only have hundreds/thousands of articles that they care about. So, simply download the embeddings for articles and do the comparisons/grouping locally.
This wouldn't be super hard to do. There are lots of OSS models that you can run locally as well. But they are kind of slow. So the trick is to amortize that over many users and share the burden.
The key challenge here is the finances. The centralized embeddings juggling gets costly quickly and you need a revenue model to finance that. That's why most of this stuff is happening by paywalls and staying kind of niche. All the "free" stuff is essentially ad sponsored.
But with some MCP layered on top and a few other bits and bobs, you could fairly easily implement an intelligent LLM based news agent that summarizes personalized news based on exactly your own preferences and news subscriptions. I haven't really seen anything like this done right. But we technically have all the OSS tech and models to do all of this now. It's just the compute cost that kills the use case.
If that could be decentralized bittorrent style, it wouldn't actually be that much of a burden. Given enough users, distributing say thousands of article updates per minute among tens/hundreds of thousands of readers means each of them expending maybe a couple of seconds of compute once in a while to calculate embeddings for articles that they are pulling that don't have embeddings yet. If you make that eventually consistent, it's not that big of a deal if you don't get embeddings for all the new stuff right away. And any finished embeddings could be uploaded and shared. Anything popular would quickly get embeddings. And you could make the point that publishers themselves could be providing embeddings as well for their own articles. Why not? If you only publish a handful of articles, the cost is next to nothing.
If I had more spare time, I might have a go at this. Sadly, I don't.