blogspot and syndication

[ritaxis]

I have syndications of several non-lj blogs. Two of them, Respectful of Otters, and The Early Days of A Better Nation, update rather more rarely than others. They're also both blogspot blogs. Everytime they do update I get many many posts at once -- old ones.

ANybody know how to fix this?

Comments

[brooksmoses]

The problem, fundamentally, is that Livejournal doesn't handle such things correctly -- any fix would have to be in their code, in the part of their system that keeps track of syndicated feeds. It's not something that we as users can do anything about, or that the blogs can do anything about either.

Thus, I think we're stuck with filing a help-desk support request with Livejournal about it.

[thomasyan.livejournal.com]

Oh, it's an LJ bug, not something wrong at the other end? I had assumed it wasn't LJ's fault that the nausicaa.net feed often regurgitates tons of old posts with each new post.

[brooksmoses]

I think it's an LJ bug, particularly when it's happening for infrequently-updated blogs every time they update. It may be more accurately described as "LJ's algorithms are stupid, and sometimes blog feeds do things that a smart algorithm could handle properly but a stupid one wouldn't", though -- I don't know that much about it.

My guess, and this is just a guess, is that it's a combination of how LJ figures out which posts are new, and the fact that it purges old posts from the local pseudojournal copies of the feed. The most likely way to do such a thing is to keep the "most recently updated" timestamp from the last new message, check that frequently against the feed site, and when that changes to go find the new posts and add them to the pseudojournal. And the naive algorithm for finding which posts are new is to start at the newest one, and work backwards until it gets to a post that's already in the pseudo-journal or reaches some other "this is too old" point. This will have an obvious bug if all of the past posts have already expired out of the pseudojournal....

I haven't seen enough data to adequately test the hypothesis that that's the problem, but it's the conjecture I'd start with.

On the other hand, there have been times when such regurgitations are caused by journal bugs, too. (Though I am pretty sure LJ could do a far better job of handling them, regardless.)

[sciamanna.livejournal.com]

Not uniquely LJ in any case. I used to follow The earlier days of a better nation through an RSS aggregator called Pluck, and the same thing happened (and happened with a couple other blogs too). Yes, bad handling of other journal bugs sounds very plausible as an explanation.

[del-c.livejournal.com]

I don't get that with Ken's blog, but I do sometimes with Digby's "Hullabaloo", another blogspot blog. That's a feed with all sorts of other things wrong with it, like no proper subject titles etc., which suggest to me the editor doesn't handle the RSS technology well, or doesn't want to bother with catering to his/her feed readers.