Bot posts flood of old/historic items (limit not working)

[pReya]

Some RSS Feeds are not well implemented, and contain a lot of historic items. It would be nice if there was an option to ignore "old" items, and only push items that are newer than the time of subscription.

This should be optional, so there needs to be a "switch" to enable this behavior. Probably within the !rss subscribe <feed URL> command. How about !rss subscribe ignoreOld <feed URL>?

If this is something you agree with, I'd be happy to provide a PR.

[turt2live]

Sorry, I don't quite understand what behaviour you're seeing. Do you have an example/screenshots?

[pReya]

Sorry, I don't quite understand what behaviour you're seeing. Do you have an example/screenshots?

This is not a bug – sorry if this was unclear.

If I subscribe to a new feed, the bot will post all items that are part of the feed. Typically these are limited, by the creator of the feed, to contain X most recent items (something like 20 or 50 are common default values).

However some feeds contain ALL items without any limit (could be hundreds or thousands of items). Typically you don't want the bot to "spam" hundreds or thousands of messages into the channel at once. So I am suggesting a flag, which will ignore all "old" items and only post items that are newer than the time of subscription (it will not post all the existing historical items).

[turt2live]

Ah, the initial setup should only be posting ~20 feed items at most, but tracking all of them in the database (so it doesn't post about them later).

I would classify this as a bug if it's posting more than 20 at the point of initial subscription. Ideally, that number would be configurable in the config.yaml, but not as a separate command/per-room option.

[pReya]

Ah, the initial setup should only be posting ~20 feed items at most, but tracking all of them in the database (so it doesn't post about them later).

Then there seems to be a bug indeed. I tested this yesterday, and had about 370 messages in my channel, before I was able to kick the bot from the room. Also I couldn't find any such limit in the code. Maybe you forgot it during the recent refactoring?

And yes, configuring this in config file seems to be a sensible way to do this.

[turt2live]

"Recent refactoring" is kind of you :p

It was written from scratch in about a weekend - I very likely forgot to add such a limit and probably assumed I already did by the time I pushed the code live.

[ahlrodrigues]

Hi! Same news aboul Thais bug?

[turt2live]

I unfortunately don't have the time to make it work, but if someone contributes a PR then happy to get it deployed.