1
0
Fork 0
Commit Graph

67 Commits

Author SHA1 Message Date
Benjamin Brahmer b1476e958a If timestamp is null use timestamp "one year ago"
if timestamp is not set during creation of a feed use date one year ago
code fixes and linting fixes.

Co-authored-by: Sean Molenaar <SMillerDev@users.noreply.github.com>
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2023-03-23 12:12:06 +01:00
Robert Wunderer eeabb4189b Use httpLastModified field for If-Modified-Since header when fetching feed updates
Signed-off-by: Robert Wunderer <robert.wunderer@caprisys.at>
2023-03-23 12:12:06 +01:00
Benjamin Brahmer be95c17f46 store useragent in one place
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2023-03-23 12:11:38 +01:00
Sean Molenaar e5f75d7a4c fix: do not request favicon for empty base URL
Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2023-02-14 10:18:08 +01:00
Benjamin Brahmer 502e4b923e don't process link if it's null
don't process body of feed if it's null

Co-authored-by: Sean Molenaar <SMillerDev@users.noreply.github.com>
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2023-01-12 18:27:32 +01:00
Benjamin Brahmer c09cca75b5 Fix deprecations
add phpstan deprecation rules

Co-authored-by: Sean Molenaar <SMillerDev@users.noreply.github.com>
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2022-08-19 09:14:27 +02:00
Benjamin Brahmer 1c91928e59
If items of feed do not provide an author fallback to feed author (#1803)
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2022-05-31 15:46:01 +02:00
Accalia Elementia fdf037e728
Use Feed Link as GUID when Feed omits Guid. (#1785)
* Use Feed Link as GUID when Feed omits Guid.

As noted in nextcloud/news#1702 some feeds omit the GUID and are
therefore not a valid RSS feed.

nextcloud/news#1738 resolved the issue to allow valid feeds to
update correctly when an invalid feed is present.

This commit allows parsing of the invalid feed as well by assuming
that the item link of the feed is unique to the feed and using
it in place of the GUID when the feed omits the GUID.

This will allow NextCloud News to accept and behave like many other
popular feed aggregators when presented with such an invalid feed.

Signed-off-by: Accalia <Accalia@Elementia.me>

* Add basic Logging when using fallback guid

Signed-off-by: Accalia <Accalia@Elementia.me>

* Add basic Logging when using fallback guid - Fix Fatfinger Typo

Signed-off-by: Accalia <Accalia@Elementia.me>

* Add basic Logging when using fallback guid - Update tests to account for additional logging

Signed-off-by: Accalia <Accalia@Elementia.me>
2022-05-24 17:07:50 +02:00
nextcloud486153 032d48ad7b Fix malformed feeds (without GUIDs) stopping the update process (fixes #1738)
Signed-off-by: nextcloud486153 <78801830+nextcloud486153@users.noreply.github.com>
2022-04-18 13:16:01 +02:00
Benjamin Brahmer a5eaf9ec42 Use rawurlencode as it encodes spaces correctly as %20
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2022-02-27 20:04:14 +01:00
Michael Hamann f071c05a48 Fix catching connect exceptions while fetching feed logos #1570
Display the message instead of the response when catching an exception
during feed logo fetching. As there is no response and thus no method
`getResponse()` in `ConnectException`, this fixes a "call to undefined
method"-error that prevented feeds from fetching if fetching the logo
resulted in a `ConnectException`.

Signed-off-by: Michael Hamann <michael@content-space.de>
2021-11-30 23:39:39 +01:00
Benjamin Brahmer 9f3b63b5f9 Catch network errors while fetching feed logos, fixes #1570
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2021-11-14 20:16:16 +01:00
Benjamin Brahmer 137af71bb3 Smarter error logging
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>

Co-authored-by: Sean Molenaar <SMillerDev@users.noreply.github.com>
2021-10-18 17:57:18 +02:00
Benjamin Brahmer 74586396f8 make use of constant
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>

Co-authored-by: Sean Molenaar <SMillerDev@users.noreply.github.com>
2021-10-18 17:57:18 +02:00
Benjamin Brahmer 59a888481f make use of base_url to handle relative urls
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2021-10-18 17:57:18 +02:00
Benjamin Brahmer 7d61a1cb09 Download feed logos via guzzle to have better error handling
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2021-10-18 17:57:18 +02:00
Benjamin Brahmer 56ebffbf76 [Backport] Prevent some of the favicon fetching errors
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2021-05-03 10:19:23 +02:00
anoy 91db7113a8 replace deprecated getDescription call
Signed-off-by: anoy <anoymouserver+github@mailbox.org>
2021-04-24 14:21:57 +02:00
Chris Danser 6e845124df
Null check category label before trying to decode (#1282)
* Null check category label before trying to decode

Signed-off-by: skiingwiz <skiingwiz@gmail.com>
2021-04-08 08:10:41 +02:00
Sean Molenaar 2baa7a2b91 Make PHPstan stricter
Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2021-03-25 18:47:38 +01:00
Marco Nassabain 7e0aab358c 🎨 Item: store categories as json & add helper fns
+ added setter/getters that work with arrays to simplify use case

Signed-off-by: Marco Nassabain <marco.nassabain@hotmail.com>
2021-03-17 08:58:42 +01:00
Jimmy Huynh 3dfbf64a6b ♻️ Renaming 'tags' into 'categories'
Signed-off-by: Jimmy Huynh <jimmy.huynh@etu.unistra.fr>
2021-03-17 08:58:42 +01:00
Jimmy Huynh fd13be0f7a Update FeedFetcher: import 'tags' to 'item'
Signed-off-by: Jimmy Huynh <jimmy.huynh@etu.unistra.fr>
2021-03-17 08:58:42 +01:00
Benjamin Brahmer 6cd83ea1da remove unused lastModified from feed fetcher
Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2021-02-16 11:01:10 +01:00
Benjamin Brahmer c09b4d8d33 prefer the feeds logo over the favicon
The logo of the feed is prefered if it is a square picture,
else the favicon is returned.

Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
Co-authored-by: Sean Molenaar <SMillerDev@users.noreply.github.com>
2021-02-16 10:16:15 +01:00
Sean Molenaar b4fa772bc5 Remove V1 item API
Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2021-02-13 13:22:57 +01:00
Sean Molenaar 8fa1537cbf Fetcher: Update client and add test
Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2021-01-20 22:52:09 +01:00
Sean Molenaar 67b6c4e1b0 Fix psalm issues
Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2021-01-12 13:29:08 +01:00
Sean Molenaar 05377d023e Remove PHPunit integration tests
Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2020-12-30 11:21:44 +01:00
Kevin Decherf 42ea24f2f4 Remove LastModified-based cursor when updating feeds
We remove the call to readSince() as some feeds push new articles with
pubDate prior to the lastModified time stored for these feeds (e.g.
lemonde.fr). As we go through all items of a feed again and again, we
prevent the constant update of an item's lastModified timestamp by
keeping the previous one if its fingerprint does not change.

Fixes #921

Signed-off-by: Kevin Decherf <kevin@kdecherf.com>
2020-12-15 20:02:21 +01:00
anoy cb32a2c4ed remove deprecated YouTube playlist API
Signed-off-by: anoy <anoymouserver+github@mailbox.org>
2020-11-07 09:57:20 +01:00
Sean Molenaar 48a130d3c0 Allow titles to be null
Issue GH-867

Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2020-10-12 21:40:23 +02:00
Sean Molenaar e528361141 Add feed autodiscovery
Issue GH-415

Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2020-09-29 21:15:53 +02:00
Sean Molenaar 35b53ecd40 OPML export command and fixes
Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2020-09-29 14:56:07 +02:00
Sean Molenaar bc01761221 Fix repair step and test it
Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2020-09-29 13:02:03 +02:00
Sean Molenaar d00d1ab2a2 Create V2 mapper, Service and management commands
Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2020-09-27 15:35:31 +02:00
Sean Molenaar 60ab4941cc Move to nextcloud config and update phpunit
Signed-off-by: Sean Molenaar <sean@seanmolenaar.eu>
2020-09-25 19:18:04 +02:00
Gioele Falcetti bf0e693e23 Fix crash if full-text is enabled and the news item does not provide an URL to be fetched
Signed-off-by: Gioele Falcetti <thegio.f@gmail.com>
2020-08-31 00:59:43 +02:00
Chris Noxz 81d2407e6c
fix 'If-Modified-Since' causing BAD REQUEST (#684)
fix 'If-Modified-Since' causing BAD REQUEST

Signed-off-by: Chris Noxz <chris@noxz.tech>
Co-authored-by: kesselb <mail@danielkesselberg.de>
2020-04-28 18:25:34 +02:00
Éloi Rivard 306d3cdc60
Basic Media-RSS support (#599)
Signed-off-by: Éloi Rivard <azmeuk@gmail.com>
2020-02-16 14:26:06 +01:00
kesselb f045bd0ba3 Update httpLastModified from the feed response (#594)
LastModified is used (in a feed context to see if a user edited a feed). httpLastModified to store the last-modified response from the source.

Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-12-27 03:51:06 +01:00
DriverXX 6673cbc3d9 Reimplement full-text scraping (#563)
Add readability.php scraper
Fixes #482
Signed-off-by: Gioele Falcetti <thegio.f@gmail.com>
2019-12-24 09:33:19 +01:00
b_b 35c6c91e26 Get content:encoded of item if available (#565)
Get content:encoded of item if available fixes #564

Signed-off-by: brunob <bruno@eliaz.fr>
2019-11-16 14:21:40 +01:00
Benjamin Brahmer ae99d52aad
Update to feedio 4+ (#494)
- bumped dependencies to last supported version
travis:
- removed php 7.0
- move main target to nc 16
- drop support for nc 14 & 15

Signed-off-by: Benjamin Brahmer <info@b-brahmer.de>
2019-08-16 07:57:34 +02:00
Sean Molenaar b37e237c7a Fix xkcd commics 2019-03-26 09:31:03 +01:00
Sean Molenaar 62bd42cb17 Stop assuming encoding of xml, let simplexml throw silently (#457) 2019-03-22 09:19:05 +01:00
Sean Molenaar c69bcb8e68 Allow empty-ish lastmodified and clean up FeedFetcher Test (#458) 2019-03-21 05:16:35 +01:00
Jasper Knockaert 84af098d01 reset modified since filter when updating feeds
When updating feeds the modified since date is added as a filter. This needs to be reset when looping over the feeds. Fixes #444.
2019-03-20 07:51:45 +01:00
Sean Molenaar 9b915e24af Use a copy of the FeedIO client that doesn't specify a useragent 2019-03-18 10:39:32 +01:00
Sean Molenaar 2698214c41 fix/allow CDATA encoding (#428) 2019-03-17 08:23:37 +01:00