Selenium based fetch of different sources

This commit is contained in:
Luciano Gervasoni
2025-07-08 18:18:26 +02:00
parent f729bd1cb2
commit 0cf61026e8
10 changed files with 235 additions and 31 deletions

View File

@@ -15,6 +15,8 @@
- TODO: Proxy / VPN?
- TooManyRequests, ...
- TODO: Search per locale (nl-NL, fr-FR, en-GB)
- Fetch keyword search for selenium sources
- URLs Processing -> Updates raw URLs
- Extracts title, description, content, image and video URLs, main image URL, language, keywords, authors, tags, published date
@@ -52,12 +54,10 @@
* Dev mode
```
docker compose -f docker-compose-dev.yml down -v
docker compose -f docker-compose-dev.yml build --progress=plain
docker compose -f docker-compose-dev.yml up
docker compose -f docker-compose-dev.yml up --no-deps --build
```
* Prod mode
```
docker compose down -v
docker compose build --progress=plain
docker compose up -d
docker compose up -d --no-deps --build
```