matitos/matitos_news
1
0 Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity
173 Commits 2 Branches 0 Tags
fa9d552873a88494882a8beedb81c6d1cc895443
Commit Graph
138 Commits
Author SHA1 Message Date
Luciano Gervasoniandopencode fa9d552873 Point Search-Engines-Scraper dependency to Gitea mirror
Tests / test (pull_request) Successful in 7m9s
Details 
The Gitea CI runner has no outbound access to github.com.
Mirror the repository on the local Gitea instance so pip install
succeeds in CI.

Co-Authored-By: opencode <noreply@opencode.ai>
 2026-09-02 11:39:31 -03:00
Luciano Gervasoni d18eda2fc4 Add unit tests and Gitea CI workflow for bug fixes
Tests / test (pull_request) Failing after 3m11s
Details 
- Add fetcher/tests/ with 21 tests covering all bug fixes:
  - test_utils: get_with_protocol URL normalization (5 tests)
  - test_config: scheduled_tasks.json PK uniqueness, Celery queue
    settings (7 tests)
  - test_publish: URL routing and view for /task/publish_<id>/ (4 tests)
  - test_template_field: url_detail.html renders images_url field (2 tests)
  - test_search_postprocess: URL host filtering and param stripping (3 tests)
- Replace empty fetcher/tests.py with tests/ package
- Add .gitea/workflows/test.yml: runs on PRs and pushes to main,
  installs dependencies and runs `manage.py test fetcher.tests`
 2026-09-02 09:42:07 -03:00
Luciano Gervasoni 3a33697ca3 Fix critical bugs in app_urls 
- init_db.py: parameterize SQL queries to prevent SQL injection
- scheduled_tasks.json: resolve duplicate periodic task pk (4 -> 15)
- db_utils.py: use function parameter in _get_status_pattern_matching
  instead of capturing the enclosing scope variable
- url_detail.html: fix wrong field name image_urls -> images_url
- settings.py: align Celery queues (default/light/heavy) with the task
  queues and supervisord workers
- Add /task/publish_<id>/ route to fix broken publish links in templates
 2026-09-01 15:08:14 -03:00
Luciano Gervasoni cbc422df36 Notify status bot token 2025-10-16 10:38:32 +02:00
Luciano Gervasoni dc784dabec Notify status schedule 2025-10-16 10:12:17 +02:00
Luciano Gervasoni d8ef738d19 Notifier fix 2025-10-14 13:10:54 +02:00
Luciano Gervasoni 2f035a4222 Notifier fix 2025-10-14 12:23:05 +02:00
Luciano Gervasoni 7924857fe5 Schools NL tuples, traceback on notify err 2025-10-14 11:33:17 +02:00
Luciano Gervasoni f44b784715 Notifications, info and warning, try catch 2025-09-09 22:06:23 +02:00
Luciano Gervasoni 24510d26e2 Notifications, info and warning 2025-09-08 17:55:03 +02:00
Luciano Gervasoni ef51a96db6 Process missing kids url based on API endpoint, fix2 2025-09-08 16:20:39 +02:00
Luciano Gervasoni 079b2473f8 Process missing kids url based on API endpoint 2025-09-08 16:12:27 +02:00
Luciano Gervasoni 1fbc5beb6e URL Logs 2025-09-08 12:45:53 +02:00
Luciano Gervasoni 2ed86e31ec Workers light,default,heavy 2025-09-08 12:34:47 +02:00
Luciano Gervasoni 892fb984d1 Debug enlarge 2025-09-05 14:18:33 +02:00
Luciano Gervasoni c17f09a94f Debug enlarge 2025-09-05 14:06:10 +02:00
Luciano Gervasoni e4a325d6b4 Request timeout debugging 2025-09-05 14:00:50 +02:00
Luciano Gervasoni 2fae0a3a9d Request timeout 2025-09-05 13:52:34 +02:00
Luciano Gervasoni 35f9260b94 Debug process raw url 2025-09-05 13:45:16 +02:00
Luciano Gervasoni 346d7c9187 Debug workers 2025-09-04 09:04:04 +02:00
Luciano Gervasoni a21ff9c5d6 Celery scheduler DB based 2025-09-04 08:46:04 +02:00
Luciano Gervasoni a9074f45b5 Redis cache and celery, avoid overflow 2025-09-03 23:07:03 +02:00
Luciano Gervasoni 569e7d4676 Disable fetch missing kids all 2025-08-28 11:23:47 +02:00
Luciano Gervasoni 4883b097db Disable missing kids all urls check 2025-08-26 10:16:18 +02:00
Luciano Gervasoni a0ced90d7c Views base fix 2025-08-22 13:26:14 +02:00
Luciano Gervasoni 883dfcd3bd URL redirect get before newspaper processing 2025-08-22 13:11:02 +02:00
Luciano Gervasoni 9b0a84c16a Books category 2025-08-18 16:39:26 +02:00
Luciano Gervasoni b08ea574d9 Notify status task 2025-08-14 15:13:18 +02:00
Luciano Gervasoni da078a6f0f Notify status task 2025-08-14 15:09:20 +02:00
Luciano Gervasoni 4ccff2bc02 Notify status task 2025-08-14 15:06:53 +02:00
Luciano Gervasoni ffb0f85475 View fix (10) 2025-08-14 14:58:16 +02:00
Luciano Gervasoni c939624687 View fix (9) 2025-08-14 14:56:00 +02:00
Luciano Gervasoni 6fb14d5e72 View fix (8) 2025-08-14 14:53:56 +02:00
Luciano Gervasoni 260a505766 View fix (7) 2025-08-14 14:50:03 +02:00
Luciano Gervasoni bae5329b1e View fix (6) 2025-08-14 14:48:03 +02:00
Luciano Gervasoni b3d63f820e View fix (5) 2025-08-14 14:28:52 +02:00
Luciano Gervasoni 1fbf4cf3d4 View fix (4) 2025-08-14 14:25:56 +02:00
Luciano Gervasoni 856a9e7562 View fix (3) 2025-08-14 14:24:03 +02:00
Luciano Gervasoni 4080154f2b View fix (2) 2025-08-14 14:12:57 +02:00
Luciano Gervasoni 015f92a06b View fix 2025-08-14 14:04:33 +02:00
Luciano Gervasoni 3d09c1acff Notify status 2025-08-14 13:56:06 +02:00
Luciano Gervasoni 02f756d3c2 Ride missing kids exception 2025-08-14 10:59:19 +02:00
Luciano Gervasoni 6b5073d1b6 Pattern matching, foxnews request with header 2025-08-13 14:29:44 +02:00
Luciano Gervasoni e3d6cf8000 Pattern matching, foxnews request with header 2025-08-13 14:12:54 +02:00
Luciano Gervasoni 1502f09e22 Selenium kill process to release mem, supervisor conf rotate log file 2025-07-28 11:16:15 +02:00
Luciano Gervasoni 54e41139bb Duckduckgo search update 2025-07-22 22:53:53 +02:00
Luciano Gervasoni b112da8bd0 Supervisor based run 2025-07-22 00:51:09 +02:00
Luciano Gervasoni cb621c9d6b Switching to django celery for workers 2025-07-17 22:29:06 +02:00
Luciano Gervasoni 50e8666162 Django tasks workers logger 2025-07-17 00:46:48 +02:00
Luciano Gervasoni 202e58776d Django tasks workers 2025-07-17 00:21:26 +02:00