matitos/matitos_news
1
0 Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity
174 Commits 2 Branches 0 Tags
9323afd68fd90fd3416058258b3b06afed82421c
Commit Graph
87 Commits
Author SHA1 Message Date
Luciano Gervasoniandopencode 9323afd68f Make optional search imports lazy to avoid GitHub dependency in CI
Tests / test (pull_request) Failing after 3m29s
Details 
Wrap gnews, GoogleNews, and search_engines imports in try/except
so the module loads without those packages installed. This allows
the test suite to run on the Gitea runner without outbound GitHub
access.

Also revert requirements.txt back to the GitHub URL for the
Search-Engines-Scraper dependency.

Co-Authored-By: opencode <noreply@opencode.ai>
 2026-09-02 12:00:14 -03:00
Luciano Gervasoni d18eda2fc4 Add unit tests and Gitea CI workflow for bug fixes
Tests / test (pull_request) Failing after 3m11s
Details 
- Add fetcher/tests/ with 21 tests covering all bug fixes:
  - test_utils: get_with_protocol URL normalization (5 tests)
  - test_config: scheduled_tasks.json PK uniqueness, Celery queue
    settings (7 tests)
  - test_publish: URL routing and view for /task/publish_<id>/ (4 tests)
  - test_template_field: url_detail.html renders images_url field (2 tests)
  - test_search_postprocess: URL host filtering and param stripping (3 tests)
- Replace empty fetcher/tests.py with tests/ package
- Add .gitea/workflows/test.yml: runs on PRs and pushes to main,
  installs dependencies and runs `manage.py test fetcher.tests`
 2026-09-02 09:42:07 -03:00
Luciano Gervasoni 3a33697ca3 Fix critical bugs in app_urls 
- init_db.py: parameterize SQL queries to prevent SQL injection
- scheduled_tasks.json: resolve duplicate periodic task pk (4 -> 15)
- db_utils.py: use function parameter in _get_status_pattern_matching
  instead of capturing the enclosing scope variable
- url_detail.html: fix wrong field name image_urls -> images_url
- settings.py: align Celery queues (default/light/heavy) with the task
  queues and supervisord workers
- Add /task/publish_<id>/ route to fix broken publish links in templates
 2026-09-01 15:08:14 -03:00
Luciano Gervasoni cbc422df36 Notify status bot token 2025-10-16 10:38:32 +02:00
Luciano Gervasoni d8ef738d19 Notifier fix 2025-10-14 13:10:54 +02:00
Luciano Gervasoni 2f035a4222 Notifier fix 2025-10-14 12:23:05 +02:00
Luciano Gervasoni 7924857fe5 Schools NL tuples, traceback on notify err 2025-10-14 11:33:17 +02:00
Luciano Gervasoni f44b784715 Notifications, info and warning, try catch 2025-09-09 22:06:23 +02:00
Luciano Gervasoni 24510d26e2 Notifications, info and warning 2025-09-08 17:55:03 +02:00
Luciano Gervasoni ef51a96db6 Process missing kids url based on API endpoint, fix2 2025-09-08 16:20:39 +02:00
Luciano Gervasoni 079b2473f8 Process missing kids url based on API endpoint 2025-09-08 16:12:27 +02:00
Luciano Gervasoni 1fbc5beb6e URL Logs 2025-09-08 12:45:53 +02:00
Luciano Gervasoni 2ed86e31ec Workers light,default,heavy 2025-09-08 12:34:47 +02:00
Luciano Gervasoni 892fb984d1 Debug enlarge 2025-09-05 14:18:33 +02:00
Luciano Gervasoni c17f09a94f Debug enlarge 2025-09-05 14:06:10 +02:00
Luciano Gervasoni e4a325d6b4 Request timeout debugging 2025-09-05 14:00:50 +02:00
Luciano Gervasoni 2fae0a3a9d Request timeout 2025-09-05 13:52:34 +02:00
Luciano Gervasoni 35f9260b94 Debug process raw url 2025-09-05 13:45:16 +02:00
Luciano Gervasoni a0ced90d7c Views base fix 2025-08-22 13:26:14 +02:00
Luciano Gervasoni 883dfcd3bd URL redirect get before newspaper processing 2025-08-22 13:11:02 +02:00
Luciano Gervasoni b08ea574d9 Notify status task 2025-08-14 15:13:18 +02:00
Luciano Gervasoni da078a6f0f Notify status task 2025-08-14 15:09:20 +02:00
Luciano Gervasoni 4ccff2bc02 Notify status task 2025-08-14 15:06:53 +02:00
Luciano Gervasoni ffb0f85475 View fix (10) 2025-08-14 14:58:16 +02:00
Luciano Gervasoni c939624687 View fix (9) 2025-08-14 14:56:00 +02:00
Luciano Gervasoni 6fb14d5e72 View fix (8) 2025-08-14 14:53:56 +02:00
Luciano Gervasoni 260a505766 View fix (7) 2025-08-14 14:50:03 +02:00
Luciano Gervasoni bae5329b1e View fix (6) 2025-08-14 14:48:03 +02:00
Luciano Gervasoni 1fbf4cf3d4 View fix (4) 2025-08-14 14:25:56 +02:00
Luciano Gervasoni 856a9e7562 View fix (3) 2025-08-14 14:24:03 +02:00
Luciano Gervasoni 4080154f2b View fix (2) 2025-08-14 14:12:57 +02:00
Luciano Gervasoni 015f92a06b View fix 2025-08-14 14:04:33 +02:00
Luciano Gervasoni 3d09c1acff Notify status 2025-08-14 13:56:06 +02:00
Luciano Gervasoni 02f756d3c2 Ride missing kids exception 2025-08-14 10:59:19 +02:00
Luciano Gervasoni e3d6cf8000 Pattern matching, foxnews request with header 2025-08-13 14:12:54 +02:00
Luciano Gervasoni 54e41139bb Duckduckgo search update 2025-07-22 22:53:53 +02:00
Luciano Gervasoni b112da8bd0 Supervisor based run 2025-07-22 00:51:09 +02:00
Luciano Gervasoni cb621c9d6b Switching to django celery for workers 2025-07-17 22:29:06 +02:00
Luciano Gervasoni 50e8666162 Django tasks workers logger 2025-07-17 00:46:48 +02:00
Luciano Gervasoni b9ba0d8f3d Logger for worker 2025-07-15 16:51:22 +02:00
Luciano Gervasoni 6612a50d13 Logger fix, env sample ram 2025-07-14 23:48:34 +02:00
Luciano Gervasoni 6c88759e7b Workers ttl 2025-07-14 23:37:48 +02:00
Luciano Gervasoni 0cb68a876b Logger parent pid 2025-07-10 14:19:57 +02:00
Luciano Gervasoni fdc3263785 Django multi worker, logging pid 2025-07-10 13:08:37 +02:00
Luciano Gervasoni 0fa4482711 Missing kids trigger types 2025-07-08 21:30:20 +02:00
Luciano Gervasoni 4985f09e56 Filters fix 2025-07-08 18:28:43 +02:00
Luciano Gervasoni 0cf61026e8 Selenium based fetch of different sources 2025-07-08 18:18:26 +02:00
Luciano Gervasoni 522c1cb8b3 Missing kids selenium fixes 2025-07-08 09:43:40 +02:00
Luciano Gervasoni dd8e71aaa3 Missing kid verify timeout handle 2025-07-07 16:51:55 +02:00
Luciano Gervasoni 8cf2b52325 Selenium based missing kid verify url fix (2) 2025-07-07 16:34:21 +02:00