Pewarta Proxy

📋 Server Request & Proxy Logs

Loading active proxy settings...
Developed by Khairul Adha (adhexa.id)
Mengambil log server...

⚙️ Pengaturan Proxy Target

Skip to content

[FIX] Stop leaking DB, cache and storage details in API errors - #2400

Open
ritwik-g wants to merge 4 commits into
mainfrom
fix/codeql-backend-leaks
Open

ritwik-g wants to merge 4 commits into
mainfrom
fix/codeql-backend-leaks

Conversation

@ritwik-g

Copy link
Copy Markdown
Contributor

What

Stops three API error paths from handing internal details to callers, and deletes dead duplicate views. Split out of #2397, which now covers only x2text.

Metrics health leak (closes CodeQL #448)

GET .../metrics/health/ is open to any org member. It returned str(e) from the DB and cache checks, which names the DB/Redis host, port and user. It now returns "Database unavailable" / "Cache unavailable" instead, and still logs the exception with logger.exception. Status codes are unchanged (200/503).

Storage path in API errors (closes CodeQL #358, #359)

DestinationConnector.get_tool_execution_result_from_metadata and _read_file_content_for_queue put the full execution-storage path in the APIException detail. The caller now gets a message without the path:

  • output not found
  • output is not valid JSON
  • Failed to read file content for queue: <file name>

The path stays in the server log. The workers' copy (workers/shared/workflow/destination_connector.py) had the same leak and gets the same fix.

Dead duplicate internal views (closes CodeQL #451, #452, #453)

WorkflowExecutionInternalViewSet and FileBatchCreateInternalAPIView in workflow_v2/views.py were unrouted copies of the classes in workflow_manager/internal_views.py, which workflow_execution_internal_urls.py routes. I grepped this repo (urls, tests, workers) and unstract-cloud: nothing imports them from workflow_v2.views or patches them there. I deleted both, along with the imports only they used.

Testing

  • backend/dashboard_metrics/tests/test_health_check.py: DB failure, cache failure and the healthy case.
  • backend/workflow_manager/endpoint_v2/tests/test_destination_error_detail.py and workers/tests/test_queue_read_error_detail.py: the path is absent from the caller's message and present in the log.
  • Mutation-verified: reverting each guard makes a test fail.
  • Unit tests for dashboard_metrics, workflow_manager, inbox and the org-scoping views: 200 passed. The one failure, test_api_storage_mime_validation::test_pdf_marker_inside_other_content..., also fails on main locally because of libmagic, and is unrelated. manage.py check is clean. The workers tests pass.

🤖 Generated with Claude Code

- Metrics health check returns a fixed message instead of str(e), which
  named the DB/Redis host, port and user; the exception is still logged.
- Destination read failures (backend and the workers' copy) no longer
  put the execution-storage path in the error returned to the caller;
  the path stays in the server log.
- Delete WorkflowExecutionInternalViewSet / FileBatchCreateInternalAPIView
  from workflow_v2/views.py: unrouted copies of the classes in
  workflow_manager/internal_views.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@ritwik-g ritwik-g left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 1 — review-deep (INITIAL), tip c43401e

Review: unstract:review-deep. Lite was refused by its gate on auth_authz_secrets (lens 4).
Security deep-dive: done. The find sub-task returned no candidates at or above 0.7.
boundary-sweep: 6 sinks traced (two health messages, two destination details, the backend queue detail, the worker OSError) — clean.

Verdict — APPROVE WITH NITS

Summary — Critical: 0 · High: 0 · Medium: 0 · Low: 2 · Lenses run: 19/19

Findings (both below the severity floor)

  1. [Low] [Lens 3] — the HITL queue read error no longer separates "missing" from "unreadable" for the caller — backend/workflow_manager/endpoint_v2/destination.py:L1147-L1157, workers/shared/workflow/destination_connector.py:L1901-L1911. The server log keeps the distinction. Accepted.
  2. [Low] [Lens 16] — the test docstring says a cache error names "host, port and user"; Redis names host and port only — backend/dashboard_metrics/tests/test_health_check.py:L3-L4.

Lens checklist

  1. Spec & intent — Clean
  2. Architectural fit — Clean
  3. Correctness & edge cases — See findings #1
  4. Security — Clean (deep-dive done + boundary sweep, orchestrator)
  5. Data integrity & migrations — N/A — no schema/model change (orchestrator)
  6. Concurrency — N/A — no shared state (orchestrator)
  7. API & contract compatibility — Clean (no consumer parses the changed strings; the deleted classes have no references in this repo or unstract-cloud)
  8. Reliability — N/A — no external calls changed (orchestrator)
  9. Performance & cost — Clean
  10. Observability — Clean
  11. Operational safety — N/A — no infra (orchestrator)
  12. LLM/agent — N/A — none (orchestrator)
  13. Testing — Clean (exact-string assertions fail against the base)
  14. Dependencies & build — N/A — none (orchestrator)
  15. Code quality — Clean (pre-commit passed on these files in #2397, orchestrator)
  16. Doc & comment accuracy — See findings #2
  17. Cross-document consistency — Clean (the merge sweep found the backend and worker messages consistent, orchestrator)
  18. On-prem experience — N/A — no on-prem surface (orchestrator)
  19. Scaling & workload lifecycle — N/A — none (orchestrator)

Open questions for the author

  • The routed workflow_manager/internal_views.py (L201, L649, L1097) and other views in workflow_v2/views.py still return str(e) on the internal service-to-service API. That code predates this PR and is not in the requested alert set. It is a follow-up candidate.

Assumptions made — file_name is the caller's own upload name.

Security deep-dive: done

cost: agents=7 wall=14:00 lenses-examined=10 of 19 files=7 lines=544

ritwik-g added a commit that referenced this pull request Oct 28, 2026
The metrics-health, storage-path and dead-view changes now live in
#2400; this PR keeps only the x2text work.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ritwik-g and others added 2 commits October 28, 2026 21:18
SonarCloud S8572/S5778: the caller's message no longer carries the
error, so destination read failures (backend and workers) log with
logger.exception to keep the traceback. Tests now build the connector
before pytest.raises and assert the handled exception, which carries the
storage path, is what reaches the log.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@ritwik-g ritwik-g left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 2 — review-deep (FOLLOWUP), tip 24b494e, delta c43401e..24b494e

Review: unstract:review-deep. The delta addresses SonarCloud S8572 and S5778, and includes a pre-commit.ci format commit.
boundary-sweep: 3 sinks traced (three logger.exception sites). Clean. A traceback adds frames and the provider's cause chain, but no locals. The JSONDecodeError .doc (file content) is never printed.

Verdict — APPROVE WITH NITS

Prior context fetched — the round-1 findings are below the severity floor. Since boundary: c43401e.

Scope change — NO.

Summary — Critical: 0 · High: 0 · Medium: 1 · Low: 0 · Lenses run: 19/19

New findings

  1. [Medium] [Lens 13] — the queue-read tests do not pin that a traceback is logged: logger.exception(..., exc_info=False) survives
    • Location: backend/workflow_manager/endpoint_v2/tests/test_destination_error_detail.py:L46-L51, workers/tests/test_queue_read_error_detail.py:L44-L57
    • Failure mode: the path now reaches the log only through the traceback, and the mock ignores exc_info.
    • Suggested fix: record nothing as handled when exc_info is falsy.
    • Confidence: High. Verification: VERIFIED by a mutant run on the workers test.

Lens checklist

  1. Spec & intent — Clean
  2. Architectural fit — Clean
  3. Correctness & edge cases — Clean (WorkerLogger is a stdlib Logger, so .exception works)
  4. Security — Not covered — deep-dive not re-run for this logging-level delta (ran round 1); boundary sweep clean
  5. Data integrity & migrations — N/A — none (orchestrator)
  6. Concurrency — N/A — none (orchestrator)
  7. API & contract compatibility — N/A — no response change (orchestrator)
  8. Reliability — N/A — none (orchestrator)
  9. Performance & cost — Clean (orchestrator: extra traceback volume is negligible)
  10. Observability — Clean
  11. Operational safety — N/A — none (orchestrator)
  12. LLM/agent — N/A — none (orchestrator)
  13. Testing — See findings #1
  14. Dependencies & build — N/A — none (orchestrator)
  15. Code quality — Clean (pre-commit.ci autofix applied, orchestrator)
  16. Doc & comment accuracy — Clean (orchestrator)
  17. Cross-document consistency — Clean (backend and workers match, orchestrator)
  18. On-prem experience — N/A — none (orchestrator)
  19. Scaling & workload lifecycle — N/A — none (orchestrator)

Open questions — none.

Security deep-dive: unavailable — ran in round 1; this delta only changes logging levels and tests

cost: agents=3 wall=3:00 lenses-examined=7 of 19 files=4 lines=99

The recorder now ignores a call made with exc_info disabled, so a log
without the traceback (and so without the storage path) fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

@ritwik-g ritwik-g left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 3 — review-deep (FOLLOWUP), tip b3b52e5, delta 24b494e..b3b52e5

Review: unstract:review-deep. The delta is test-only. boundary-sweep: N/A, since the delta adds no sinks.

Verdict — APPROVE

Prior context fetched — round-2 finding #1, from review 5685638668. Since boundary: 24b494e.

Scope change — NO.

Summary — Critical: 0 · High: 0 · Medium: 0 · Low: 0 · Lenses run: 19/19

Prior findings status

  • #1 [Medium] the exc_info=False mutant survived → RESOLVED. The recorders at backend/.../test_destination_error_detail.py:46-54 and workers/tests/test_queue_read_error_detail.py:44-51 now kill it at all four sites. The old recorder let it survive.

New findings — none.

Lens checklist

  1. Spec & intent — Clean
  2. Architectural fit — N/A — test-only (orchestrator)
  3. Correctness & edge cases — N/A — test-only (orchestrator)
  4. Security — N/A — test-only (orchestrator)
  5. Data integrity & migrations — N/A — none (orchestrator)
  6. Concurrency — N/A — none (orchestrator)
  7. API & contract compatibility — N/A — none (orchestrator)
  8. Reliability — N/A — none (orchestrator)
  9. Performance & cost — N/A — none (orchestrator)
  10. Observability — N/A — none (orchestrator)
  11. Operational safety — N/A — none (orchestrator)
  12. LLM/agent — N/A — none (orchestrator)
  13. Testing — Clean
  14. Dependencies & build — N/A — none (orchestrator)
  15. Code quality — Clean (pre-commit, orchestrator)
  16. Doc & comment accuracy — Clean (the new comments match, orchestrator)
  17. Cross-document consistency — Clean (backend and workers recorders match, orchestrator)
  18. On-prem experience — N/A — none (orchestrator)
  19. Scaling & workload lifecycle — N/A — none (orchestrator)

Open questions — none.

Security deep-dive: unavailable — this delta is test-only; the deep-dive ran in round 1

cost: agents=1 wall=3:00 lenses-examined=4 of 19 files=2 lines=12

@ritwik-g
ritwik-g marked this pull request as ready for review October 29, 2026 14:23
@greptile-apps

greptile-apps Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

via Greptile

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no actionable regressions were found.

Summary

This PR keeps database, cache, and execution-storage details out of the changed caller-facing errors.

  • Health checks give callers simple database and cache errors.
  • Workflow file-read errors no longer show storage paths to callers.
  • The unused duplicate internal views are removed from workflow views.

No actionable issues were found. No tests were run during this review.

Reviews (1) · Last reviewed commit: "test: pin that the read-failure log carr..." · Reviewed by Greptile

@github-actions

Copy link
Copy Markdown
Contributor

Unstract test results

Per-group results

Status Group Tier Passed Failed Errors Skipped Duration (s)
⚪ e2e-agent-kv e2e 0 0 0 1 0.1
✅ e2e-api-deployment e2e 3 0 0 0 16.6
✅ e2e-coowners e2e 1 0 0 0 1.8
✅ e2e-etl e2e 1 0 0 0 14.9
✅ e2e-login e2e 2 0 0 0 1.3
✅ e2e-prompt-studio e2e 1 0 0 0 11.5
✅ e2e-smoke e2e 2 0 0 0 1.1
✅ e2e-workflow e2e 1 0 0 0 16.6
✅ frontend unit 629 0 0 0 12.5
✅ integration-backend integration 1448 0 0 26 88.6
✅ integration-connectors integration 1 0 0 7 10.1
✅ integration-workers integration 181 0 0 1 57.4
❌ ui e2e 0 1 0 0 0.0
✅ unit-agent-kv-schema unit 143 0 0 0 1.1
✅ unit-backend unit 2063 0 0 1 41.6
✅ unit-connectors unit 82 0 0 0 7.9
✅ unit-core unit 281 0 0 0 2.1
✅ unit-filesystem unit 1 0 0 0 2.8
✅ unit-platform-service unit 18 0 0 0 2.0
✅ unit-rig unit 120 0 0 0 3.2
✅ unit-runner unit 12 0 0 0 2.2
✅ unit-sdk1 unit 878 0 0 0 30.7
✅ unit-tool-sidecar unit 8 0 0 0 1.3
✅ unit-workers unit 1536 0 0 2 108.9
TOTAL 7412 1 0 38 436.4

Critical paths

⚠️ Critical paths not yet covered

  • workflow-execution-fan-out — Multi-file workflow execution fans out to file-processing workers and rejoins. (declared coverage: no groups declared)
✅ Covered critical paths
  • auth-login — covered by e2e-login
  • adapter-register-llm — covered by integration-backend
  • workflow-author — covered by integration-backend
  • co-owner-manage — covered by integration-backend, e2e-coowners
  • workflow-create-execute — covered by e2e-workflow
  • api-deployment-provision — covered by integration-backend
  • api-deployment-auth — covered by integration-backend
  • api-deployment-run — covered by e2e-api-deployment
  • mcp-server-auth — covered by integration-backend
  • mcp-platform-auth — covered by integration-backend
  • platform-key-whoami — covered by integration-backend
  • prompt-studio-author — covered by integration-backend
  • prompt-studio-fetch-response — covered by e2e-prompt-studio
  • connector-register-test — covered by integration-backend
  • pipeline-etl-execute — covered by e2e-etl
  • usage-aggregate-read — covered by integration-backend
  • usage-token-tracking — covered by e2e-api-deployment
  • callback-result-delivery — covered by e2e-api-deployment

@chandrasekharan-zipstack chandrasekharan-zipstack left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, however consider removing code comments similar to what I've suggested since they go stale or less relevant with time and would bloat agent context in the future

Comment on lines +1165 to +1166
# The exception text names the DB host, port and user, and this
# endpoint answers any org member, so it stays in the server log.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# The exception text names the DB host, port and user, and this
# endpoint answers any org member, so it stays in the server log.

return {"status": "error", "message": "Cache read/write mismatch"}
except Exception as e:
except Exception:
# Same as the database check: the Redis error names host and port.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Same as the database check: the Redis error names host and port.

Comment on lines +639 to +641
# The error carries the execution-storage path, which stays in the
# server log (with the traceback); the caller gets a message
# without it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# The error carries the execution-storage path, which stays in the
# server log (with the traceback); the caller gets a message
# without it.

Comment on lines +1151 to +1152
# The error names the execution-storage path: log it with the
# traceback, but keep it out of the message returned to the caller.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# The error names the execution-storage path: log it with the
# traceback, but keep it out of the message returned to the caller.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants