Repository navigation
Add fixless audit support - #109
Open
GabeBlanchard wants to merge 3 commits into
Open
GabeBlanchard wants to merge 3 commits into
GabeBlanchard wants to merge 3 commits into
Conversation
Adds ScaleClient.create_fixless_audit (PUT /v1/audits), get_audit and get_task_audits, plus Task.create_fixless_audit / Task.audits. Feedback items are built with FeedbackItem and Scope helpers instead of hand-written JSON; validation stays on the API. Also treats 201 Created as success in the HTTP layer: PUT /v1/audits responds 201, which previously raised even though the audit was created. Bumps version to 2.19.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Covers get_audit, get_task_audits (including an empty list), returned Audit objects, and Task.create_fixless_audit / Task.audits forwarding. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The API no longer serves POST /v1/task/documenttranscription (the task type was removed server-side in July 2026), so these tests 404 on every CI run. TaskType.DocumentTranscription is left in place; removing it would be a breaking change for a separate PR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request Summary
Description
Adds first-class support for fixless audits: structured feedback on specific parts of a completed task's response, without modifying the response. Until now customers had to build the nested JSON for
PUT /v1/auditsby hand.New public API:
ScaleClient.create_fixless_audit(task_id, target_response_url, accepted, feedback_items=None, comments=None, metadata=None)→PUT /v1/auditsScaleClient.get_audit(audit_id)→GET /v1/audit/:idScaleClient.get_task_audits(task_id)→GET /v1/audits/task/:taskId(newest first)Task.create_fixless_audit(target_response_url, accepted, **kwargs)andTask.audits()scaleapi.audits:FeedbackItem(.error/.comment/.confirmation/.flag),Scope(.annotation/.scene/.point/.polygon/.lss), and theErrorCategory,Severity,FeedbackItemTypeandAuditResultenums.framestakes a frame index or an inclusive(start, end)range.Design notes:
ScaleInvalidRequest(400).target_response_urlis required. Customers already have it from the completed tasks they pulled down (task.response["annotations"]["url"]).stage_instance_idparameter.FeedbackItem.errordefaults toSeverity.Standard, the baseline grading weight. A code comment explains this.Api._api_request: it only treated200as success, andPUT /v1/auditsresponds201 Created. A successful audit would have raisedScaleException(code=201)even though it was created. It now accepts200or201; other status codes behave as before.PUTis not in the retry list, so no duplicate audits can come from retries.Assumptions and dependencies:
409both when fixless audits aren't enabled and whentarget_response_urlisn't the task's latest response. The client maps409toScaleDuplicateResource, which is misleading but unchanged here. Both cases are documented in the docstring.422. This SDK does not depend on it.How did you test your code?
tests/for the newly implemented methods:tests/test_audits.py, 6 tests. They run offline with mocks (payload and scope serialization, thePUTrequest body, and 201 handling), so they don't needSCALE_TEST_API_KEY. The live test project can't run fixless audits, so there is no live end-to-end test.tests/to cover changes made to existing methodsRan
pytest tests/test_audits.pyon Python 3.8 and the repo's pre-commit hooks (isort, black, flake8, pylint) on all changed files.README.rstrenders cleanly withreadme_renderer, the same checktwine check --strictuses.Checklist
_version.py(2.18.4 → 2.19.0, a new feature)🤖 Generated with Claude Code


Confidence Score: 5/5
The PR appears safe to merge; no new actionable issues were found.Summary
Adds helpers to create and fetch fixless audits without changing task responses.
Diagram
%%{init: {'theme': 'neutral'}}%% flowchart LR Task["Task.create_fixless_audit"] --> Client["ScaleClient.create_fixless_audit"] Feedback["FeedbackItem and Scope"] --> Payload["Build audit payload"] Client --> Payload Payload --> API["PUT /v1/audits"] API --> Created["201 response"] Created --> Audit["Audit object"]Reviews (3) · Last reviewed commit: "Remove DocumentTranscription live tests" · Reviewed by Greptile