Pewarta Proxy

📋 Server Request & Proxy Logs

Loading active proxy settings...
Developed by Khairul Adha (adhexa.id)
Mengambil log server...

⚙️ Pengaturan Proxy Target

Skip to content

Preserve batches when a distributed worker gets only one - #3097

Open
antonsoo wants to merge 1 commit into
speechbrain:developfrom
antonsoo:fix/distributed-sampler-single-batch
Open

antonsoo wants to merge 1 commit into
speechbrain:developfrom
antonsoo:fix/distributed-sampler-single-batch

Conversation

@antonsoo

@antonsoo antonsoo commented Oct 9, 2026

Copy link
Copy Markdown

What does this PR do?

Keep each sampled item intact when DistributedSamplerWrapper assigns zero or one item to a worker. For example:

from speechbrain.dataio.sampler import DistributedSamplerWrapper

sampler = DistributedSamplerWrapper(
    [[0, 1], [2, 3]], num_replicas=2, rank=0, shuffle=False
)
print(list(sampler))  # before: [0, 1]; after: [[0, 1]]

itemgetter returns the item itself for one index, so a single batch becomes individual indices and the data loader raises TypeError: 'int' object is not iterable. A scalar sampler also raises, and an empty assignment calls itemgetter without arguments. Yielding one item per assigned index handles all three cases and preserves the existing multi-item behavior.

Added a runnable doctest and eight regression cases for scalar/batch samplers, padding, empty inputs and drop_last. Six cases fail before the change. Sampler, dataloader and batching tests plus sampler doctests: 24 passed, one CUDA-only test skipped. pre-commit run -a passes.

Also exercised Brain.fit with a DynamicBatchSampler in two actual CPU/Gloo processes, using the shipped CTC example's data pipeline and eight speech recordings (18.96 seconds). With one four-utterance batch per worker, the baseline fails before the first update. The fix completes four updates per worker; losses, all 10 model-state tensors and all 27 optimizer tensors exactly match a control using explicitly assigned batches. This uses a recurrent-only GRU encoder to run DDP on CPU; it does not claim GPU or convergence validation.

No new dependencies or API changes.

Before submitting
  • Read the contributor guideline.
  • This PR does one thing.
  • Updated documentation with a runnable example.
  • Added necessary regression tests.
  • New and existing affected tests pass locally.
  • Listed breaking changes: none.
  • Followed the project code style and conventions.

PR review

Reviewer checklist
  • Is this pull request ready for review?
  • Check that all items from Before submitting are resolved.
  • Make sure the title and description explain the PR.
  • Add labels and milestones as appropriate.
  • Confirm compatibility requirements.
  • Review the self-review checklist.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant