Repository navigation
Conversation
…lled on The context parallelism guide only calls `pipeline.transformer.enable_parallelism(...)`. A pipeline with more than one transformer, such as Wan 2.2 A14B with `transformer` and `transformer_2`, needs it on each one, or the steps the other transformers run aren't split across GPUs. Add a short note after the Ring Attention example. Signed-off-by: menglcai <menglcai@amd.com>
menglcai
marked this pull request as ready for review
November 18, 2026 07:54
stevhliu
approved these changes
Nov 18, 2026
stevhliu
left a comment
Member
There was a problem hiding this comment.
thanks! maybe move this up to line 17 to put it in front of everyone, and not just readers who are in the Ring Attention section since this applies to every variant on the page
| To try the other variants, replace the `enable_parallelism` call in `ring_attention.py` with the one from their section. | ||
|
|
||
| > [!NOTE] | ||
| > [`~ModelMixin.enable_parallelism`] only applies to the model you call it on. If a pipeline has more than one transformer, such as `transformer` and `transformer_2`, call it on each one. Otherwise, the steps the other transformers run aren't split across GPUs. |
Member
There was a problem hiding this comment.
Suggested change
| > [`~ModelMixin.enable_parallelism`] only applies to the model you call it on. If a pipeline has more than one transformer, such as `transformer` and `transformer_2`, call it on each one. Otherwise, the steps the other transformers run aren't split across GPUs. | |
| > [`~ModelMixin.enable_parallelism`] only applies to the model you call it on. If a pipeline has more than one transformer, such as `transformer` and `transformer_2`, call it on each one. Otherwise, the denoising steps that use the other transformers aren't split across GPUs. |
Contributor
Author
There was a problem hiding this comment.
Thank you for the review and the suggestions! I've moved the note up to right after the attention backend paragraph so readers see it before any of the variants, and applied your wording. Please let me know if anything else needs adjusting. Thanks again!
Signed-off-by: menglcai <menglcai@amd.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes #15317
Adds a short note to the context parallelism guide:
enable_parallelismonly applies to the model you call it on, so a pipeline with more than one transformer (such astransformerandtransformer_2) needs it on each one. Otherwise, the steps the other transformers run aren't split across GPUs.The note goes right after the Ring Attention example, since every other variant section points back to it. Following @stevhliu's suggestion in the issue, it is general rather than specific to Wan. The Wan 2.2 A14B measurements that motivated it are in the issue.
Self-review notes
Verdict: READY, no blocking issues. Docs-only change, so no dead code and no tests apply. I didn't run
make qualitylocally; CI runs the doc style check.Left for the reviewer:
> [!NOTE]form as the other notes indocs/source/en/optimization/, and the same doc-builder link toModelMixin.enable_parallelismas the rest of the page.transformer. The note's rule applies there too, so I didn't add a second note.Before submitting
self-reviewskill on the diff?documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
@stevhliu @sayakpaul