LTX-2 v1.6.0 Fixes Attention Gate Issue and Adds New Features

What happened

The LTX-2 v1.6.0 update addressed a problem where the STG incorrectly applied per-head attention gates to the value projection in video and audio processing. This fix improves the accuracy of the model's handling of these media types.

Additional features include the ICLoraPipeline, which allows optional audio input and enables refinement during processing. The encode_media_segments function now supports incremental encoding of video and audio streams into SDR H.264 or HDR HLG formats.

Why it matters

These updates enhance the reliability and flexibility of the LTX-2 system, particularly for users working with audio and video streams. The changes also improve the efficiency of model training and media encoding, supporting a broader range of applications.

Key details

The update includes improvements to the DiffusionStage, which now uses an active_weights dictionary for shared transformer allocation. Several command-line interfaces, such as Dub-It and IC-LoRA, now encode stream_chunks directly without separating temporal chunks.

The code in the repository is now licensed under Apache 2.0, while the LTX models remain under their respective licenses. The encode_video function now writes ProRes 4444 video from RGBA data, with specific tagging and pixel markers.