Abstract
Video diffusion transformers rely on 3D full attention over tens of thousands of spatiotemporal tokens, and its quadratic cost dominates inference latency. Sparse attention reduces this cost, but existing methods face a dilemma: static methods apply predefined masks and cannot capture input-dependent attention, whereas dynamic methods reconstruct the sparse mask repeatedly, paying online to rediscover positional structure that is largely stable across inputs.
We propose PoCoAttn, a training-free sparse attention framework that separates input-stable from input-dependent attention and resolves the former offline and the latter online. Across four video diffusion models and three tasks, PoCoAttn consistently achieves a better speed–fidelity Pareto frontier than state-of-the-art (SOTA) methods, improving PSNR by 1.9 to 5.0 dB over baselines and delivering up to 4.1× attention and 2.0× denoising speedups over dense FlashAttention-3 attention.
Method
The Proposed PoCoAttn Framework
Same prompt, same seed
Dense vs. PoCoAttn
Each clip plays the dense-attention output on top and the PoCoAttn output below it.
LTX-2.3-22B
1080p text-to-video
2.02×DiT speedup15% attention density
INT. CRYOGENIC BAY OF A DEEP-SPACE FREIGHTER - EMERGENCY LIGHTINGShow full promptCollapse prompt
INT. CRYOGENIC BAY OF A DEEP-SPACE FREIGHTER - EMERGENCY LIGHTING. A tight 85mm close-up on a cryopod window laced with frost. Behind the glass a woman with short dark hair lies with eyes closed, skin pale and dusted with ice crystals. The pod hisses, the frost melts into running droplets, and cool blue standby lights give way to warm amber as power returns. Her eyes open slowly and focus on the camera; she takes a sharp first breath that fogs the glass. Rows of identical dark pods blur into the background, one lamp blinking far away. Realistic, clean subject-background separation, strong cold-to-warm light transition, visible face, no helmet, no text, no cartoon styling. Audio: hydraulic hiss, dripping water, a low ship hum, one deep inhale.
EXT. SMALL FISHING HARBOR - DAWN, THIN FOGShow full promptCollapse prompt
EXT. SMALL FISHING HARBOR - DAWN, THIN FOG. A bright 85mm close-up of an old fisherman with a weathered face, white stubble and a faded wool cap, mending a net on the deck of a wooden boat. Soft golden sunrise light breaks through the fog and travels slowly across his hands, then his face, as the boat rocks gently. He pauses, looks up toward the open sea, and squints with a faint smile. Masts, ropes and gulls blur into creamy bokeh behind him. The camera makes a slow eye-level push-in. Natural documentary realism, warm highlights, visible skin texture, gentle water reflections, no text, no logos. Audio: gulls, water lapping against the hull, creaking rope.
INT. OLD BALLET REHEARSAL HALL - LATE AFTERNOONShow full promptCollapse prompt
INT. OLD BALLET REHEARSAL HALL - LATE AFTERNOON. Tall windows pour low sunlight across a worn wooden floor, dust floating in the beams. A young dancer in a grey leotard and a loose knit cardigan rises onto pointe and turns slowly, her reflection following in a long mirror. The camera begins in a wide shot and pushes in to an 85mm portrait as she finishes the turn facing the window, light crossing her face, breath visible for a moment in the cool air. She steadies herself with a hand on the barre and looks down with a quiet exhale. Natural warm backlight, soft lens haze, visible fabric texture, clean subject-background separation, no text, no logos, no audience. Audio: creaking floorboards, soft breathing, a distant piano.
EXT. DERELICT ORBITAL STATION - EARTH TERMINATOR LINEShow full promptCollapse prompt
EXT. DERELICT ORBITAL STATION - EARTH TERMINATOR LINE. A slow cinematic drift along the scarred hull of an abandoned space station, half of it in hard sunlight, half in shadow. An astronaut in a worn, patched EVA suit pulls herself hand over hand along a railing, Earth's blue curve and city lights reflected in her gold visor. Loose cables and frozen droplets tumble past the lens in zero gravity. As she reaches a sealed hatch, a strip of red emergency lights flickers on inside the window and brightens her face through the visor. The camera makes a slow push-in to a tight 85mm portrait of her eyes. Photoreal, tactile suit fabric, hard rim light from the sun, deep black space, no text, no logos, no lens flare overload. Audio: suit breathing, muffled metal contact, faint radio static.
INT. STARSHIP BRIDGE - APPROACH TO A NEBULAShow full promptCollapse prompt
INT. STARSHIP BRIDGE - APPROACH TO A NEBULA. An 85mm portrait of a middle-aged captain with a grey beard and a dark high-collared uniform, seated in a command chair. His face is lit from below by a floating holographic star chart, cyan lines tracing orbits across his cheek and eyes. He leans forward, the hologram rotates, and the main viewport behind him fills slowly with the pink and teal folds of a vast nebula, its glow spreading across the bridge consoles and warming his face. He narrows his eyes and gives a small decisive nod. The camera holds steady with a slow push-in. Photoreal skin, soft volumetric light, blurred crew silhouettes in the background, no text, no subtitles, no lens flare clusters. Audio: low engine drone, soft console chimes, a quiet exhale.
EXT. MOUNTAIN SWITCHBACK ROAD - RAINY EVENINGShow full promptCollapse prompt
EXT. MOUNTAIN SWITCHBACK ROAD - RAINY EVENING. A low tracking shot follows a dark green vintage 1960s sports car climbing a wet asphalt road through pine forest, headlights carving through drizzle and thin mist. Rain beads on the polished hood and streams off the chrome; the tires throw up fine spray. The camera moves alongside, then swings to a three-quarter front view as the car sweeps through a bend and its headlights wash over the lens. Low grey clouds hang between the ridges, and a distant valley town glows faintly below. Realistic, wet reflections, cinematic motion, muted cool palette with warm headlights, no text, no logos, no cartoon styling. Audio: tires on wet road, rain on metal, a steady engine note.
Wan2.1-T2V-14B
720p text-to-video
1.59×DiT speedup15% attention density
Close-up shot captures the steam gently rising from a coffee cup.Show full promptCollapse prompt
A close-up shot captures the gentle rise of steam from a freshly brewed coffee cup. The camera slowly pans upwards, starting from the swirling steam, revealing the rich brown liquid inside the ceramic cup. The warm glow from the coffee creates a soft, inviting ambiance. The background is subtly blurred, focusing attention on the intricate dance of the steam as it dissipates into the air. Warm lighting enhances the cozy atmosphere of the scene.
On a frigid winter night, a homeless person is huddled in a street corner, trembling from the cold. His clothing is worn and insufficient. The camera is positioned at the knee level of a passerby, shooting horizontally.Show full promptCollapse prompt
In a frigid winter night scene, a homeless person, dressed in tattered and thin clothing, is huddled and shivering on a street corner. The person’s face shows signs of fatigue and discomfort. The camera is positioned at knee level from the perspective of a passing pedestrian, capturing the scene horizontally. The background includes dimly lit street lamps and the silhouette of other buildings, emphasizing the isolation and harshness of the environment. The focus remains on the homeless individual as they struggle to stay warm. Knee-level horizontal shot.
A man in a suit slowly ties his tie,present in the style of the movie "The Godfather".Show full promptCollapse prompt
In the classic style of "The Godfather," a well-dressed man in a tailored dark suit and slicked-back hair slowly and meticulously ties his tie in front of a luxurious wooden desk. The room is dimly lit with warm amber lighting casting shadows across the walls. He stands with confidence, his posture straight and authoritative. His face is serious, reflecting the power and complexity of his character. The background shows elegant furnishings and rich tapestries, adding to the opulent atmosphere. Medium shot focusing on the man from the chest up, capturing the subtle motions of tying the tie and the intensity in his gaze.
Wan2.1-I2V-14B
720p image-to-video
1.60×DiT speedup15% attention density
an aerial view of a busy city with a bridge in the backgroundShow full promptCollapse prompt
An aerial view captures a bustling cityscape at dusk, with a prominent suspension bridge stretching across a river in the background. The city skyline is filled with tall buildings, their lights beginning to twinkle as the day transitions into night. Below, a busy highway is filled with cars, their headlights creating streaks of light as they move along the road. The water reflects the warm hues of the sunset, adding a serene contrast to the urban activity below. The sky is painted with soft pinks and oranges, signaling the end of the day. A wide-angle shot from a high vantage point.
a bird with a fish in its beak flying over a fieldShow full promptCollapse prompt
A majestic heron gracefully soars over a lush green field, its wings spread wide as it carries a small fish in its beak. The bird's sleek feathers are a mix of gray and white, with black accents on its wings and tail. Its long legs trail behind, and its sharp talons are visible. The field below is dotted with vibrant green plants and yellow flowers, creating a serene backdrop. The sky above is a soft blue, adding to the tranquil atmosphere. A dynamic side view captures the bird mid-flight, emphasizing its powerful and elegant motion.
two men in white clothing standing next to each otherShow full promptCollapse prompt
A black-and-white photograph captures two men in traditional white clothing standing side by side against a textured wall. The man on the left, wearing a white robe and a turban, appears to be engaged in conversation with the man on the right, who is dressed in a long white robe with a hood. Both men have serious expressions, suggesting a moment of deep discussion. The lighting creates dramatic shadows, adding depth to the scene. The background is minimalistic, emphasizing the interaction between the two figures. A medium shot with a slight angle from the front.
HunyuanVideo-13B
720p text-to-video
1.79×DiT speedup15% attention density
In the center of the room, there is a bed, above which hangs a ceiling fan that is rotating.Show full promptCollapse prompt
A cozy bedroom with a single bed positioned at the center of the room. Above the bed, a ceiling fan rotates slowly, its blades catching the light and casting gentle shadows on the walls. The bed is neatly made with a comforter and pillows arranged symmetrically. Soft ambient lighting from a nearby lamp creates a warm and inviting atmosphere. The camera captures a medium shot, focusing on the bed and the rotating ceiling fan.
In the rain, there is a telephone booth, with rainwater sliding down the edges, forming curtains of water. As the camera zooms out, it reveals the telephone booth standing alone in a street soaked by the rain.Show full promptCollapse prompt
In a rain-soaked city street, a vintage telephone booth stands alone. Rainwater slides down the glass sides, creating curtains of water that shimmer in the dim light. As the camera slowly zooms out from a close-up of the water droplets, the booth's weathered exterior comes into focus. The camera continues to pull back, revealing the wet cobblestone street and puddles reflecting the overcast sky. The scene is enveloped in a soft, melancholic ambiance, with the rain continuing to fall steadily. Medium to wide shot, starting from a close-up of the water droplets and gradually pulling back to reveal the entire street.
Inside the cabin, passengers lean back in their seats, resting with closed eyes. As the camera pushes closer to the windows, it switches to the departure hall where crowds are surging and passengers are rushing about.Show full promptCollapse prompt
In the cozy interior of an airplane cabin, passengers are leaning back in their seats, their eyes gently closed as they rest. Soft ambient lighting bathes the scene, highlighting the serene expressions on their faces. As the camera gradually zooms in towards the window, it captures the reflection of the passengers before smoothly transitioning to the bustling departure hall outside. Here, the scene is filled with energetic movement; crowds surge forward, passengers rush about, and luggage carts weave through the throng. The camera maintains a dynamic perspective, capturing the lively energy and chaos of the busy airport environment. Wide shots transition to close-ups, emphasizing the hustle and bustle of travelers.
HyperFlow-H3-Flash-14B
768p text-to-video with audio (T2VA, 5 s) · drag the line: Dense on the left, PoCoAttn on the right
3.08×Attention speedup15% attention density
A black dragon and its rider sweep over a frozen castle wall and set it ablaze.Show full promptCollapse prompt
integrated_multimodal_description: [Shot 1] Dark medieval fantasy epic, gritty photorealism, cold desaturated palette, prestige fantasy TV drama look. Wide aerial tracking shot at dusk over a snow-covered northern castle with massive stone walls. A rider in dark scale armor and a heavy fur cloak sits on the back of an enormous black dragon with leathery wings and red-glowing scales along its throat, sweeping low over the battlements as soldiers below duck behind their shields. The dragon opens its jaws and unleashes a roaring torrent of orange fire across the frozen wall; steam and embers burst into the snowy air and the rider's cloak whips in the wind. overall_soundscape: Thunderous wingbeats, a deafening dragon roar, the whoosh and crackle of fire, soldiers shouting, howling wind. non_diegetic_music: Brooding orchestral theme with deep cellos and pounding war drums.
A heroine lands from the sky and sends a shockwave down a ruined street.Show full promptCollapse prompt
integrated_multimodal_description: [Shot 1] Blockbuster superhero film look, photorealistic, slow motion, low-angle shot. In a ruined city street at dusk thick with dust and burning cars, a heroine in a matte black tactical suit traced with glowing electric-blue lines, short silver hair, drops from the sky in a superhero landing. Her fist strikes the ground and a shockwave ring ripples outward, shattering windows, lifting debris and bouncing nearby cars. She slowly rises and looks up into the lens as the camera pushes in, lightning crackling along her forearms. overall_soundscape: A whoosh of descent, a massive ground-impact boom, glass shattering, crackling electricity, wailing car alarms. non_diegetic_music: Heroic orchestral superhero theme, rising brass and pounding percussion.
A drone circles the bell tower as its bells ring over the autumn quad.Show full promptCollapse prompt
integrated_multimodal_description: [Shot 1] Cinematic aerial drone shot at golden hour in autumn, slowly orbiting and descending. The Purdue University campus in West Lafayette, Indiana: the tall red-brick Purdue Bell Tower with white stone trim rises at the center of a tree-lined quad, maple trees blazing orange and red around it, students in black-and-gold sweatshirts walking along the paths. The camera glides past the tower as its bells begin to ring and reveals a modern fountain in the plaza below, its jets of water catching the warm sunlight. Warm, nostalgic, highly detailed. overall_soundscape: Bell tower chimes ringing out across campus, rustling autumn leaves, distant student chatter, a group of students cheering "Boiler Up!". non_diegetic_music: Warm, uplifting orchestral strings with a hint of marching-band brass.
A white mobile suit boosts past Earth and fires a beam rifle at an enemy mech.Show full promptCollapse prompt
integrated_multimodal_description: [Shot 1] Cinematic Japanese mecha anime with detailed 3D rendering, wide tracking shot in Earth orbit. A sleek white humanoid mobile suit with blue chest armor and angular shoulder plates boosts across the frame, backpack thrusters blazing blue, Earth's curved horizon glowing below. It spins and fires a beam rifle; a brilliant pink-white beam streaks across space and strikes a dark enemy mech in the distance, which bursts into an expanding orange fireball and scattering debris. The hero mech's twin eyes flash green as the sun flares behind it. overall_soundscape: A roaring thruster burn, the rising whine of the rifle charging, a sharp beam blast, a deep distant explosion. non_diegetic_music: Heroic orchestral anime theme with a bright brass fanfare.
An adventurer on a cliff above a sea of clouds summons a swirl of wind energy.Show full promptCollapse prompt
integrated_multimodal_description: [Shot 1] High-quality Japanese anime style, clean cel shading with soft painterly backgrounds, vibrant saturated colors. Medium wide shot, camera slowly orbiting. A young adventurer girl with long silver-blue hair tied with a small golden ornament, wearing a flowing white-and-azure fantasy outfit with gold trim and a short cape, stands on a cliff edge above a sea of clouds. Floating stone islands with waterfalls and glowing cyan crystals drift in the background beside an enormous ancient tree. A gust of wind lifts her hair and cape; she raises one hand and a swirl of glowing teal wind energy and spinning leaves spirals around her arm, scattering sparkling particles. Bright midday sun, soft lens flare, dreamy atmosphere. overall_soundscape: A soft rushing wind, crystalline chimes as the magic swirls, distant birdsong. non_diegetic_music: Uplifting orchestral fantasy theme with flute, piano and light strings.
A young sorcerer ignites violet energy and charges a many-eyed shadow creature.Show full promptCollapse prompt
integrated_multimodal_description: [Shot 1] Modern Japanese action anime style, sharp ink linework, high-contrast shading, dynamic fluid animation with speed lines and impact frames. Night, an empty city crossroad, street lights flickering. A young male sorcerer in a dark navy high-collared academy uniform, messy black hair and a small bandage on his cheek, faces a giant grotesque shadow creature with many glowing eyes rising out of cracked asphalt in clouds of purple-black smoke. He clenches his fist and violet-blue energy flames ignite around his forearm, crackling with sparks; the camera snaps in fast as he shouts in Japanese "行くぞ!" and dashes forward, the road splitting beneath his feet. overall_soundscape: A low demonic growl, crackling energy, sneakers scraping on concrete, asphalt cracking, a sharp booming impact. non_diegetic_music: Aggressive J-rock with distorted guitars and fast drums.
HyperFlow H3 Flash (14B, INT8, derived from MiniMax-H3), text to video and stereo audio, 1344×768, 124 frames at 24 fps (5.2 s), 6 steps, same prompt and seed for both. The clips use PoCoAttn at 25% attention density on steps 2–5 (steps 0–1 and block 0 dense). Attention speedup: 15% density over FlashAttention-3 per attention call on one H100.
Bernini
720p video editing
1.72×DiT speedup15% attention density
add objectAdd a realistic white button-up shirt on the man in the video. The shirt should fit naturally to his body shape and posture, with visible fabric folds, seams, and slight movement that reacts to his motion. Use a soft, breathable cotton texture with subtle shading and highlights to reflect the scene's lighting.
change backgroundTransport the scene by changing the background to a serene beach at sunrise, complete with gentle ocean waves, golden sand, and palm trees swaying lightly in the breeze, infusing the video with a warm, tranquil glow that interacts softly with the subject.
add effectAdding swirling golden light trails and magical glowing fireflies drifting around the cat.
DiT speedup is measured over the FlashAttention-3 dense baseline at 15% attention density.
Citation
BibTeX
@article{wu2026pocoattn,
title = {PoCoAttn: Position--Content Collaborative Sparse Attention
for Fast and High-Fidelity Video Generation},
author = {Wu, Jianwen and Wang, Meiling and Zhou, Junkang and Shen, Yipeng and You, Haoran},
journal = {arXiv preprint arXiv:2611.20307},
year = {2026}
}