Foundry has launched SmartRoto, a machine-learning plugin for Nuke, NukeX, Nuke Studio and Nuke Indie that takes on the slowest part of rotoscoping, fxguide reported on 16 July after talking to Adam Cherbetji, Foundry's director of product for AI research. It costs $499 a year, with node-locked and floating licences.
Foundry is explicit that it does not draw the shapes. The artist still breaks a subject into articulated splines on a clean frame, which Cherbetji regards as skilled judgement worth keeping; SmartRoto then handles the tedious keyframing, moving and deforming those splines through the shot. In his telling, keyframing is where most of the hours go, so that is what the team targeted.
It works with two kinds of keys. Frames the artist sets or corrects become user keys, which the model treats as fixed and never overwrites; the frames it predicts are smart keys. The artist keys a second significant frame, triggers the model to snap the shapes into place, fixes anything off, and reruns it across the whole shot or a section, with each correction feeding the next pass. Locks apply per shape, so some shapes on a frame can be fixed while others keep being refined, and a bad result only needs adjusting, never redoing. During development the user keys were nicknamed gold keys, which says a lot about how the tool ranks the artist's decisions over its own.
The output is dense with keys, but only user keys need attention, and a thinning tool looks for stretches that can be interpolated instead; Cherbetji says that part is still being improved. The approach mirrors how experienced roto artists already work: set the extremes, refine between them, and lock what is right.
Technically, Foundry says it rejected optical flow, its first attempt, because it fails at edges where precision matters most, and it rejected the common approach of tracing a spline around a segmentation mask. Instead the model follows image features and moves the splines themselves, keeping sub-pixel accuracy, and uses the spatial relationships between shapes as extra information. That is why one big outline around a subject performs poorly, as it always has for human artists. For fxguide, the point is that splines remain the deliverable compositors can edit, and a tool that only outputs mattes is not solving the job they actually do.
The model was trained with motion blur and tends to place edges mid-blur, though heavily blurred frames need more manual keys; blur can still be rendered into the mattes. Partial occlusion is handled, full occlusion still needs shape lifetimes set by hand, and two near-identical figures can occasionally be confused. It needs around 8GB of VRAM for typical work, more at high resolutions with many shapes, which is far below what the big foundation segmentation models demand, and it runs locally, with nothing sent to the cloud.
SmartRoto took years. An earlier attempt with DNEG and the University of Bath six or seven years ago did not crack the problem; Foundry restarted internally about four years ago, building its own model and datasets. Training footage comes from the research director's own filmmaking, rotoscoped to a high standard in-house and by contractors, and Foundry says the model uses only licensed data and is commercially safe.
In a two-week test, professional roto artists worked shots in plain Nuke and with SmartRoto in random order and were up to four times faster by the end. Cherbetji calls that the upper end and suggests planning for about twice the speed. Several testers said they did not want to go back to working without it. Beta tester Benjamin Bratt, a compositor who wrote a textbook on rotoscoping, said he was surprised how good it is and praised how easily generated keys can be fixed without starting over.
fxguide places the launch in a wider race. Roto looks like an obvious job for AI, repetitive and backed by decades of studio examples, yet tools rarely manage precise edges, stable motion across frames and editable output all at once. Sam Hodge at Kognat has shown promising work, and OTOY's RotoShop paper for SIGGRAPH 2026 goes the other way from Foundry: it takes masks from a segmentation model and fits rigged Bezier splines to them, mainly to build datasets, storing 100,000 frames of splines in about 30MB against 15GB of raster masks.
Sources · 1