Every time a bolt of lightning cracks across Larissa Supnik’s TV screen, she likes to pause the video.
While her show-stopping habit can sometimes annoy her partner, who prefers his TV shows uninterrupted like the rest of us, Supnik can’t help but get a closer look at the fine details. That’s because Supnik is a lead software engineer on Boris FX’s Sapphire product, and her team developed the special effects plugin that makes the lightning happen.
For the past 20 years, Sapphire has been developing a suite of plugins that help video editors manipulate pixels in post-production through film editing software like Adobe After Effects and Avid Media Composer. Sapphire’s product contains more than 250 of these plugins, which can add lens flares, alter pixel color, or introduce screen blurring. And if you’ve ever wondered what allowed Yoda to summon lightning or gave Frodo’s sword that magic glow, the answer is Sapphire.
But to succeed throughout two decades in the special effects industry, which has seen evolutions like higher pixel rates, faster production times and more special effects per video, Sapphire’s engineers have had to adapt and evolve the platform.
Supnik, who has been on the team for 12 years, has watched the transformation firsthand. It started in 2008, when the team initiated the process of rewriting its platform in C++.
Mastering the art of language migration
- When selecting a new language, consider what new features it adds. The Sapphire team rewrote portions of its software from C to C++ because of its ability to process both fixed point and floating point math operations side-by-side, its compatibility with components written in C and the fact that most film editing software supports it.
- Taking a piecemeal approach makes the process more manageable and gives engineers time to master the new language. Sapphire rewrote functions as needed to reduce errors and give team members time to become familiar with C++.
- Consider carefully which parts need to be replaced with the new language, and which you can keep. While writing a new system from scratch might be cleaner, you can risk losing some of the things your old codebase did well.
- Writing tests as you develop new code can reduce bugs and ensure a smooth port. The Sapphire team wrote its own tests for C++ to ensure the plugins worked appropriately, even for edge use cases.
- Keeping a style guide can help unify the code base for future changes. Writing code using a style guide helps Supnik eliminate instances of siloed code, making it easier to update Sapphire’s 20-year-old codebase.
To make the change easier, consider a stylistically similar language
For the first eight years of Sapphire’s existence, the platform’s special effects functions were written in C. At the time, image sizes were smaller and computer RAM was limited. C not only ensured Sapphire was compatible with most film editing APIs, but it ran fixed-point functions, which crucially reduced the file size of the special effect. It also offered performance benefits and allowed the team to fine-tune its math operations.
But then, the video industry started trending toward higher pixel density and image sizes, as well as high-dynamic range footage. Fixed point data representation couldn’t keep up. It was then that the team decided to port over a new language for its special effect functions, Supnik said.
Initially, the team debated about whether to stay in C and duplicate code to accommodate floating point, or adopt C++. However, C++’s ability to support floating point functions with minimal code duplication via templates gave it an edge. That, plus its compatibility with the majority of film editing APIs and ability to reduce the amount of code and coding errors made it the logical choice, Supnik said.
“One of the weird things about being a plugin developer is that a lot of choices have actually already been made for you by the thing you’re trying to plug in to.”
“One of the weird things about being a plugin developer is that a lot of choices have actually already been made for you by the thing you’re trying to plug in to,” Supnik said.
Meanwhile, floating point allowed film editors to manipulate pixels at a higher resolution and accommodate the super whites and super blacks of HD and HDR. Where fixed point might exhibit a value of zero to one, floating point could process a pixel in an effectively infinite space for purposes of color.
“If you record a candle, there will be some bright values at the center of the flame,” Supnik said. “You don’t want a plug in to assume every value over one is one because that would flatten the image. It would look crunchy.”
While C could have been converted to accommodate that, C++ templates allowed developers to share code among the fixed point, floating point and later, GPU code, Supnik said.
With a new language in chosen, the next step was to begin the rewriting process.
Complete the rewrite in pieces to make sure the code works
One of the first steps Sapphire took in adopting C++ was to examine how much of its code base actually needed to be rewritten.
At the onset, the team knew it didn’t want to write a second system in the new language. Not only would that be a massive undertaking — they would also lose important notes and troubleshooting that they’d accrued over the years working with other APIs, Supnik said.
“Rewriting everything does give you the freedom of not having to deal with some of the problems that you’ve coded into your code, but it also means you don’t carry over the knowledge and the bug fixes,” Supnik said. “And that was big for us.”
Instead, the engineers focused on rewriting code that dealt with special effects math operations, leaving things like its interface code alone since rewriting it wouldn’t offer a tangible performance boost.
“Rewriting everything does give you the freedom of not having to deal with some of the problems that you’ve coded into your code, but it also means you don’t carry over the knowledge and the bug fixes.”
To simplify the process, the team built sophisticated emacs macros that would take in a C function and spit out instructions for the C++ rewrite. The instructions included the class it belonged in, the API it’s built for and its function. From there, an engineer would rewrite the code in C++ following those instructions.
Since the engineers already wrote each math operation to work the same on different APIs, it wasn’t difficult to consolidate it into classes and rewrite it in C++, Supnik said. Still, the team proceeded with caution, making sure it only rewrote math operations in the new language.
“We were nervous about switching the entire code base from C to C++ at once, which is why we didn’t do it. We didn’t want to accidentally break something like licensing if we flipped it in C++,” Supnik said. “We pretty much stayed inside our box. Here’s what we’re changing, and everything else should stay the same.”
Conduct unit tests at each phase
In order to make sure the new code worked, Sapphire engineers built a command line tool to test their math operations.
Those tests played a critical role in Sapphire’s language transition, Supnik said. The team would start with taking the original code, placing it in its appropriate class and testing it to make sure it worked. The command tool allowed them to run a dummy version of the code’s function, meaning they could set image values, run it and compare the image to see if it worked properly.
From there, they would add the floating point version of the operation, run more tests and tweak the code based on the results until it reached the desired result. This process also allowed them to test edge cases and make sure the code behaved correctly at all times before they finalized it.
“One of our promises to customers is that they can go to the latest version of Sapphire confidently and know their old projects will still look the same.”
It was important that the C++ function could produce both fixed point and floating point images appropriately, Supnik said. Since users have created projects throughout Sapphire’s life span — and may have ongoing projects that still rely on old versions of the plugin — the company didn’t want anyone to lose any work just because they upgraded.
“One of our promises to customers is that they can go to the latest version of Sapphire confidently and know their old projects will still look the same,” Supnik said. “The test harness gave us some amount of confidence to tell customers we added floating point, but everything will still be the way they expect it to be.”
It took a year and a half to complete the first phase of rewrites, Supnik said. In total, she estimated that they changed more than 200 math operations into C++.
Rewrite the code as needed
When it comes to maintaining its codebase, Sapphire has never been shy about making changes, Supnik said.
“Our philosophy internally is ‘The right tool for the right job,’ rather than bowing to any language dogma,” Supnik said.
Since the initial C++ rewrite, Sapphire’s engineers have continued to make incremental rewrites of its code base, but only when it makes sense.
“If a problem comes up and C++ has a better way to solve it, we’d slowly transition the code over,” Supnik said. “One year we added Lambdas, another year we learned optional arguments. It was nice not to learn all of C++ at once.”
Changing its code to C++ has paid dividends, Supnik said. For one, its plugins are able to handle larger image sizes and produce more dynamic pixel ranges. It’s also allowed them to add support for NVIDIA’s graphic processing unit language, CUDA. The code translates C++ into a GPU-readable language, allowing those large files to render faster through a graphics processor.
“If a problem comes up and C++ has a better way to solve it, we’d slowly transition the code over.”
While the platform today remains flexible for new features and rewrites, Supnik has made sure the style is constant. The team has a style guide centered around the principles of C++11. Since the company produces new features on a six-month cycle now, Supnik said it’s important that everyone writes code in the same way to prevent any silos of information. That allows team members to help each other on projects when needed.
Ten years after the code rewrite, Supnik said the language port has ultimately made the company’s code base more readable and writable. For a company that produces more than 250 special effect plugins across different APIs, that consistency has made the transformation worth it.
“The thing we care about most is that, at the end of the day, our code runs very fast for our customers,” Supnik said. “And we are very good at doing that in C and C++.”
