Work in Parallel to Streamline the VQ Measurement Workflow

Every once in while you write a column knowing that a certain percentage of readers will think you’re a simpleton for using a certain workflow for so long, while another percentage will secretly thank you for showing them a better way. 这就是这样一列.

为了解释，大多数视频质量测量任务包含多个操作. 第一个, 对文件进行编码, then if it’s a new format (like AV1) or a certain tool (like vmafossexec, the open-source Netflix tool for computing Video Multimethod Assessment Fusion, 或VMAF)，只接受YUV文件, 转换成YUV. Then you run the objective quality measurement tool and compute the score. Then you delete the files and start on the next batch.

There are several positive outcomes of performing these tasks serially. 第一个, the batch files are simple to write, which minimizes the risk of error. 第二个, often you have checks to run on the encoded files before you process further, 比如检查文件大小. However, there are also several key negatives. 第一个, you have to be there to trigger the subsequent events, so if encoding stops at 3 a.m., you probably won’t start measuring quality until the next morning, losing valuable machine hours. 除了, because YUV files are incredibly large, even if you have very large SSD disks you often have to limit the number of files you process to avoid running out of disk space, 失去更多的时间.

I started with this serial workflow back in 2014 when I began working with video quality metrics. I have thousands of batch files on my various test systems, efficiently labeled encode _.Bat用于编码任务，scale_.bat用于缩放，MSU_.测量VMAF的蝙蝠, 结构相似度(SSIM), or peak signal-to-noise ratio (PSNR) with the Moscow State University 视频质量 Measurement Tool (VQMT). Since deleting large files is easily managed through Windows Explorer, 我只是手动删除, avoiding yet a fourth class of batch files.

就在最近, 在做咨询项目的时候, a colleague sent some of his batch files for my review and verification. 而不是连续工作, 他的工作相对平行, 首先对文件进行编码, 然后进行转换, 然后测量质量, 然后删除YUV文件. When I saw his approach, I didn’t slap my forehead and proclaim “D’Oh!“但我觉得我应该这么做, since it cures several of the key problems of my serial approach and had several key advantages.

第一个, since his single batch accomplished all four tasks, I didn’t need to be there to launch the separate batch files, keeping the process going day and night. 第二个, it eliminated the disk space issues, so I could process an almost-unlimited number of files. 最后, when his process was done, it was done; all that was left was to record the data. With my serial approach, I had to take three separate steps before I could harvest my data.

Of course, every silver lining has a dark cloud, and this parallel approach is no exception. 第一个, the parallel batch file is much more challenging to create and debug, increasing the risk of failure and taking much longer to create. 第二个, you really can’t build file-checking into the batch file, so if the file’s data rate is too low or two high or if the file is otherwise defective, 无论如何你都会处理它, 浪费你的机器时间. 最后, many times you want to visualize the YUV or Y4M files in a tool like VQMT and you’ll have already deleted them. 也就是说, conversion to YUV is generally a fast operation, so you can quickly recreate them for viewing. 如果你需要一些天平.bat files, I’ve got thousands I can send.)

在很大程度上, 然而, I’ve seen the light and am changing my workflow from serial to (closer to) parallel. This will undoubtedly streamline future projects, but it leaves me uncomfortably wondering how many other tasks I’m performing equally inefficiently simply because no one has ever shown me a better way.

[This article appears in the April/May 2019 issue of 流媒体杂志 as "Streamlining Your VQ Measurement Workflow"]