从视频密集光流显示运动方向与幅度

光流把相邻两帧中可见的运动表示成二维向量场:每个向量描述图像上的一点从上一帧到下一帧发生了怎样的位移。物体移动会产生光流,摄像机移动也会产生光流。因此,光流图首先描述的是图像中的表观运动,不能直接当成真实世界里的速度测量。

本文依据 OpenCV 4.13.0 官方《Optical Flow》教程,完整整理其中的 Python 密集光流分支,并补充输入检查和文件保存方面的静态审查。原页未列出可确认的个人作者,发布与维护方为 OpenCV。原教程同时介绍 Lucas–Kanade 稀疏跟踪及 C++、Java 实现;按本篇选题范围,这些分支只作概念对照,不作为本文的复现路线。

相邻灰度帧经过 Farneback 算法得到水平和垂直位移,再把方向映射为色相、每帧归一化幅度映射为亮度。
图:未完纪为本文绘制的算法示意,不是实际视频截图或运行结果。色彩表示方向,亮度表示本帧内的相对幅度。

从两帧图像到每个像素的运动

原教程使用两个常见假设介绍光流:同一物体上的像素亮度在相邻帧之间大致保持不变;邻近像素具有相似运动。若上一帧的像素位于 (x, y),经过时间 dt 移到 (x + dx, y + dy),亮度恒常假设可以写成:

I(x, y, t) = I(x + dx, y + dy, t + dt)
f_x · u + f_y · v + f_t = 0
u = dx / dt, v = dy / dt

第二行来自对右侧作一阶泰勒展开。f_x、f_y 是空间梯度,f_t 是时间梯度;一条方程却有两个运动分量,必须引入更多约束。Lucas–Kanade 使用邻域信息估计选定特征点的位移,配合金字塔处理较大的运动。它通常只跟踪一组角点,得到的是稀疏光流。

密集光流则为整帧各像素位置估计运动。本文使用 cv.calcOpticalFlowFarneback(),对应 Gunnar Farnebäck 在 2003 年论文 Two-Frame Motion Estimation Based on Polynomial Expansion 中介绍的方法。它以相邻灰度图为输入,返回两通道光流数组;flow[..., 0] 和 flow[..., 1] 分别表示水平与垂直位移分量。这里的位移单位是相邻输入帧之间的像素,不是米每秒。

固定版本,并把输入视频准备齐

本文采用 CPython 3.11、opencv-python==4.13.0.92、numpy==2.2.6 作为复现基线。这是按版本资料核对的组合,本次没有安装或运行验证。4.13.0.92 对 Python 3.9 及以上的 NumPy 依赖为 2.x;不能把网页上较宽的 Requires-Python 字段理解为每个平台都一定有可用轮子。是否支持你的操作系统、架构和系统库,应以该发布版本的文件列表为准。

下面是供读者在独立实验目录采用的安装示意。Linux/macOS 使用已安装的 Python 3.11 创建虚拟环境;Windows 可将第一行换成 py -3.11 -m venv .venv,再使用 .venv\Scripts\python.exe 执行第二行对应的 pip 命令。安装会从包索引下载依赖,本次未执行。

python3.11 -m venv .venv
.venv/bin/python -m pip install --only-binary=:all: "opencv-python==4.13.0.92" "numpy==2.2.6"

opencv-python、opencv-contrib-python 及其 headless 版本共享 cv2 命名空间,一套环境只安装其中一种。本文使用标准 CPU 包并调用 imshow,所以需要可用的图形会话。无显示环境的 headless 包不能原样运行窗口代码;改成纯文件输出时,应删除窗口交互并另外设计保存策略。

官方 Python 示例读取 vtest.avi。文件位于 OpenCV 4.13.0 的 samples/data;代码位于同一 tag 的 optical_flow_dense.py。应从同一固定版本准备输入,不能假定 pip 安装包已经附带完整样例视频。cv.samples.findFile() 负责在样例搜索路径中找文件,不会下载缺失的视频。本文包含自绘示意与官方密集光流原结果图;未包含或解码该视频。

官方密集光流示例如何工作

下面保留固定 tag 的原始 Python 示例,便于对照。它先读取首帧并转为灰度,随后循环处理下一帧、计算光流、转换颜色并显示。按 Esc 结束,按 s 保存当前原帧与流场图。原例会写入当前工作目录的固定文件名,连续保存会覆盖旧文件;下节给出明确标注的整理版。

import numpy as np
import cv2 as cv
cap = cv.VideoCapture(cv.samples.findFile("vtest.avi"))
ret, frame1 = cap.read()
prvs = cv.cvtColor(frame1, cv.COLOR_BGR2GRAY)
hsv = np.zeros_like(frame1)
hsv[..., 1] = 255
while(1):
    ret, frame2 = cap.read()
    if not ret:
        print('No frames grabbed!')
        break
    next = cv.cvtColor(frame2, cv.COLOR_BGR2GRAY)
    flow = cv.calcOpticalFlowFarneback(prvs, next, None, 0.5, 3, 15, 3, 5, 1.2, 0)
    mag, ang = cv.cartToPolar(flow[..., 0], flow[..., 1])
    hsv[..., 0] = ang*180/np.pi/2
    hsv[..., 2] = cv.normalize(mag, None, 0, 255, cv.NORM_MINMAX)
    bgr = cv.cvtColor(hsv, cv.COLOR_HSV2BGR)
    cv.imshow('frame2', bgr)
    k = cv.waitKey(30) & 0xff
    if k == 27:
        break
    elif k == ord('s'):
        cv.imwrite('opticalfb.png', frame2)
        cv.imwrite('opticalhsv.png', bgr)
    prvs = next
cv.destroyAllWindows()

算法调用的参数顺序是:上一帧、下一帧、可选初始光流、pyr_scale、levels、winsize、iterations、poly_n、poly_sigma、flags。原例使用 None, 0.5, 3, 15, 3, 5, 1.2, 0,分别涉及金字塔、局部窗口、迭代和多项式近似。本文保持原例配置,没有通过测试证明它们适合其他视频。窗口大小会影响运动场的平滑程度与细节表现,调参应结合具体场景和计算成本验证。

cv.cartToPolar() 把水平、垂直分量转换成幅度与角度,默认以弧度表示角度。8 位 OpenCV HSV 图的色相使用半角度标度,因此原例以 ang * 180 / np.pi / 2 写入 H 通道;S 通道固定为 255;V 通道保存幅度的 0~255 归一化结果。最后通过 cv.COLOR_HSV2BGR 转成窗口显示使用的 BGR 图像。

读懂颜色,也读懂它的限制

同一种色相表示同一个运动方向,越亮通常表示本帧内越大的位移幅度。但 cv.NORM_MINMAX 对每一对相邻帧独立计算最小值、最大值并缩放。某一帧的亮黄色与另一帧的亮黄色不能据此认定为相同速度:色相只表示方向,亮度也没有使用跨帧固定标尺。

比较跨帧位移时,应保存原始 flow 或幅度,或者采用明确的固定幅度到亮度映射并说明饱和阈值。报告实际物理速度还需要真实时间间隔、空间标定和相机运动处理。waitKey(30) 是窗口事件等待,不等于精确的采样时间,也不能代替视频时间戳。曝光变化、遮挡、反光、纹理不足和大幅度运动,都可能使光流与真实物体运动偏离。

整理版:检查输入,避免反复覆盖快照

静态检查发现,原例第一次 cap.read() 后没有检查 ret;视频缺失或解码失败时,cvtColor 可能直接报错。它也没有检查 imwrite 的结果,没有显式释放视频句柄。以下是基于原例的修订稿,不是官方逐字代码,也未经过执行测试:保留光流参数与 HSV 映射;只接收实际存在的本地文件;检查首帧与帧尺寸;在新建的独立输出目录中用帧编号保存;检查写入结果,并用 finally 释放资源。

# Based on OpenCV 4.13.0 optical_flow_dense.py (Apache-2.0).
# Changed for WWJ: local input validation, first-frame checks,
# unique output directory, checked writes, and resource cleanup.
# Static review only: this adapted example has not been executed.
import argparse
from pathlib import Path
import tempfile
import cv2 as cv
import numpy as np

parser = argparse.ArgumentParser()
parser.add_argument("video", type=Path, help="trusted local video file")
args = parser.parse_args()
video = args.video.expanduser().resolve(strict=True)
if not video.is_file():
    raise ValueError("Input must be a local regular file")
output_dir = Path(tempfile.mkdtemp(prefix="optical-flow-", dir="."))
print("Snapshots will be saved under:", output_dir.resolve())
cap = cv.VideoCapture(str(video))
try:
    if not cap.isOpened():
        raise RuntimeError("Cannot open video")
    ok, first = cap.read()
    if not ok or first is None:
        raise RuntimeError("Cannot decode the first frame")
    previous_gray = cv.cvtColor(first, cv.COLOR_BGR2GRAY)
    hsv = np.zeros_like(first)
    hsv[..., 1] = 255
    frame_number = 0
    while True:
        ok, frame = cap.read()
        if not ok or frame is None:
            print("End of input or decode failure")
            break
        frame_number += 1
        if frame.shape != first.shape:
            raise RuntimeError("Frame dimensions changed")
        current_gray = cv.cvtColor(frame, cv.COLOR_BGR2GRAY)
        flow = cv.calcOpticalFlowFarneback(
            previous_gray, current_gray, None, 0.5, 3, 15, 3, 5, 1.2, 0
        )
        magnitude, angle = cv.cartToPolar(flow[..., 0], flow[..., 1])
        hsv[..., 0] = angle * 180 / np.pi / 2
        hsv[..., 2] = cv.normalize(magnitude, None, 0, 255, cv.NORM_MINMAX)
        flow_bgr = cv.cvtColor(hsv, cv.COLOR_HSV2BGR)
        cv.imshow("Input frame", frame)
        cv.imshow("Dense optical flow", flow_bgr)
        key = cv.waitKey(30) & 0xff
        if key == 27:
            break
        if key == ord("s"):
            for suffix, picture in (("frame", frame), ("flow", flow_bgr)):
                target = output_dir / f"{frame_number:06d}-{suffix}.png"
                if not cv.imwrite(str(target), picture):
                    raise OSError(f"Could not write {target}")
        previous_gray = current_gray
finally:
    cap.release()
    cv.destroyAllWindows()

将整理版保存为 dense_flow_reviewed.py 后,可在上述实验环境中以本地视频路径为参数运行,例如 .venv/bin/python dense_flow_reviewed.py ./vtest.avi。程序创建新的 optical-flow-* 目录,按 s 时才保存图片。读取失败可能是正常到达结尾,也可能是中途解码错误,示例不会武断地将两者等同。对不可信视频,仍须考虑媒体解码器及资源消耗风险;这里的路径检查不构成完整安全沙箱。

来源与审查说明

本文为授权中文整理,来源:OpenCV 4.13.0 — Optical Flow,并对照该 tag 的教程 Markdown、Python 示例和 Object Tracking API 文档。环境信息参见 opencv-python 4.13.0.92 与 NumPy 2.2.6。资料核对日期:2026-10-05。

OpenCV 4.13.0 仓库采用 Apache License 2.0,完整许可证文本公开可读。本文明确标注翻译整理及代码修改,保留 OpenCV 和 Farnebäck 的归属;没有把仓库许可证扩大解释为所有第三方照片、视频的独立许可。

仅作源文与代码静态审查,没有运行安装命令、视频解码、图形窗口或保存测试,也没有生成运行截图。未发现硬编码秘密或命令注入路径,不代表不存在其他漏洞。

官方密集光流结果图

OpenCV官方示例:上方为行人原帧,下方为Farnebäck密集光流HSV结果,静止背景较暗,不同行进方向呈不同色相。
OpenCV 4.13.0 Optical Flow 原图 opticalfb.jpg;保留原貌。此图是原教程结果,并非本文重新运行得到。来源:OpenCV官方原图。

OpenCV Apache License 2.0 完整通知

Apache License
                           Version 2.0, January 2004
                        http://www.apache.org/licenses/

   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION

   1. Definitions.

      "License" shall mean the terms and conditions for use, reproduction,
      and distribution as defined by Sections 1 through 9 of this document.

      "Licensor" shall mean the copyright owner or entity authorized by
      the copyright owner that is granting the License.
      "Legal Entity" shall mean the union of the acting entity and all
      other entities that control, are controlled by, or are under common
      control with that entity. For the purposes of this definition,
      "control" means (i) the power, direct or indirect, to cause the
      direction or management of such entity, whether by contract or
      otherwise, or (ii) ownership of fifty percent (50%) or more of the
      outstanding shares, or (iii) beneficial ownership of such entity.
      "You" (or "Your") shall mean an individual or Legal Entity
      exercising permissions granted by this License.

      "Source" form shall mean the preferred form for making modifications,
      including but not limited to software source code, documentation
      source, and configuration files.
      "Object" form shall mean any form resulting from mechanical
      transformation or translation of a Source form, including but
      not limited to compiled object code, generated documentation,
      and conversions to other media types.

      "Work" shall mean the work of authorship, whether in Source or
      Object form, made available under the License, as indicated by a
      copyright notice that is included in or attached to the work
      (an example is provided in the Appendix below).
      "Derivative Works" shall mean any work, whether in Source or Object
      form, that is based on (or derived from) the Work and for which the
      editorial revisions, annotations, elaborations, or other modifications
      represent, as a whole, an original work of authorship. For the purposes
      of this License, Derivative Works shall not include works that remain
      separable from, or merely link (or bind by name) to the interfaces of,
      the Work and Derivative Works thereof.
      "Contribution" shall mean any work of authorship, including
      the original version of the Work and any modifications or additions
      to that Work or Derivative Works thereof, that is intentionally
      submitted to Licensor for inclusion in the Work by the copyright owner
      or by an individual or Legal Entity authorized to submit on behalf of
      the copyright owner. For the purposes of this definition, "submitted"
      means any form of electronic, verbal, or written communication sent
      to the Licensor or its representatives, including but not limited to
      communication on electronic mailing lists, source code control systems,
      and issue tracking systems that are managed by, or on behalf of, the
      Licensor for the purpose of discussing and improving the Work, but
      excluding communication that is conspicuously marked or otherwise
      designated in writing by the copyright owner as "Not a Contribution."
      "Contributor" shall mean Licensor and any individual or Legal Entity
      on behalf of whom a Contribution has been received by Licensor and
      subsequently incorporated within the Work.
   2. Grant of Copyright License. Subject to the terms and conditions of
      this License, each Contributor hereby grants to You a perpetual,
      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
      copyright license to reproduce, prepare Derivative Works of,
      publicly display, publicly perform, sublicense, and distribute the
      Work and such Derivative Works in Source or Object form.
   3. Grant of Patent License. Subject to the terms and conditions of
      this License, each Contributor hereby grants to You a perpetual,
      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
      (except as stated in this section) patent license to make, have made,
      use, offer to sell, sell, import, and otherwise transfer the Work,
      where such license applies only to those patent claims licensable
      by such Contributor that are necessarily infringed by their
      Contribution(s) alone or by combination of their Contribution(s)
      with the Work to which such Contribution(s) was submitted. If You
      institute patent litigation against any entity (including a
      cross-claim or counterclaim in a lawsuit) alleging that the Work
      or a Contribution incorporated within the Work constitutes direct
      or contributory patent infringement, then any patent licenses
      granted to You under this License for that Work shall terminate
      as of the date such litigation is filed.
   4. Redistribution. You may reproduce and distribute copies of the
      Work or Derivative Works thereof in any medium, with or without
      modifications, and in Source or Object form, provided that You
      meet the following conditions:

      (a) You must give any other recipients of the Work or
          Derivative Works a copy of this License; and

      (b) You must cause any modified files to carry prominent notices
          stating that You changed the files; and
      (c) You must retain, in the Source form of any Derivative Works
          that You distribute, all copyright, patent, trademark, and
          attribution notices from the Source form of the Work,
          excluding those notices that do not pertain to any part of
          the Derivative Works; and
      (d) If the Work includes a "NOTICE" text file as part of its
          distribution, then any Derivative Works that You distribute must
          include a readable copy of the attribution notices contained
          within such NOTICE file, excluding those notices that do not
          pertain to any part of the Derivative Works, in at least one
          of the following places: within a NOTICE text file distributed
          as part of the Derivative Works; within the Source form or
          documentation, if provided along with the Derivative Works; or,
          within a display generated by the Derivative Works, if and
          wherever such third-party notices normally appear. The contents
          of the NOTICE file are for informational purposes only and
          do not modify the License. You may add Your own attribution
          notices within Derivative Works that You distribute, alongside
          or as an addendum to the NOTICE text from the Work, provided
          that such additional attribution notices cannot be construed
          as modifying the License.
      You may add Your own copyright statement to Your modifications and
      may provide additional or different license terms and conditions
      for use, reproduction, or distribution of Your modifications, or
      for any such Derivative Works as a whole, provided Your use,
      reproduction, and distribution of the Work otherwise complies with
      the conditions stated in this License.
   5. Submission of Contributions. Unless You explicitly state otherwise,
      any Contribution intentionally submitted for inclusion in the Work
      by You to the Licensor shall be under the terms and conditions of
      this License, without any additional terms or conditions.
      Notwithstanding the above, nothing herein shall supersede or modify
      the terms of any separate license agreement you may have executed
      with Licensor regarding such Contributions.
   6. Trademarks. This License does not grant permission to use the trade
      names, trademarks, service marks, or product names of the Licensor,
      except as required for reasonable and customary use in describing the
      origin of the Work and reproducing the content of the NOTICE file.
   7. Disclaimer of Warranty. Unless required by applicable law or
      agreed to in writing, Licensor provides the Work (and each
      Contributor provides its Contributions) on an "AS IS" BASIS,
      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
      implied, including, without limitation, any warranties or conditions
      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
      PARTICULAR PURPOSE. You are solely responsible for determining the
      appropriateness of using or redistributing the Work and assume any
      risks associated with Your exercise of permissions under this License.
   8. Limitation of Liability. In no event and under no legal theory,
      whether in tort (including negligence), contract, or otherwise,
      unless required by applicable law (such as deliberate and grossly
      negligent acts) or agreed to in writing, shall any Contributor be
      liable to You for damages, including any direct, indirect, special,
      incidental, or consequential damages of any character arising as a
      result of this License or out of the use or inability to use the
      Work (including but not limited to damages for loss of goodwill,
      work stoppage, computer failure or malfunction, or any and all
      other commercial damages or losses), even if such Contributor
      has been advised of the possibility of such damages.
   9. Accepting Warranty or Additional Liability. While redistributing
      the Work or Derivative Works thereof, You may choose to offer,
      and charge a fee for, acceptance of support, warranty, indemnity,
      or other liability obligations and/or rights consistent with this
      License. However, in accepting such obligations, You may act only
      on Your own behalf and on Your sole responsibility, not on behalf
      of any other Contributor, and only if You agree to indemnify,
      defend, and hold each Contributor harmless for any liability
      incurred by, or claims asserted against, such Contributor by reason
      of your accepting any such warranty or additional liability.
   END OF TERMS AND CONDITIONS

   APPENDIX: How to apply the Apache License to your work.
      To apply the Apache License to your work, attach the following
      boilerplate notice, with the fields enclosed by brackets "[]"
      replaced with your own identifying information. (Don't include
      the brackets!)  The text should be enclosed in the appropriate
      comment syntax for the file format. We also recommend that a
      file or class name and description of purpose be included on the
      same "printed page" as the copyright notice for easier
      identification within third-party archives.
   Copyright [yyyy] [name of copyright owner]

   Licensed under the Apache License, Version 2.0 (the "License");
   you may not use this file except in compliance with the License.
   You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0
   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
© 版权声明
THE END
喜欢就支持一下吧
点赞0 分享
评论 抢沙发

请登录后发表评论

    暂无评论内容