Skip to content

S
skill_tree_opencv

CSDN 技术社区 / skill_tree_opencv

通知 44
Star 9
Fork 1
S
skill_tree_opencv
提交 aee3f863 编写于 作者: F feilong
浏览文件
操作

fix png

上级 c65c49b6
隐藏空白更改
内联 并排
Showing with 453 addition and 453 deletion
data/1.OpenCV初阶/1.OpenCV基础/3.图像的基本操作/ball.md
浏览文件 @ aee3f863
# 梅西足球的轨迹
使用 OpenCV 可以方便的剪切粘贴图像上的区域。例如下图是梅西在踢足球
![](./ball.jpg)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/1.OpenCV基础/3.图像的基本操作/ball.jpg)
通过剪切粘贴可以获得足球连续运行的轨迹
![](./ball_continue.jpg)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/1.OpenCV基础/3.图像的基本操作/ball_continue.jpg)
实现代码如下，需要补全TODO部分：
......
data/1.OpenCV初阶/1.OpenCV基础/4.IO与GUI/doodle.md
浏览文件 @ aee3f863
# 矩形涂鸦画板
![](./doodle.png)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/1.OpenCV基础/4.IO与GUI/doodle.png)
编写一个矩形涂鸦画板，实现功能：
......
data/1.OpenCV初阶/1.OpenCV基础/4.IO与GUI/img_buffer_convert.md
浏览文件 @ aee3f863
# 甲壳虫的Base64之旅
如下的一只甲壳虫，我们希望把它编码成 Base64，再从Base64解码出来。
![](./bug.jpg)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/1.OpenCV基础/4.IO与GUI/bug.jpg)
代码框架如下：
......
data/1.OpenCV初阶/1.OpenCV基础/4.IO与GUI/img_read_write.md
浏览文件 @ aee3f863
# 甲壳虫乐队
一只甲壳虫想组个乐队，但是临时找不到队友。请使用 OpenCV 读取下面的彩色甲壳虫图片 `'bug.jpg'`，帮助他变身灰色甲壳虫，然后完成组队。
![](./bug.jpg)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/1.OpenCV基础/4.IO与GUI/bug.jpg)
**显示甲壳虫乐队并写入到 `'bug_band.jpg'`**：
![](./bug_band.jpg)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/1.OpenCV基础/4.IO与GUI/bug_band.jpg)
以下实现正确的是？
......
data/1.OpenCV初阶/2.二值图像处理/2.腐蚀与膨胀/erosion_dilation.md
浏览文件 @ aee3f863
......@@ -6,7 +6,7 @@ OpenCV的膨胀与腐蚀，让“普通鸬鹚”不普通，下图从左到右
* 原图经过腐蚀得到的图
* 原图经过膨胀得到的图
![](./bird_erosion_dilation.jpeg)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/2.二值图像处理/2.腐蚀与膨胀/bird_erosion_dilation.jpeg)
下面对腐蚀和膨胀实现代码正确的是？
......
data/1.OpenCV初阶/2.二值图像处理/3.开运算与闭运算/close.md
浏览文件 @ aee3f863
......@@ -2,7 +2,7 @@
OpenCV 里先膨胀再腐蚀操作叫做“闭运算”。小鸊鷉(pi ti)的名片被小朋友画了几笔，尝试通过先膨胀再腐蚀修复，完成任务
![](./img_closed.jpeg)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/2.二值图像处理/3.开运算与闭运算/img_closed.jpeg)
框架代码如下：
......
data/1.OpenCV初阶/2.二值图像处理/3.开运算与闭运算/open.md
浏览文件 @ aee3f863
......@@ -2,7 +2,7 @@
OpenCV 里先腐蚀再膨胀操作叫做“开运算”。小鸊鷉(pi ti)的名片被小朋友画了几笔，尝试通过先腐蚀再膨胀修复，效果不明显
![](./img_opened.jpeg)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/2.二值图像处理/3.开运算与闭运算/img_opened.jpeg)
框架代码如下：
......
data/1.OpenCV初阶/2.二值图像处理/5.轮廓/Contours.md
浏览文件 @ aee3f863
......@@ -17,7 +17,7 @@ cv.findContours(image, mode, method[, contours[, hierarchy[, offset]]] ) → con
* 康熙御笔碑帖图
* 康熙御笔轮廓图
![](./ContuorsOutput.jpg)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/2.二值图像处理/5.轮廓/ContuorsOutput.jpg)
下面对康熙御笔寻找轮廓实现代码正确的是？
......
data/1.OpenCV初阶/3.图像增强和滤波/1.彩色空间/flower.md
浏览文件 @ aee3f863
......@@ -2,7 +2,7 @@
OpenCV 的颜色空间主要有 BGR、HSV、Lab等，`cvtColor` 函数可以让图像在不同颜色空间转换。例如通过将花的图像转换到 HSV 颜色空间，在HSV空间内过滤出只含有花瓣颜色的像素，从而提取出花瓣。
![](./flower_only.jpeg)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/3.图像增强和滤波/1.彩色空间/flower_only.jpeg)
以下实现正确的是？
......
data/1.OpenCV初阶/3.图像增强和滤波/2.直方图均衡化/fish.md
浏览文件 @ aee3f863
......@@ -2,7 +2,7 @@
通过调整图像的直方图调整图像的整体细节，下图左图是浑水鱼，右边清澈鱼。
![](./fish_enhance.jpeg)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/3.图像增强和滤波/2.直方图均衡化/fish_enhance.jpeg)
框架代码如下：
......
data/1.OpenCV初阶/3.图像增强和滤波/4.梯度/Gradient.md
浏览文件 @ aee3f863
......@@ -14,7 +14,7 @@ cv.Sobel(src, ddepth, dx, dy[, dst[, ksize[, scale[, delta[, borderType]]]]])
我们用 Sobel 算子从 Lena 图像提取边缘，看看会产生什么样的效果吧。
![](./gradientOutput.png)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/3.图像增强和滤波/4.梯度/gradientOutput.png)
下面对 Sobel 梯度算子实现代码正确的是？
......
data/1.OpenCV初阶/4.几何变换和图像特征/1.仿射变换/Affine1.md
浏览文件 @ aee3f863
......@@ -12,7 +12,7 @@ cv2.warpAffine(src, M, dsize[, dst[, flags[, borderMode[, borderValue]]]]) → d
旋转也是一种简单仿射变换，我们用变换矩阵将下面的色块图旋转一下，看看会产生什么样的效果吧。
![](./affine2.png)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/4.几何变换和图像特征/1.仿射变换/affine2.png)
下面对仿射变换实现代码正确的是？
......
data/1.OpenCV初阶/4.几何变换和图像特征/2.单应性变换/rust_face.md
浏览文件 @ aee3f863
......@@ -2,17 +2,17 @@
下图左边的参赛作品（图片来自网络）掉到地上，重新做太麻烦了，据说改了个《挤地铁》的名字获奖了。
![](./rust_face_origin.jpeg)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/4.几何变换和图像特征/2.单应性变换/rust_face_origin.jpeg)
我们可以通过 OpenCV 单应性变换的方式帮助挤地铁的人脸还原。
把没挤扁和挤扁的图的坐标标注出来
![](./rust_face_rule.jpg)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/4.几何变换和图像特征/2.单应性变换/rust_face_rule.jpg)
可以看到还原后的侧脸
![](./rust_face_homography.jpeg)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/4.几何变换和图像特征/2.单应性变换/rust_face_homography.jpeg)
以下正确的实现是
......
data/1.OpenCV初阶/4.几何变换和图像特征/3.图像特征与特征提取/harris.md
浏览文件 @ aee3f863
......@@ -2,11 +2,11 @@
下图是一个湖边的亭子，通过 OpenCV 的 Harris 特征提取可以提取到亭子的瓦片特征
![](./tower.jpeg)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/4.几何变换和图像特征/3.图像特征与特征提取/tower.jpeg)
上图背景图里的植被的角点也会被 Harris 特征提取到，因此我们先用之前学习过的知识过滤出前景图后再做 Harris 特征提取，过程图片如下：
![](./tower_harris.jpeg)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/4.几何变换和图像特征/3.图像特征与特征提取/tower_harris.jpeg)
框架代码如下
......
data/1.OpenCV初阶/4.几何变换和图像特征/4.特征匹配/match.md
浏览文件 @ aee3f863
......@@ -2,7 +2,7 @@
通过特征匹配，可以把亭子和水中的倒影做特征匹配
![](./tower_match.jpeg)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/4.几何变换和图像特征/4.特征匹配/tower_match.jpeg)
框架代码如下：
......
data/1.OpenCV初阶/5.图像识别/1.人脸识别/face_detect.md
浏览文件 @ aee3f863
# 被挤压的地铁人脸检测
![](./fake_face.jpeg)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/5.图像识别/1.人脸识别/fake_face.jpeg)
即使是地铁挤压的人脸，也是有尊严的，值得被检测，经过 OpenCV 的努力，成功检测：
![](./rust_face_detect.jpeg)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/5.图像识别/1.人脸识别/rust_face_detect.jpeg)
* 左图是正常被识别的人脸
......
data/1.OpenCV初阶/5.图像识别/2.鸟图识别/detect_bird.md
浏览文件 @ aee3f863
......@@ -2,7 +2,7 @@
使用基本的OpenCV轮廓检测识别出野外拍摄照片里的鸟
![](./birds_detect.jpeg)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/5.图像识别/2.鸟图识别/birds_detect.jpeg)
基本框架如下：
......
data/1.OpenCV初阶/6.视频分析/1.光流/flow.md
浏览文件 @ aee3f863
......@@ -2,7 +2,7 @@
使用OpenCV光流分析，跟踪蚂蚁的轨迹
![](./ant_flow.jpg)
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/6.视频分析/1.光流/ant_flow.jpg)
代码框架：
......
data/1.OpenCV初阶/6.视频分析/2.目标跟踪/obj_tracker.md
浏览文件 @ aee3f863
# 目标跟踪（track by detection）
![](pic/tracker.gif)
在视频分析（或视频结构化）应用开发中，多目标跟踪是非常重要的一个环节。它能有效弥补上一个目标检测环节中算法的不足，如检测算法输出坐标不稳定、漏检等。与此同时，跟踪算法输出的目标轨迹（track-id）对于应用下阶段的行为分析环节也有着至关重要的作用。下面是常见视频分析类应用系统结构：
![](pic/video_process.png)
目标检测算法输出单帧检测结果，目标跟踪算法负责将前后2帧中的目标关联起来、给予唯一标识track-id。假设t帧中检测到了M个目标，t+1帧中检测到了N个目标，跟踪算法本质上是M->N的匹配关联过程。
![](pic/t_and_t+1_match.png)
匹配过程中，目标可以分为以下三大类：
1. `matched_tracks`，t帧目标出现，t+1帧该目标仍然出现，算法匹配上。
2. `unmatched_tracks`，t帧目标出现，t+1帧该目标消失，算法未匹配上。
3. `unmatched_detections`，t帧目标不存在，t+1帧该目标出现，新增检测目标。
其中，对于2和3来说，跟踪算法需要考虑：
> t帧目标出现，t+1帧目标其实仍然存在，但是检测算法出现短暂漏检，误认为其消失。此时的解决方案是： 某帧未被匹配到的tracks不要立即清除，而是做若干帧的缓存，等待若干帧后检测算法恢复检测<br>
> t帧目标不存在，t+1帧该目标仍然不存在，但是检测算法出现短暂误检，误认为其出现。此时的解决方案是：新增的检测目标不要立即生效，而是做若干帧的缓存，等检测算法连续检测超过若干帧、并且都能匹配关联上后再生效
之所以要考虑以上2点，主要原因是对于连续视频帧而言，大部分检测算法基本无法做到100%连续、稳定检测，出现短暂的误检、漏检非常正常。
<br/>
**题目上下文说明**:
现假设有以下跟踪代码，
```python
# 定义跟踪算法类
class Tracker(object):
# 初始化参数
def __init__(self, max_age=1, min_hits=3, iou_threshold=0.3):
self.max_age = max_age
self.min_hits = min_hits
self.iou_threshold = iou_threshold
self.trackers = []
self.frame_count = 0
# 跟踪函数，每帧检测结果返回后，调用一次update
def update(self, dets=np.empty((0, 5))):
self.frame_count += 1
trks = np.zeros((len(self.trackers), 5))
to_del = []
ret = []
for t, trk in enumerate(trks):
pos = self.trackers[t].predict()[0]
trk[:] = [pos[0], pos[1], pos[2], pos[3], 0]
if np.any(np.isnan(pos)):
to_del.append(t)
trks = np.ma.compress_rows(np.ma.masked_invalid(trks))
for t in reversed(to_del):
self.trackers.pop(t)
# 匹配关联
matched, unmatched_dets, unmatched_trks = associate_detections_to_trackers(dets, trks, self.iou_threshold)
# 后处理逻辑
# TO-DO your code...
# 返回跟踪结果 [[left, top, right, bottom, track-id]...]
if(len(ret) > 0):
return np.concatenate(ret)
return np.empty((0, 5))
```
其中：
1. `self.max_age`代表跟踪算法允许出现的最大漏检帧数
2. `self.min_hints`代表跟踪算法要求的最低连续匹配帧数
3. `self.trackers`代表跟踪算法维持的目标集合（已生成track-id）
4. `update(self, dets)`代表跟踪函数，其中参数`dets`代表t+1帧中目标检测结果list[[left, top, right, bottom, score]...]，即t+1帧中待匹配的detections
5. `associate_detections_to_trackers(...)` 代表IOU+卡尔曼滤波匹配算法，返回上面提到的`matched_tracks`，`unmatched_tracks`，`unmatched_detections`三个值
6. `time_since_update`代表目标当前漏检帧数
7. `hit_streak`代表目标当前连续匹配帧数
请你根据上面陈述补充TO-DO部分代码。
## 正确答案
```python
# 更新matched_tracks
for m in matched:
self.trackers[m[1]].update(dets[m[0], :])
# 初始化unmatched_detections，假设是当前帧新出现的检测目标
for i in unmatched_dets:
trk = KalmanBoxTracker(dets[i,:])
self.trackers.append(trk)
i = len(self.trackers)
for trk in reversed(self.trackers):
d = trk.get_state()[0]
# 输出满足条件的tracks
if (trk.time_since_update <= self.max_age) and (trk.hit_streak >= self.min_hits or self.frame_count <= self.min_hits):
ret.append(np.concatenate((d,[trk.id+1])).reshape(1,-1))
i -= 1
# 移除超过self.max_age次的漏检目标
if(trk.time_since_update > self.max_age):
self.trackers.pop(i)
```
## 未考虑unmatched_detections
```python
# 更新matched_tracks
for m in matched:
self.trackers[m[1]].update(dets[m[0], :])
i = len(self.trackers)
for trk in reversed(self.trackers):
d = trk.get_state()[0]
# 输出满足条件的tracks
if (trk.time_since_update <= self.max_age) and (trk.hit_streak >= self.min_hits or self.frame_count <= self.min_hits):
ret.append(np.concatenate((d,[trk.id+1])).reshape(1,-1))
i -= 1
# 移除超过self.max_age次的漏检目标
if(trk.time_since_update > self.max_age):
self.trackers.pop(i)
```
## 未考虑移除长时间未检测到的缓存数据
```python
# 更新matched_tracks
for m in matched:
self.trackers[m[1]].update(dets[m[0], :])
# 初始化unmatched_detections，假设是当前帧新出现的检测目标
for i in unmatched_dets:
trk = KalmanBoxTracker(dets[i,:])
self.trackers.append(trk)
for trk in reversed(self.trackers):
d = trk.get_state()[0]
# 输出满足条件的tracks
if (trk.time_since_update <= self.max_age) and (trk.hit_streak >= self.min_hits or self.frame_count <= self.min_hits):
ret.append(np.concatenate((d,[trk.id+1])).reshape(1,-1))
```
## 未考虑需要连续检出并匹配self.min_hints次才生效
```python
# 更新matched_tracks
for m in matched:
self.trackers[m[1]].update(dets[m[0], :])
# 初始化unmatched_detections，假设是当前帧新出现的检测目标
for i in unmatched_dets:
trk = KalmanBoxTracker(dets[i,:])
self.trackers.append(trk)
i = len(self.trackers)
for trk in reversed(self.trackers):
d = trk.get_state()[0]
# 输出满足条件的tracks
if (trk.time_since_update <= self.max_age):
ret.append(np.concatenate((d,[trk.id+1])).reshape(1,-1))
i -= 1
# 移除超过self.max_age次的漏检目标
if(trk.time_since_update > self.max_age):
self.trackers.pop(i)
```
# 目标跟踪（track by detection）
![](pic/tracker.gif)
在视频分析（或视频结构化）应用开发中，多目标跟踪是非常重要的一个环节。它能有效弥补上一个目标检测环节中算法的不足，如检测算法输出坐标不稳定、漏检等。与此同时，跟踪算法输出的目标轨迹（track-id）对于应用下阶段的行为分析环节也有着至关重要的作用。下面是常见视频分析类应用系统结构：
![](pic/video_process.png)
目标检测算法输出单帧检测结果，目标跟踪算法负责将前后2帧中的目标关联起来、给予唯一标识track-id。假设t帧中检测到了M个目标，t+1帧中检测到了N个目标，跟踪算法本质上是M->N的匹配关联过程。
![](pic/t_and_t+1_match.png)
匹配过程中，目标可以分为以下三大类：
1. `matched_tracks`，t帧目标出现，t+1帧该目标仍然出现，算法匹配上。
2. `unmatched_tracks`，t帧目标出现，t+1帧该目标消失，算法未匹配上。
3. `unmatched_detections`，t帧目标不存在，t+1帧该目标出现，新增检测目标。
其中，对于2和3来说，跟踪算法需要考虑：
> t帧目标出现，t+1帧目标其实仍然存在，但是检测算法出现短暂漏检，误认为其消失。此时的解决方案是： 某帧未被匹配到的tracks不要立即清除，而是做若干帧的缓存，等待若干帧后检测算法恢复检测<br>
> t帧目标不存在，t+1帧该目标仍然不存在，但是检测算法出现短暂误检，误认为其出现。此时的解决方案是：新增的检测目标不要立即生效，而是做若干帧的缓存，等检测算法连续检测超过若干帧、并且都能匹配关联上后再生效
之所以要考虑以上2点，主要原因是对于连续视频帧而言，大部分检测算法基本无法做到100%连续、稳定检测，出现短暂的误检、漏检非常正常。
<br/>
**题目上下文说明**:
现假设有以下跟踪代码，
```python
# 定义跟踪算法类
class Tracker(object):
# 初始化参数
def __init__(self, max_age=1, min_hits=3, iou_threshold=0.3):
self.max_age = max_age
self.min_hits = min_hits
self.iou_threshold = iou_threshold
self.trackers = []
self.frame_count = 0
# 跟踪函数，每帧检测结果返回后，调用一次update
def update(self, dets=np.empty((0, 5))):
self.frame_count += 1
trks = np.zeros((len(self.trackers), 5))
to_del = []
ret = []
for t, trk in enumerate(trks):
pos = self.trackers[t].predict()[0]
trk[:] = [pos[0], pos[1], pos[2], pos[3], 0]
if np.any(np.isnan(pos)):
to_del.append(t)
trks = np.ma.compress_rows(np.ma.masked_invalid(trks))
for t in reversed(to_del):
self.trackers.pop(t)
# 匹配关联
matched, unmatched_dets, unmatched_trks = associate_detections_to_trackers(dets, trks, self.iou_threshold)
# 后处理逻辑
# TO-DO your code...
# 返回跟踪结果 [[left, top, right, bottom, track-id]...]
if(len(ret) > 0):
return np.concatenate(ret)
return np.empty((0, 5))
```
其中：
1. `self.max_age`代表跟踪算法允许出现的最大漏检帧数
2. `self.min_hints`代表跟踪算法要求的最低连续匹配帧数
3. `self.trackers`代表跟踪算法维持的目标集合（已生成track-id）
4. `update(self, dets)`代表跟踪函数，其中参数`dets`代表t+1帧中目标检测结果list[[left, top, right, bottom, score]...]，即t+1帧中待匹配的detections
5. `associate_detections_to_trackers(...)` 代表IOU+卡尔曼滤波匹配算法，返回上面提到的`matched_tracks`，`unmatched_tracks`，`unmatched_detections`三个值
6. `time_since_update`代表目标当前漏检帧数
7. `hit_streak`代表目标当前连续匹配帧数
请你根据上面陈述补充TO-DO部分代码。
## 正确答案
```python
# 更新matched_tracks
for m in matched:
self.trackers[m[1]].update(dets[m[0], :])
# 初始化unmatched_detections，假设是当前帧新出现的检测目标
for i in unmatched_dets:
trk = KalmanBoxTracker(dets[i,:])
self.trackers.append(trk)
i = len(self.trackers)
for trk in reversed(self.trackers):
d = trk.get_state()[0]
# 输出满足条件的tracks
if (trk.time_since_update <= self.max_age) and (trk.hit_streak >= self.min_hits or self.frame_count <= self.min_hits):
ret.append(np.concatenate((d,[trk.id+1])).reshape(1,-1))
i -= 1
# 移除超过self.max_age次的漏检目标
if(trk.time_since_update > self.max_age):
self.trackers.pop(i)
```
## 未考虑unmatched_detections
```python
# 更新matched_tracks
for m in matched:
self.trackers[m[1]].update(dets[m[0], :])
i = len(self.trackers)
for trk in reversed(self.trackers):
d = trk.get_state()[0]
# 输出满足条件的tracks
if (trk.time_since_update <= self.max_age) and (trk.hit_streak >= self.min_hits or self.frame_count <= self.min_hits):
ret.append(np.concatenate((d,[trk.id+1])).reshape(1,-1))
i -= 1
# 移除超过self.max_age次的漏检目标
if(trk.time_since_update > self.max_age):
self.trackers.pop(i)
```
## 未考虑移除长时间未检测到的缓存数据
```python
# 更新matched_tracks
for m in matched:
self.trackers[m[1]].update(dets[m[0], :])
# 初始化unmatched_detections，假设是当前帧新出现的检测目标
for i in unmatched_dets:
trk = KalmanBoxTracker(dets[i,:])
self.trackers.append(trk)
for trk in reversed(self.trackers):
d = trk.get_state()[0]
# 输出满足条件的tracks
if (trk.time_since_update <= self.max_age) and (trk.hit_streak >= self.min_hits or self.frame_count <= self.min_hits):
ret.append(np.concatenate((d,[trk.id+1])).reshape(1,-1))
```
## 未考虑需要连续检出并匹配self.min_hints次才生效
```python
# 更新matched_tracks
for m in matched:
self.trackers[m[1]].update(dets[m[0], :])
# 初始化unmatched_detections，假设是当前帧新出现的检测目标
for i in unmatched_dets:
trk = KalmanBoxTracker(dets[i,:])
self.trackers.append(trk)
i = len(self.trackers)
for trk in reversed(self.trackers):
d = trk.get_state()[0]
# 输出满足条件的tracks
if (trk.time_since_update <= self.max_age):
ret.append(np.concatenate((d,[trk.id+1])).reshape(1,-1))
i -= 1
# 移除超过self.max_age次的漏检目标
if(trk.time_since_update > self.max_age):
self.trackers.pop(i)
```
data/1.OpenCV初阶/7.OpenCV中的深度学习/5.车辆检测/opencv-yolo-inference-vehicle.md
浏览文件 @ aee3f863
# opencv-yolo-tiny车辆检测
`opencv.dnn`模块已经支持大部分格式的深度学习模型推理，该模块可以直接加载`tensorflow`、`darknet`、`pytorch`等常见深度学习框架训练出来的模型，并运行推理得到模型输出结果。`opecnv.dnn`模块已经作为一种模型部署方式，应用在工业落地实际场景中。
![](./vehicle-detection.gif)
模型具体加载和使用流程如下：
1. 加载网络，读取模型、网络结构配置等文件
2. 创建输入，`opencv.dnn`模块对图片输入有特殊格式要求
3. 运行推理
4. 解析输出
5. 应用输出、显示输出
下面是`opencv.dnn`模块加载`yolov3-tiny`车辆检测模型并运行推理的代码，请你补充TO-DO相关代码（本题考察`yolo系列`检测模型输出解析）：
```python
import numpy as np
import cv2
import os
import time
from numpy import array
# some variables
weightsPath = './yolov3-tiny.weights'
configPath = './yolov3-tiny.cfg'
labelsPath = './obj.names'
LABELS = open(labelsPath).read().strip().split("\n")
colors = [(255, 255, 0), (255, 0, 255), (0, 255, 255), (0, 255, 0), (255, 0, 255)]
min_score = 0.3
# read darknet weights using opencv.dnn module
net = cv2.dnn.readNetFromDarknet(configPath, weightsPath)
# read video using opencv
cap = cv2.VideoCapture('./MY_TEST/8.h264')
# loop for inference
while True:
boxes = []
confidences = []
classIDs = []
start = time.time()
ret, frame = cap.read()
frame = cv2.resize(frame, (744, 416), interpolation=cv2.INTER_CUBIC)
image = frame
(H, W) = image.shape[0: 2]
# get output layer names
ln = net.getLayerNames()
out = net.getUnconnectedOutLayers()
x = []
for i in out:
x.append(ln[i[0]-1])
ln = x
# create input data package with current frame
blob = cv2.dnn.blobFromImage(image, 1 / 255.0, (416, 416), swapRB=True, crop=False)
# set as input
net.setInput(blob)
# run!
layerOutputs = net.forward(ln)
# post-process
# parsing the output and run nms
# TO-DO your code...
cv2.namedWindow('Image', cv2.WINDOW_NORMAL)
cv2.imshow("Image", image)
# print fps
stop = time.time()
fps = 1/(stop - start)
print('fps>>> :', fps)
# normal codes when displaying video
c = cv2.waitKey(1) & 0xff
if c == 27:
cap.release()
break
cv2.destroyAllWindows()
```
## 答案
```python
for output in layerOutputs:
for detection in output:
scores = detection[5:]
# class id
classID = np.argmax(scores)
# get score by classid
score = scores[classID]
# ignore if score is too low
if score >= min_score:
box = detection[0:4] * np.array([W, H, W, H])
(centerX, centerY, width, height)= box.astype("int")
x = int(centerX - (width / 2))
y = int(centerY - (height / 2))
boxes.append([x, y, int(width), int(height)])
confidences.append(float(score))
classIDs.append(classID)
# run nms using opencv.dnn module
idxs = cv2.dnn.NMSBoxes(boxes, confidences, 0.2, 0.3)
# render on image
idxs = array(idxs)
box_seq = idxs.flatten()
if len(idxs) > 0:
for seq in box_seq:
(x, y) = (boxes[seq][0], boxes[seq][1])
(w, h) = (boxes[seq][2], boxes[seq][3])
# draw what you want
color = colors[classIDs[seq]]
cv2.rectangle(image, (x, y), (x + w, y + h), color, 2)
text = "{}: {:.3f}".format(LABELS[classIDs[seq]], confidences[seq])
cv2.putText(image, text, (x, y - 5), cv2.FONT_HERSHEY_SIMPLEX, 0.3, color, 1)
```
## scores解析错误
```python
for output in layerOutputs:
for detection in output:
scores = detection[5:]
# class id
classID = np.argmax(scores)
# get score
score = detection[4]
# ignore if score is too low
if score >= min_score:
box = detection[0:4] * np.array([W, H, W, H])
(centerX, centerY, width, height)= box.astype("int")
x = int(centerX - (width / 2))
y = int(centerY - (height / 2))
boxes.append([x, y, int(width), int(height)])
confidences.append(float(score))
classIDs.append(classID)
# run nms using opencv.dnn module
idxs = cv2.dnn.NMSBoxes(boxes, confidences, 0.2, 0.3)
# render on image
idxs = array(idxs)
box_seq = idxs.flatten()
if len(idxs) > 0:
for seq in box_seq:
(x, y) = (boxes[seq][0], boxes[seq][1])
(w, h) = (boxes[seq][2], boxes[seq][3])
# draw what you want
color = colors[classIDs[seq]]
cv2.rectangle(image, (x, y), (x + w, y + h), color, 2)
text = "{}: {:.3f}".format(LABELS[classIDs[seq]], confidences[seq])
cv2.putText(image, text, (x, y - 5), cv2.FONT_HERSHEY_SIMPLEX, 0.3, color, 1)
```
## box坐标没有还原到原始输入尺寸
```python
for output in layerOutputs:
for detection in output:
scores = detection[5:]
# class id
classID = np.argmax(scores)
# get score by classid
score = scores[classID]
# ignore if score is too low
if score >= min_score:
box = detection[0:4]
(centerX, centerY, width, height)= box.astype("int")
x = int(centerX - (width / 2))
y = int(centerY - (height / 2))
boxes.append([x, y, int(width), int(height)])
confidences.append(float(score))
classIDs.append(classID)
# run nms using opencv.dnn module
idxs = cv2.dnn.NMSBoxes(boxes, confidences, 0.2, 0.3)
# render on image
idxs = array(idxs)
box_seq = idxs.flatten()
if len(idxs) > 0:
for seq in box_seq:
(x, y) = (boxes[seq][0], boxes[seq][1])
(w, h) = (boxes[seq][2], boxes[seq][3])
# draw what you want
color = colors[classIDs[seq]]
cv2.rectangle(image, (x, y), (x + w, y + h), color, 2)
text = "{}: {:.3f}".format(LABELS[classIDs[seq]], confidences[seq])
cv2.putText(image, text, (x, y - 5), cv2.FONT_HERSHEY_SIMPLEX, 0.3, color, 1)
```
## box左上角坐标解析错误
```python
for output in layerOutputs:
for detection in output:
scores = detection[5:]
# class id
classID = np.argmax(scores)
# get score by classid
score = scores[classID]
# ignore if score is too low
if score >= min_score:
box = detection[0:4] * np.array([W, H, W, H])
(x, y, width, height)= box.astype("int")
boxes.append([x, y, int(width), int(height)])
confidences.append(float(score))
classIDs.append(classID)
# run nms using opencv.dnn module
idxs = cv2.dnn.NMSBoxes(boxes, confidences, 0.2, 0.3)
# render on image
idxs = array(idxs)
box_seq = idxs.flatten()
if len(idxs) > 0:
for seq in box_seq:
(x, y) = (boxes[seq][0], boxes[seq][1])
(w, h) = (boxes[seq][2], boxes[seq][3])
# draw what you want
color = colors[classIDs[seq]]
cv2.rectangle(image, (x, y), (x + w, y + h), color, 2)
text = "{}: {:.3f}".format(LABELS[classIDs[seq]], confidences[seq])
cv2.putText(image, text, (x, y - 5), cv2.FONT_HERSHEY_SIMPLEX, 0.3, color, 1)
```
# opencv-yolo-tiny车辆检测
`opencv.dnn`模块已经支持大部分格式的深度学习模型推理，该模块可以直接加载`tensorflow`、`darknet`、`pytorch`等常见深度学习框架训练出来的模型，并运行推理得到模型输出结果。`opecnv.dnn`模块已经作为一种模型部署方式，应用在工业落地实际场景中。
![](https://gitcode.net/csdn/skill_tree_opencv/-/raw/master/data/1.OpenCV初阶/7.OpenCV中的深度学习/5.车辆检测/vehicle-detection.gif)
模型具体加载和使用流程如下：
1. 加载网络，读取模型、网络结构配置等文件
2. 创建输入，`opencv.dnn`模块对图片输入有特殊格式要求
3. 运行推理
4. 解析输出
5. 应用输出、显示输出
下面是`opencv.dnn`模块加载`yolov3-tiny`车辆检测模型并运行推理的代码，请你补充TO-DO相关代码（本题考察`yolo系列`检测模型输出解析）：
```python
import numpy as np
import cv2
import os
import time
from numpy import array
# some variables
weightsPath = './yolov3-tiny.weights'
configPath = './yolov3-tiny.cfg'
labelsPath = './obj.names'
LABELS = open(labelsPath).read().strip().split("\n")
colors = [(255, 255, 0), (255, 0, 255), (0, 255, 255), (0, 255, 0), (255, 0, 255)]
min_score = 0.3
# read darknet weights using opencv.dnn module
net = cv2.dnn.readNetFromDarknet(configPath, weightsPath)
# read video using opencv
cap = cv2.VideoCapture('./MY_TEST/8.h264')
# loop for inference
while True:
boxes = []
confidences = []
classIDs = []
start = time.time()
ret, frame = cap.read()
frame = cv2.resize(frame, (744, 416), interpolation=cv2.INTER_CUBIC)
image = frame
(H, W) = image.shape[0: 2]
# get output layer names
ln = net.getLayerNames()
out = net.getUnconnectedOutLayers()
x = []
for i in out:
x.append(ln[i[0]-1])
ln = x
# create input data package with current frame
blob = cv2.dnn.blobFromImage(image, 1 / 255.0, (416, 416), swapRB=True, crop=False)
# set as input
net.setInput(blob)
# run!
layerOutputs = net.forward(ln)
# post-process
# parsing the output and run nms
# TO-DO your code...
cv2.namedWindow('Image', cv2.WINDOW_NORMAL)
cv2.imshow("Image", image)
# print fps
stop = time.time()
fps = 1/(stop - start)
print('fps>>> :', fps)
# normal codes when displaying video
c = cv2.waitKey(1) & 0xff
if c == 27:
cap.release()
break
cv2.destroyAllWindows()
```
## 答案
```python
for output in layerOutputs:
for detection in output:
scores = detection[5:]
# class id
classID = np.argmax(scores)
# get score by classid
score = scores[classID]
# ignore if score is too low
if score >= min_score:
box = detection[0:4] * np.array([W, H, W, H])
(centerX, centerY, width, height)= box.astype("int")
x = int(centerX - (width / 2))
y = int(centerY - (height / 2))
boxes.append([x, y, int(width), int(height)])
confidences.append(float(score))
classIDs.append(classID)
# run nms using opencv.dnn module
idxs = cv2.dnn.NMSBoxes(boxes, confidences, 0.2, 0.3)
# render on image
idxs = array(idxs)
box_seq = idxs.flatten()
if len(idxs) > 0:
for seq in box_seq:
(x, y) = (boxes[seq][0], boxes[seq][1])
(w, h) = (boxes[seq][2], boxes[seq][3])
# draw what you want
color = colors[classIDs[seq]]
cv2.rectangle(image, (x, y), (x + w, y + h), color, 2)
text = "{}: {:.3f}".format(LABELS[classIDs[seq]], confidences[seq])
cv2.putText(image, text, (x, y - 5), cv2.FONT_HERSHEY_SIMPLEX, 0.3, color, 1)
```
## scores解析错误
```python
for output in layerOutputs:
for detection in output:
scores = detection[5:]
# class id
classID = np.argmax(scores)
# get score
score = detection[4]
# ignore if score is too low
if score >= min_score:
box = detection[0:4] * np.array([W, H, W, H])
(centerX, centerY, width, height)= box.astype("int")
x = int(centerX - (width / 2))
y = int(centerY - (height / 2))
boxes.append([x, y, int(width), int(height)])
confidences.append(float(score))
classIDs.append(classID)
# run nms using opencv.dnn module
idxs = cv2.dnn.NMSBoxes(boxes, confidences, 0.2, 0.3)
# render on image
idxs = array(idxs)
box_seq = idxs.flatten()
if len(idxs) > 0:
for seq in box_seq:
(x, y) = (boxes[seq][0], boxes[seq][1])
(w, h) = (boxes[seq][2], boxes[seq][3])
# draw what you want
color = colors[classIDs[seq]]
cv2.rectangle(image, (x, y), (x + w, y + h), color, 2)
text = "{}: {:.3f}".format(LABELS[classIDs[seq]], confidences[seq])
cv2.putText(image, text, (x, y - 5), cv2.FONT_HERSHEY_SIMPLEX, 0.3, color, 1)
```
## box坐标没有还原到原始输入尺寸
```python
for output in layerOutputs:
for detection in output:
scores = detection[5:]
# class id
classID = np.argmax(scores)
# get score by classid
score = scores[classID]
# ignore if score is too low
if score >= min_score:
box = detection[0:4]
(centerX, centerY, width, height)= box.astype("int")
x = int(centerX - (width / 2))
y = int(centerY - (height / 2))
boxes.append([x, y, int(width), int(height)])
confidences.append(float(score))
classIDs.append(classID)
# run nms using opencv.dnn module
idxs = cv2.dnn.NMSBoxes(boxes, confidences, 0.2, 0.3)
# render on image
idxs = array(idxs)
box_seq = idxs.flatten()
if len(idxs) > 0:
for seq in box_seq:
(x, y) = (boxes[seq][0], boxes[seq][1])
(w, h) = (boxes[seq][2], boxes[seq][3])
# draw what you want
color = colors[classIDs[seq]]
cv2.rectangle(image, (x, y), (x + w, y + h), color, 2)
text = "{}: {:.3f}".format(LABELS[classIDs[seq]], confidences[seq])
cv2.putText(image, text, (x, y - 5), cv2.FONT_HERSHEY_SIMPLEX, 0.3, color, 1)
```
## box左上角坐标解析错误
```python
for output in layerOutputs:
for detection in output:
scores = detection[5:]
# class id
classID = np.argmax(scores)
# get score by classid
score = scores[classID]
# ignore if score is too low
if score >= min_score:
box = detection[0:4] * np.array([W, H, W, H])
(x, y, width, height)= box.astype("int")
boxes.append([x, y, int(width), int(height)])
confidences.append(float(score))
classIDs.append(classID)
# run nms using opencv.dnn module
idxs = cv2.dnn.NMSBoxes(boxes, confidences, 0.2, 0.3)
# render on image
idxs = array(idxs)
box_seq = idxs.flatten()
if len(idxs) > 0:
for seq in box_seq:
(x, y) = (boxes[seq][0], boxes[seq][1])
(w, h) = (boxes[seq][2], boxes[seq][3])
# draw what you want
color = colors[classIDs[seq]]
cv2.rectangle(image, (x, y), (x + w, y + h), color, 2)
text = "{}: {:.3f}".format(LABELS[classIDs[seq]], confidences[seq])
cv2.putText(image, text, (x, y - 5), cv2.FONT_HERSHEY_SIMPLEX, 0.3, color, 1)
```
src/img.py
浏览文件 @ aee3f863
......@@ -37,5 +37,5 @@ class ImgWalker():
md_new.append(new_line)
md_new.append('')
simple_list_md_dump(md_file, md_new)
import sys
sys.exit(0)
# import sys
# sys.exit(0)
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或