OpenCV Features Analysis

Posted on | Post modified: | In | | Visitors:

Features Analysis 特征分析

特征提取

在图像特征比对过程中,首先需要对进行比对的两张图进行分别的特征提取.

顶点提取 Harris Detector

选取特征点时很自然地想到要提取角的顶点.不过究竟是为什么呢?其实是因为两张图上点之间的对应关系很好确定;如果是边上的点,只能确定边的方向关系,而两张图的尺度(缩放)却无法确定;如果选取的是平面中的任意一点,则方向和尺度都无法确定.

那么如何让计算机检测到这些特殊点呢?

比较常用的方法是Harris Detector,具体原理见老师PPT😈.OpenCV中给我提供了CornerHarris函数实现这个功能

程序代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
#! /usr/local/bin/python
# -*- coding: UTF-8 -*-
import cv2
import numpy as np

img=cv2.imread("building.png",1)# flags>0,以BGR格式读入,忽略透明度的channel 0-grayscale
cv2.imshow("Original",img)# 显示图像
gray = cv2.cvtColor(img,cv2.COLOR_RGB2GRAY) 
chess_32 = np.float32(gray)
harris = cv2.cornerHarris(chess_32,2,3,0.04)#blockSize(区域大小),kSize(求导窗口),经验参数α
#result is dilated for marking the corners, not important
dst = cv2.dilate(harris,None)
# Threshold for an optimal value, it may vary depending on the image.
img[dst>0.01*dst.max()]=[0,0,255]
cv2.imshow("Harris Cornor Detector",img)# Harris

运行结果

运行后发现效果并不是很好,调整blockSize,kSize,可以获得更好一点的效果

原始图像

Harris Detector图像

SIFT 尺度不变的特征变换

提取两张图片中的同一特征时,我们通常理所应当地认为,同一区域中的均值和最大值应当是一致的,但实际上,当两张图片的尺度差异很大时,选取某一scale后,计算得到的分布结果可能会有较大的差别.

通过拉普拉斯变换输出最强的值来确定合适尺度是一种较好的方法

SIFT则是将图像切分成4×4块,分别计算每块在8个方向上的梯度,形成直方图,能较大程度地克服旋转和光照的影响.它在不同的尺度下用高斯滤波器(Gaussian filters)进行卷积(convolved)，然后利用连续高斯模糊化影像差异来找出关键点.

OpenCV中给出了直接建立SIFT detector检测特征点的函数(注意:opencv-contrib-python==3.4.2.17库而opencv-python库中没有这个模块)

程序代码

1
2
3
4
5
6
7
# 接前面代码
detector = cv2.xfeatures2d.SIFT_create() # 创建detector
kps,des = detector.detectAndCompute(gray,None) # 计算-矩阵
img=cv2.drawKeypoints(gray,kps,img) #将标记点绘制到原图像上
cv2.imshow("Features",img)# 显示图像
cv2.waitKey(0) #关闭窗口/键盘ESC退出
cv2.destroyAllWindows()

运行结果

Hough Transform 直线检测

OpenCV中提供了cv2.HoughLines函数进行Hough变换.

关于Hough Transform的原理,可以参看下面这个网页.实际上Hough不止能够检测直线,只要是能用数学形式表达的形状都可以进行检测.

https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_houghlines/py_houghlines.html

在HoughLines函数中,直线采用

\rho = x \cos \theta + y \sin \theta

的形式表示,其返回值为 (\rho, \theta) 构成的数组

程序代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
#! /usr/local/bin/python
# -*- coding: UTF-8 -*-
import cv2
import numpy as np

img = cv2.imread('building.png',1)# 读取为BGR
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)#转化为灰度图像
edges = cv2.Canny(gray,50,150,apertureSize = 3)# 边缘检测Canny Edge Detection
cv2.imshow("Canny",edges)# 显示Canny图像
lines = cv2.HoughLines(edges,2,np.pi/90,200)# 1-距离精度 np.pi/180-角度精度 100-识别为线的阈值(minimum length of line)
for i in range(len(lines)):
    for rho,theta in lines[i]:
        print(rho)
        print(theta)
        # 计算直线上的点
        a = np.cos(theta)
        b = np.sin(theta)
        x0 = a*rho
        y0 = b*rho
        x1 = int(x0 + 1000*(-b))
        y1 = int(y0 + 1000*(a))
        x2 = int(x0 - 1000*(-b))
        y2 = int(y0 - 1000*(a))
        # 绘制直线
        cv2.line(img,(x1,y1),(x2,y2),(0,0,255),2)
# 生成新图像
cv2.imwrite('houghlines.png',img)
hough = cv2.imread('houghlines.png',1)
cv2.imshow("Hough Lines",hough)# 显示Hough图像
cv2.waitKey(0) #关闭窗口/键盘ESC退出
cv2.destroyAllWindows()

运行结果

阈值100,精度(1,1°)

阈值200,精度(1,1°)

阈值300,精度(1,1°)

阈值300,精度(2,2°)

阈值200,精度(2,2°)

Image Stitching 图像拼接

对两幅图像分别进行(稳定一致的)特征提取后,可以根据特征点的比对进行图像拼接的工作,具体步骤包括:

  1. 特征检测 SIFT
  2. 特征匹配
  3. 确定对应关系
  4. 图像几何变换
  5. 图像拼接

比较详细的教程可以参考

https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_feature2d/py_matcher/py_matcher.html

https://www.cnblogs.com/xingnie/p/10230278.html

程序代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
#! /usr/local/bin/python
# -*- coding: UTF-8 -*-
import cv2
import numpy as np

img1=cv2.imread("stitch1.jpg",1)# 读图像BGR
cv2.imshow("Original 1",img1)# 显示图像
a,b,c=img1.shape
print(a)
print(b)
gray1=cv2.cvtColor(img1,cv2.COLOR_BGR2GRAY)#转化为灰度图像
img2=cv2.imread("stitch2.jpg",1)# 读图像BGR
cv2.imshow("Original 2",img2)# 显示图像
gray2=cv2.cvtColor(img2,cv2.COLOR_BGR2GRAY)#转化为灰度图像
# 检测特征
sift = cv2.xfeatures2d_SIFT().create()# 使用同一SIFT模型
kp1, des1 = sift.detectAndCompute(gray1, None)
kp2, des2 = sift.detectAndCompute(gray2, None)
# 特征匹配
FLANN_INDEX_KDTREE = 1
index_params = dict(algorithm=FLANN_INDEX_KDTREE, trees=5)
search_params = dict(checks=50)
flann = cv2.FlannBasedMatcher(index_params, search_params) # 生成比对结构
matches = flann.knnMatch(des1, des2, k=2) # 进行比对
# store all the good matches as per Lowe's ratio test.筛选匹配点
good = []
for m,n in matches:
    if m.distance < 0.7*n.distance:
        good.append(m)
# 确定对应关系
MIN_MATCH_COUNT=100
if len(good)>MIN_MATCH_COUNT:
    src_pts = np.float32([ kp1[m.queryIdx].pt for m in good ]).reshape(-1,1,2)
    dst_pts = np.float32([ kp2[m.trainIdx].pt for m in good ]).reshape(-1,1,2)
    M, mask = cv2.findHomography(src_pts, dst_pts, cv2.RANSAC,5.0)
    matchesMask = mask.ravel().tolist()
    h,w,d = img1.shape
    pts = np.float32([ [0,0],[0,h-1],[w-1,h-1],[w-1,0] ]).reshape(-1,1,2)
    dst = cv2.perspectiveTransform(pts,M)
    img2 = cv2.polylines(img2,[np.int32(dst)],True,255,3, cv2.LINE_AA)
else:
    print "Not enough matches are found - %d/%d" % (len(good),MIN_MATCH_COUNT)
    matchesMask = None
draw_params = dict(matchColor = (0,255,0), # draw matches in green color
                   singlePointColor = None,
                   matchesMask = matchesMask, # draw only inliers
                   flags = 2)
img3 = cv2.drawMatches(img1,kp1,img2,kp2,good,None,**draw_params)
cv2.imshow("Matching points",img3)# 显示图像
# 图像变换
top, bot, left, right = 200, 200, 0, 1200#拓宽图像,自己设置,显示的最终图像与img1对比
srcImg = cv2.copyMakeBorder(img1, top, bot, left, right, cv2.BORDER_CONSTANT, value=(0, 0, 0))
testImg = cv2.copyMakeBorder(img2, top, bot, left, right, cv2.BORDER_CONSTANT, value=(0, 0, 0))
rows, cols = srcImg.shape[:2]
warpImg = cv2.warpPerspective(testImg, np.array(M), (testImg.shape[1], testImg.shape[0]), flags=cv2.WARP_INVERSE_MAP)
for col in range(0, cols):
    if srcImg[:, col].any() and warpImg[:, col].any():
        left = col
        break
for col in range(cols-1, 0, -1):
    if srcImg[:, col].any() and warpImg[:, col].any():
        right = col
        break
res = np.zeros([rows, cols, 3], np.uint8)
for row in range(0, rows):
    for col in range(0, cols):
        if not srcImg[row, col].any():
            res[row, col] = warpImg[row, col]
        elif not warpImg[row, col].any():
            res[row, col] = srcImg[row, col]
        else:
            srcImgLen = float(abs(col - left))
            testImgLen = float(abs(col - right))
            alpha = srcImgLen / (srcImgLen + testImgLen)
            res[row, col] = np.clip(srcImg[row, col] * (1-alpha) + warpImg[row, col] * alpha, 0, 255)
cv2.imshow("Stitched",res)# 显示图像
cv2.waitKey(0) #关闭窗口/键盘ESC退出
cv2.destroyAllWindows()

运行效果

图像1

图像2

特征点匹配

拼接图像

0%