所需要的Python的包：

  • pytorch
  • matplotlib
  • opencv

此次我们以LeNet进行举例

Lenet是一个 7 层的神经网络，包含 3 个卷积层，2 个池化层，1 个全连接层。其中所有卷积层的所有卷积核都为 5x5，步长 strid=1，池化方法都为全局 pooling，激活函数为 Sigmoid，网络结构如下：

image-20220824175322171

下面我们就来可视化一下它的三个卷积层。

首先定义LeNet的网络结构。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
from torch import nn

class LeNet(nn.Module):
    def __init__(self):
        super(LeNet,self).__init__()
        self.RELU = nn.ReLU()
        self.conv1 = nn.Conv2d(in_channels=3,out_channels=6,kernel_size=(5,5),padding=(2,2))
        self.Sigmoid = nn.Sigmoid()
        self.avgpool = nn.AvgPool2d(kernel_size=(2,2),stride=2)
        self.conv2 = nn.Conv2d(in_channels=6,out_channels=16,kernel_size=(5,5))
        self.avgpool2 = nn.AvgPool2d(kernel_size=(2,2),stride=(2,2))
        self.conv3 = nn.Conv2d(in_channels=16,out_channels=120,kernel_size=(5,5))

        self.flatten = nn.Flatten()
        self.line = nn.Linear(120,84)
        self.output = nn.Sequential(
            nn.Linear(in_features=84,out_features=2,bias=False),
            nn.Sigmoid(),
        )
    def forward(self,x):
        x = self.RELU(self.conv1(x))
        x = self.avgpool(x)
        x = self.RELU(self.conv2(x))
        x = self.avgpool2(x)
        x = self.conv3(x)
        x = self.flatten(x)
        x = self.line(x)
        return self.output(x)

结构是稍微有一些改变的，比如：

  • 激活函数我改用了RELU

  • 输入层的通道数从单通道改为了三通道

不过这些都问题不大，不影响我们去可视化。

导包

1
2
3
4
5
6
import torch
import matplotlib.pyplot as plt
import cv2 as cv
from torch.autograd import Variable
from torchvision import transforms
from LeNet import LeNet  #我把网络结构的代码写到了LeNet.py，所以需要从另一个文件里导入

首先我们需要初始化我们的网络模型，并加载之前训练好的权重文件

1
2
3
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = LeNet().eval()
model.load_state_dict(torch.load("LeNet_model.pth",map_location="cpu"))

注意，如果你的网络有BN层，LeNet().eval()是一定要加的，.eval()表示模型进入测试模式。

定义两个列表容器，一个存卷积层的权重，一个存卷积层。

1
2
model_weights =[]
conv_layers = []

递归的遍历卷积神经网络，将卷积层和卷积层的权重存起来

1
2
3
4
5
6
7
8
def getConv(children):
    for child in children:
        if child.__class__.__name__ == "Conv2d":
            model_weights.append(child.weight)
            conv_layers.append(child)
        elif child.__class__.__name__=="Sequential":
            getConv(child.children())
getConv(model.children())

然后的步骤就类似于测试的步骤

  1. 读取图片

    1
    image = cv.imread("20160330_170504_473.jpg")

  2. 对图片进行transform变换

    1
    2
    3
    4
    5
    6
    7
    8
    transform = transforms.Compose(
                [
                    transforms.ToPILImage(),
                    transforms.Resize((28,28)),
                    transforms.ToTensor(),
                 ]
            )
    transform_image = transform(image)

  3. 由于transform_image 的维度是[3,28,28]的，我们要把他扩展成[1, 3, 28, 28]

    1
    x = Variable(torch.unsqueeze(transform_image,dim=0).float(),requires_grad = False)

  4. 利用之前的存起来的卷积层，对图片依次进行计算得到了featuremap，然后再依次存起来。

    1
    2
    3
    4
    5
    6
    outputs = []
    names = []
    for layer in conv_layers:
        x = layer(x)
        outputs.append(x)
        names.append(layer.__class__.__name__)

由于featuremap通道数并不是单通道，而是多通道的。我们可以把每个通道进行可视化，也可以将这些通道取平均，压缩为单通道。

  • 取平均，压缩为单通道**[建议压缩为单通道]**

    1
    2
    3
    4
    5
    results = []
    for output in outputs:
        output = output.squeeze(0)
        gray_scale = torch.sum(output,0)/output.shape[0]
        results.append(gray_scale.detach().numpy())

  • 存每一个通道

    1
    2
    3
    4
    5
    results = []
    for output in outputs:
        output = output.squeeze(0)
        for feature in output:
            results.append(feature.detach().numpy())

最后就可以使用matplotlib可视化了

1
2
3
4
for result in results:
    plt.imshow(result)
    #plt.imshow(result,cmap="gray")
    plt.show()

结果：
原图 20160330_170504_473
第一层featuremap 1
第二层featuremap 2
第三层featuremap 3

代码：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
import torch
import matplotlib.pyplot as plt
import cv2 as cv
from torch.autograd import Variable
from torchvision import transforms
from LeNet import LeNet


def getConv(children):
    for child in children:
        if child.__class__.__name__ == "Conv2d":
            model_weights.append(child.weight)
            conv_layers.append(child)
        elif child.__class__.__name__=="Sequential":
            getConv(child.children())

if __name__ == '__main__':
    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
    model = LeNet().eval()
    model.load_state_dict(torch.load("LeNet_model.pth",map_location="cpu"))
    children = model.children()
    
    
    model_weights =[]
    conv_layers = []
    
    getConv(children)
    model.to(device)
    transform = transforms.Compose(
                [
                    transforms.ToPILImage(),
                    transforms.Resize((28,28)),
                    transforms.ToTensor(),
                 ]
            )
    image = cv.imread("20160330_170504_473.jpg")
    transform_image = transform(image)
    x = Variable(torch.unsqueeze(transform_image,dim=0).float(),requires_grad = False)
    outputs = []
    names = []
    for layer in conv_layers:
        x = layer(x)
        outputs.append(x)
        names.append(layer.__class__.__name__)
    
    
    results = []
    for output in outputs:
        output = output.squeeze(0)
        gray_scale = torch.sum(output,0)/output.shape[0]
        results.append(gray_scale.detach().numpy())
    
    
    for result in results:
        plt.imshow(result)
        plt.show()

还可以

应用于其他经典网络，比如Resnet、Alexnet、VGG等等。

__END__