SSD - VGG backbone feature

VGG16 feature extract 할 때 feature shape 및 추출 부분 메모

 

input : [batch, 3, 300, 300]

 

ModuleList(
  (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) # torch.Size([32, 64, 300, 300])
  (1): ReLU(inplace=True) # torch.Size([32, 64, 300, 300])
  (2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) # torch.Size([32, 64, 300, 300])
  (3): ReLU(inplace=True) # torch.Size([32, 64, 300, 300])
  (4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False) # torch.Size([32, 64, 150, 150])
  (5): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) # torch.Size([32, 128, 150, 150])
  (6): ReLU(inplace=True) # torch.Size([32, 128, 150, 150])
  (7): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) # torch.Size([32, 128, 150, 150])
  (8): ReLU(inplace=True) # torch.Size([32, 128, 150, 150])
  (9): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False) # torch.Size([32, 128, 75, 75])
  (10): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) # torch.Size([32, 256, 75, 75])
  (11): ReLU(inplace=True) # torch.Size([32, 256, 75, 75])
  (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) # torch.Size([32, 256, 75, 75])
  (13): ReLU(inplace=True) # torch.Size([32, 256, 75, 75])
  (14): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) # torch.Size([32, 256, 75, 75])
  (15): ReLU(inplace=True) # torch.Size([32, 256, 75, 75])
  (16): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=True) # torch.Size([32, 256, 38, 38])
  (17): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) # torch.Size([32, 512, 38, 38])
  (18): ReLU(inplace=True) # torch.Size([32, 512, 38, 38])
  (19): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) # torch.Size([32, 512, 38, 38])
  (20): ReLU(inplace=True) # torch.Size([32, 512, 38, 38])
  (21): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) # torch.Size([32, 512, 38, 38])
  (22): ReLU(inplace=True) # torch.Size([32, 512, 38, 38])
  # 여기까지 처리하여 backbone feature를 가져옴
  
  (23): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (24): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (25): ReLU(inplace=True)
  (26): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (27): ReLU(inplace=True)
  (28): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (29): ReLU(inplace=True)
  (30): MaxPool2d(kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
  (31): Conv2d(512, 1024, kernel_size=(3, 3), stride=(1, 1), padding=(6, 6), dilation=(6, 6))
  (32): ReLU(inplace=True)
  (33): Conv2d(1024, 1024, kernel_size=(1, 1), stride=(1, 1))
  (34): ReLU(inplace=True) # torch.Size([32, 1024, 19, 19])
 # 또 위 부분까지 처리하여 feature append
)

 

위 (22) ReLU에서 feature를 출력한다음 l2 normalization을 취한다.

 

norm = x.pow(2).sum(dim=1, keepdim=True).sqrt() + self.eps

(23)부터 (34)까지 또 처리하여 feature map ([batch, 1024, 19, 19])를 얻는다.

 

 

 

extra layers

ModuleList(
  (0): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1)) # torch.Size([32, 256, 19, 19])
  (1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1)) # torch.Size([32, 512, 10, 10]) -- O
  (2): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1)) # torch.Size([32, 128, 10, 10])
  (3): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1)) # torch.Size([32, 256, 5, 5])  -- O
  (4): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1)) # torch.Size([32, 128, 5, 5])
  (5): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1)) # torch.Size([32, 256, 3, 3])  -- O
  (6): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1)) # torch.Size([32, 128, 3, 3])
  (7): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1)) # torch.Size([32, 256, 1, 1])  -- O
)

extra layer에서 위 o표시로 layer로 되어 있는 부분을 가져온다.

 

최종적으로 backbone에서 추출할 feature map의 개수는 6개이며 shape은 아래와 같다.

(torch.Size([32, 512, 38, 38]), 
torch.Size([32, 1024, 19, 19]), 
torch.Size([32, 512, 10, 10]), 
torch.Size([32, 256, 5, 5]), 
torch.Size([32, 256, 3, 3]), 
torch.Size([32, 256, 1, 1]))

 

https://github.com/lufficc/SSD

GitHub - lufficc/SSD: High quality, fast, modular reference implementation of SSD in PyTorch