Posts 특징 추출기 (Feature Extractor)
Post
Cancel

특징 추출기 (Feature Extractor)

1. 특징 추출기


1.1 특징 추출기

  • 미리 훈련된 모델에서는 데이터의 특징만 추출
  • 그 특징을 작은 네트워크에 통과시켜 예측하는 방법
  • 학습할때 전체 네트워크의 계산을 반복할 필요가 없음


1.2 Inception V3


1.3 Inception Load

1
2
3
4
5
6
7
8
9
import tensorflow_hub as hub
import tensorflow as tf

inception_url = 'https://tfhub.dev/google/tf2-preview/inception_v3/feature_vector/4'
feature_model = tf.keras.Sequential([
    hub.KerasLayer(inception_url, output_shape=(2048,), trainable = False)
])
feature_model.build([None, 299, 299, 3])
feature_model.summary()
1
2
3
4
5
6
7
8
9
10
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
keras_layer (KerasLayer)     (None, 2048)              21802784  
=================================================================
Total params: 21,802,784
Trainable params: 0
Non-trainable params: 21,802,784
_________________________________________________________________
  • 파라미터의 갯수가 엄청 많다.


1.4 라벨 불러오기

1
2
3
4
5
6
7
8
label_url = 'https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt'
label_file = tf.keras.utils.get_file('label', label_url)
label_text = None
with open(label_file, 'r') as f:
    label_text = f.read().split('\n')[:-1]
print(len(label_text))
print(label_text[:10])
print(label_text[-10:])
1
2
3
1001
['background', 'tench', 'goldfish', 'great white shark', 'tiger shark', 'hammerhead', 'electric ray', 'stingray', 'cock', 'hen']
['buckeye', 'coral fungus', 'agaric', 'gyromitra', 'stinkhorn', 'earthstar', 'hen-of-the-woods', 'bolete', 'ear', 'toilet tissue']


1
2
3
import pandas as pd
label_text = pd.read_csv('data/dog_data/labels.csv')
print(label_text.head())
1
2
3
4
5
6
                                 id             breed
0  000bec180eb18c7604dcecc8fe0dba07       boston_bull
1  001513dfcb2ffafc82cccf4d8bbaba97             dingo
2  001cdf01b096e06d78e9e5112d419397          pekinese
3  00214f311d5d2247d5dfe4fe24b2303d          bluetick
4  0021f9ceb3235effd7fcde7f7538ed62  golden_retriever
  • 다운받은 csv는 이런 형태


1.5 ImageDataGenerator 사용 준비

1
2
3
4
5
6
7
8
9
10
import os
import shutil

os.mkdir('./data/train_sub')

for i in range(len(label_text)):
    if os.path.exists('./data/train_sub/' + label_text.loc[i]['breed']) == False:
        os.mkdir('./data/train_sub/' + label_text.loc[i]['breed'])
    shutil.copy('./data/dog_data/train/' +
                label_text.loc[i]['id'] + '.jpg', './data/train_sub/' + label_text.loc[i]['breed'])
  • ImageDataGenerator를 사용하기 위한 준비
  • ImageDataGenerator는 사전 데이터를 증강하는 기능
  • 시간 데이터가 너무 많아서 어쩌면 메모리 부족 현상이 나타날수 있음
  • 그럴때 ImageDataGenerator를 이용하면 필요할때마다 디스크에서 배치 크기만큼 조금씩 데이터를 읽을수 있음
  • 단, ImageDataGenerator를 사용하려면 각 라벨의 이름을 하위 폴더로 가지고 있도록 해야함


1.6 ImageDataGenerator 생성

1
2
3
4
5
6
7
8
9
10
from tensorflow.python.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.applications.inception_resnet_v2 import preprocess_input

image_size = 299
batch_size = 32

train_datagen = ImageDataGenerator(rescale=1 / 255., horizontal_flip=True, shear_range=0.2,
                                   zoom_range=0.2, width_shift_range=0.2, height_shift_range=0.2, validation_split=0.25)

valid_datagen = ImageDataGenerator(rescale=1 / 255., validation_split=0.25)
  • 훈련용 데이터는 픽셀정규화, 좌우반전, 기울이기, 줌, 좌우/상하 평행이동을 함
  • 테스트용 데이터는 픽셀 정규화만함


1.7 데이터 생성

1
2
3
4
train_generator = train_datagen.flow_from_directory(directory='./data/train_sub/', subset='training',
                                                    batch_size=batch_size, seed=13, shuffle=True, class_mode='categorical', target_size=(image_size, image_size))
valid_generator = valid_datagen.flow_from_directory(directory='./data/train_sub/', subset='validation',
                                                    batch_size=1, seed=13, shuffle=True, class_mode='categorical', target_size=(image_size, image_size))
1
2
Found 7718 images belonging to 120 classes.
Found 2504 images belonging to 120 classes.
  • 7718장의 train 이미지와 2504장의 test 이미지를 각각 가져왔고, 120개의 class(종)을 가져옴


1.8 데이터의 구조


1.9 훈련 데이터를 특징 벡터로 변환

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
batch_step = (7718 * 3) // batch_size
train_features = []
train_Y = []

for idx in range(batch_step):
    if idx % 100 == 0:
        print(idx)
    x, y = train_generator.next()
    train_Y.extend(y)
    
    feature = feature_model.predict(x)
    train_features.extend(feature)
    
train_features = np.array(train_features)
train_Y = np.array(train_Y)
print(train_features.shape)
print(train_Y.shape)
1
2
3
4
5
6
7
8
9
10
0
100
200
300
400
500
600
700
(23084, 2048)
(23084, 120)
  • 시간이 꽤 오래 걸린다.


1.10 검증 데이터에서 특징 벡터 추출

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
valid_features = []
valid_Y = []

for idx in range(valid_generator.n):
    if idx % 100 == 0:
        print(idx)
        
    x, y = valid_generator.next()
    valid_Y.extend(y)
    
    feature = feature_model.predict(x)
    valid_features.extend(feature)
    
valid_features = np.array(valid_features)
valid_Y = np.array(valid_Y)
print(valid_features.shape)
print(valid_Y.shape)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
0
100
200
300
400
500
600
700
800
900
1000
1100
1200
1300
1400
1500
1600
1700
1800
1900
2000
2100
2200
2300
2400
2500
(2504, 2048)
(2504, 120)
  • 2504개의 Validation 데이터 특징 추출완료


1.11 모델 생성

1
2
3
4
5
6
7
8
9
model = tf.keras.Sequential([
    tf.keras.layers.Dense(256, activation='relu', input_shape=(2048, )),
    tf.keras.layers.Dropout(0.5),
    tf.keras.layers.Dense(120, activation='softmax')
])

model.compile(tf.optimizers.RMSprop(0.0001),
              loss='categorical_crossentropy', metrics=['accuracy'])
model.summary()
1
2
3
4
5
6
7
8
9
10
11
12
13
14
Model: "sequential_3"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_4 (Dense)              (None, 256)               524544    
_________________________________________________________________
dropout_2 (Dropout)          (None, 256)               0         
_________________________________________________________________
dense_5 (Dense)              (None, 120)               30840     
=================================================================
Total params: 555,384
Trainable params: 555,384
Non-trainable params: 0
_________________________________________________________________
  • 파라미터가 처음 2100만개에서 55만개로 줄었음


1.12 Fit

1
2
history = model.fit(train_features, train_Y, validation_data=(
    valid_features, valid_Y), epochs=10, batch_size=12)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Epoch 1/10
1924/1924 [==============================] - 7s 3ms/step - loss: 2.2162 - accuracy: 0.5509 - val_loss: 0.5050 - val_accuracy: 0.8826
Epoch 2/10
1924/1924 [==============================] - 7s 4ms/step - loss: 0.6702 - accuracy: 0.8140 - val_loss: 0.3430 - val_accuracy: 0.8938
Epoch 3/10
1924/1924 [==============================] - 7s 4ms/step - loss: 0.5090 - accuracy: 0.8479 - val_loss: 0.3194 - val_accuracy: 0.9010
Epoch 4/10
1924/1924 [==============================] - 7s 4ms/step - loss: 0.4414 - accuracy: 0.8645 - val_loss: 0.3088 - val_accuracy: 0.9014
Epoch 5/10
1924/1924 [==============================] - 7s 4ms/step - loss: 0.4002 - accuracy: 0.8763 - val_loss: 0.3083 - val_accuracy: 0.8998
Epoch 6/10
1924/1924 [==============================] - 7s 3ms/step - loss: 0.3608 - accuracy: 0.8872 - val_loss: 0.3105 - val_accuracy: 0.8990
Epoch 7/10
1924/1924 [==============================] - 6s 3ms/step - loss: 0.3401 - accuracy: 0.8936 - val_loss: 0.3172 - val_accuracy: 0.9002
Epoch 8/10
1924/1924 [==============================] - 6s 3ms/step - loss: 0.3144 - accuracy: 0.9003 - val_loss: 0.3187 - val_accuracy: 0.9010
Epoch 9/10
1924/1924 [==============================] - 6s 3ms/step - loss: 0.2909 - accuracy: 0.9076 - val_loss: 0.3164 - val_accuracy: 0.8978
Epoch 10/10
1924/1924 [==============================] - 6s 3ms/step - loss: 0.2765 - accuracy: 0.9111 - val_loss: 0.3307 - val_accuracy: 0.8970
  • 학습속도가 빠르다.


1.13 학습 상황 그래프

1
2
3
4
5
6
7
8
9
10
import matplotlib.pyplot as plt

plt.figure(figsize=(12, 8 ))
plt.plot(history.history['loss'], 'b-', label='loss')
plt.plot(history.history['val_loss'], 'r--', label='val_loss')
plt.plot(history.history['accuracy'], 'g-', label='accuracy')
plt.plot(history.history['val_accuracy'], 'k--', label='val_accuracy')
plt.xlabel('Epoch')
plt.legend()
plt.show()

  • 학습이 잘되는건지..


1.14 라벨을 알파벳순으로

1
2
3
unique_Y = label_text['breed'].unique().tolist()
unique_sorted_Y = sorted(unique_Y)
print(unique_sorted_Y)
1
['affenpinscher', 'afghan_hound', 'african_hunting_dog', 'airedale', 'american_staffordshire_terrier', 'appenzeller', 'australian_terrier', 'basenji', 'basset', 'beagle', 'bedlington_terrier', 'bernese_mountain_dog', 'black-and-tan_coonhound', 'blenheim_spaniel', 'bloodhound', 'bluetick', 'border_collie', 'border_terrier', 'borzoi', 'boston_bull', 'bouvier_des_flandres', 'boxer', 'brabancon_griffon', 'briard', 'brittany_spaniel', 'bull_mastiff', 'cairn', 'cardigan', 'chesapeake_bay_retriever', 'chihuahua', 'chow', 'clumber', 'cocker_spaniel', 'collie', 'curly-coated_retriever', 'dandie_dinmont', 'dhole', 'dingo', 'doberman', 'english_foxhound', 'english_setter', 'english_springer', 'entlebucher', 'eskimo_dog', 'flat-coated_retriever', 'french_bulldog', 'german_shepherd', 'german_short-haired_pointer', 'giant_schnauzer', 'golden_retriever', 'gordon_setter', 'great_dane', 'great_pyrenees', 'greater_swiss_mountain_dog', 'groenendael', 'ibizan_hound', 'irish_setter', 'irish_terrier', 'irish_water_spaniel', 'irish_wolfhound', 'italian_greyhound', 'japanese_spaniel', 'keeshond', 'kelpie', 'kerry_blue_terrier', 'komondor', 'kuvasz', 'labrador_retriever', 'lakeland_terrier', 'leonberg', 'lhasa', 'malamute', 'malinois', 'maltese_dog', 'mexican_hairless', 'miniature_pinscher', 'miniature_poodle', 'miniature_schnauzer', 'newfoundland', 'norfolk_terrier', 'norwegian_elkhound', 'norwich_terrier', 'old_english_sheepdog', 'otterhound', 'papillon', 'pekinese', 'pembroke', 'pomeranian', 'pug', 'redbone', 'rhodesian_ridgeback', 'rottweiler', 'saint_bernard', 'saluki', 'samoyed', 'schipperke', 'scotch_terrier', 'scottish_deerhound', 'sealyham_terrier', 'shetland_sheepdog', 'shih-tzu', 'siberian_husky', 'silky_terrier', 'soft-coated_wheaten_terrier', 'staffordshire_bullterrier', 'standard_poodle', 'standard_schnauzer', 'sussex_spaniel', 'tibetan_mastiff', 'tibetan_terrier', 'toy_poodle', 'toy_terrier', 'vizsla', 'walker_hound', 'weimaraner', 'welsh_springer_spaniel', 'west_highland_white_terrier', 'whippet', 'wire-haired_fox_terrier', 'yorkshire_terrier']
  • 예측을 위해 라벨을 알파벳순으로 정렬함


1.15 예측

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import random
import cv2

image_path = random.choice(valid_generator.filepaths)
real_y = image_path.split('/')[3]
idx = unique_sorted_Y.index(real_y)

img = cv2.imread(image_path)
img = cv2.resize(img, dsize = (299, 299))
img = img / 255.0
img = np.expand_dims(img, axis = 0)

feature_vector = feature_model.predict(img)

prediction = model.predict(feature_vector)[0]

top_5_predict = prediction.argsort()[::-1][:5]
labels = [unique_sorted_Y[index] for index in top_5_predict] 
  • real_y를 구하는 부분에서 해당 경로에 따라 split하여 위치는 다를수 있음


1.16 Top 5예측을 해보자

1
2
print(top_5_predict)
print(labels)
1
2
[ 10  97  68 105 110]
['bedlington_terrier', 'scottish_deerhound', 'lakeland_terrier', 'standard_poodle', 'toy_poodle']
  • 저 5개 중에 하나가 맞을지?


1.17 그래프로 보기

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
plt.figure(figsize = (16,6))

plt.subplot(1,2,1)
plt.imshow(plt.imread(image_path))
plt.title(real_y)
plt.axis('off')

plt.subplot(1,2,2)
color = ['gray'] * 5
if idx in top_5_predict:
    color[top_5_predict.tolist().index(idx)] = 'green'
color = color[::-1]
plt.barh(range(5), prediction[top_5_predict][::-1] * 100, color = color)
plt.yticks(range(5), labels[::-1])
plt.show()

  • 결과가 매우 잘나온다.


2. 요약


2.1 요약

  • 사전에 훈련된 모델에서 특징만 추출하여, 간단하게 작은 네트워크에 통과하도록 하는 방식이 흥미롭다.
  • 결국엔 누군가가 미리 만들어놓은 모델을 이용하여 더 발전 시킬수있고, 또한 이미 훈련이 완료되어있기에 시간도 절약이 된다.
  • 딥러닝은 많은 기술과 공부, 그리고 트랜드를 따라가는 노력이 필요한듯 하다
This post is licensed under CC BY 4.0 by the author.

사전훈련모델과 전이 학습

신경 스타일 전이 (Neural Style Transfer)

Comments powered by Disqus.