고정 헤더 영역
상세 컨텐츠
본문 제목
[코드 분석 스터디] Segmentation : Sementic Segmentation - CARLA Image Road segmentation
본문
참고 커널:
https://www.kaggle.com/wangmo/self-driving-cars-road-segmentation-task
Image Classification과 Object Detection, Image Segmentation의 차이
[Image Classification]
-입력 이미지를 클래스나 레이블로 분류, 할당
[Object Detection]
-이미지 속 여러 물체의 위치를 찾아 boxing하는 detection
[Segmentation]
-pixel-wise classification
-Sementic Segmentation과 Instance Segmentation으로 나뉨
- Semetic Segmentation: 같은 class 물체가 있더라도, 개별 object에 대한 고려 X
- Instance Segmentation: 물체 하나 하나를 개별 object로 분류
U-Net 모델 구조(Encoder-Decoder)
1. Contracting path (Encoder containing downsampling steps)
- Convolutional layer, activation function, pooling layer를 가지는 일반적인 CNN의 형태
- Layer가 깊어질수록 이미지는 downsampling되고, feature가 추출
- height와 width는 줄어들지만 number of channels는 늘어남
2. Expanding path (Decoder containing upsampling steps)
- 1단계의 반대
- 1단계에서 줄어들었던 이미지를 original size로 다시 키운다.
- Channel 수는 줄인다.
- Concatenate를 통해 size를 줄이면서 사라졌던 디테일을 다시 살린다.
3. Final Feature Mapping Block
- 마지막 layer에서 1x1 convolution을 통해 class 수를 mapping하는데 사용된다.
데이터 소개
CARLA Simulator의 이미지 데이터로 dataA, dataB, dataC, dataD, dataE로 나뉘어있다. 각 폴더는 CameraRGB와 CameraSeg 폴더로 구성되어있다. CameraRGB는 1,000개의 시뮬레이터 이미지가 들어있고 CameraSEG는 이에 대응되는 Segmentation용 파일이 있다.
코드 분석
image_path = '../input/dataA/dataA/CameraRGB/'
mask_path = '../input/dataA/dataA/CameraSeg/'
image_list = os.listdir(image_path)
mask_list = os.listdir(mask_path)
image_list = [image_path+i for i in image_list]
mask_list = [mask_path+i for i in mask_list]dataA의 CameraRGB와 CameraSeg를 각각 image_path와 mask_path로 path를 설정하고 불러온다.
image_list.sort()
mask_list.sort()이 코드는 참고한 커널에는 없던 코드로 local에서 수행할 경우 image_list와 mask_list가 순서대로 배열되어 필요 없는 코드다.
하지만 colab에서 진행할 경우 image_list와 mask_list가 순서대로 배열되지 않아 이후 이미지를 확인하는 코드나 학습할 때 제대로 결과가 나오지 않는다. 그렇기에 sort()를 통해 image_list와 mask_list를 맞춰줄 필요가 있다.
위의 두 이미지처럼 sort()를 하였을 때는 Image와 Segmentation이 같은 이미지지만 sort()를 하지 않았을 때는 Image와 Segmentation이 다른 이미지다.
N = 1
img = imageio.imread(image_list[N])
mask = imageio.imread(mask_list[N])
mask = np.array([max(mask[i, j]) for i in range(mask.shape[0]) for j in range(mask.shape[1])]).reshape(img.shape[0], img.shape[1])
fig, arr = plt.subplots(1, 2, figsize=(14, 10))
arr[0].imshow(img)
arr[0].set_title('Image')
arr[1].imshow(mask, cmap='Paired')
arr[1].set_title('Segmentation')
첫번째 Image와 이에 대응되는 Segmentation이 어떻게 나타나는지 보여주는 코드다.
road = np.zeros((600, 800))
road[np.where(mask==7)[0], np.where(mask==7)[1]]=1
plt.imshow(road)
이 Segmentation에서 우리가 segmentation하려는 road는 7이기 떄문에 mask==7을 써준다. mask==뒤에 어떤 숫자를 쓰냐에 따라 노란색으로 나타나는 segment가 달라진다. road에 1을 부여해줘 road부분을 노란색으로 나타내고 다른 부분들을 어두운 색으로 표현하였다.
from tqdm import tqdmheight, width = 600, 800
images = np.zeros((len(image_list), height, width, 3), dtype=np.int16)
masks = np.zeros((len(image_list), height, width, 1), dtype=np.int8)
for n in tqdm(range(len(image_list))):
img = imageio.imread(image_list[n])
mask = imageio.imread(mask_list[n])
mask_road = np.zeros((600, 800, 1), dtype=np.int8)
mask_road[np.where(mask==7)[0], np.where(mask==7)[1]]=1
images[n] = img
masks[n] = mask_roadimage_list와 mask_list에 저장해둔 이미지 데이터의 이름에 맞는 이미지를 읽어온다. tqdm을 통해 얼마나 진행되었는지를 보여준다. mask를 읽어올 때 road만 segmentation해준다.
plt.imshow(images[1].reshape(600, 800, 3))
Image 데이터를 잘 읽어왔는지 plt.imshow()를 통해 확인한다.
np.random.seed(123)
shuffle_ids = np.array([i for i in range(len(masks))])
np.random.shuffle(shuffle_ids)
train_ids = shuffle_ids[:int(len(masks)*0.8)]
val_ids = shuffle_ids[int(len(masks)*0.8):int(len(masks)*0.8+100)]
test_ids = shuffle_ids[int(len(masks)*0.8+100):]train_images, train_masks = images[train_ids], masks[train_ids]
val_images, val_masks = images[val_ids], masks[val_ids]
test_images, test_masks = images[test_ids], masks[test_ids]train_images.shape, val_images.shape, test_images.shape이 데이터가 시뮬레이터 데이터이기 때문에 바로 다음 데이터가 자동차가 조금 움직인 후의 데이터라서 직전 데이터와 조금의 차이만 존재한다. 그렇기 때문에 데이터를 섞지 않을 경우 local하게 학습할 수 있기 때문에 데이터를 섞은 후 train set, validation set, test set을 나누어 준다.
from keras.models import Model, load_model
from keras.layers import Input
from keras.layers.core import Lambda, RepeatVector, Reshape
from keras.layers.convolutional import Conv2D, Conv2DTranspose
from keras.layers.pooling import MaxPooling2D
from keras.layers.merge import concatenate
from keras.callbacks import EarlyStopping, ModelCheckpoint, ReduceLROnPlateau
from keras import backend as K# Build U-Net model
input_img = Input((height, width, 3), name='img')
c1 = Conv2D(8, (3, 3), activation='relu', padding='same') (input_img)
c1 = Conv2D(8, (3, 3), activation='relu', padding='same') (c1)
p1 = MaxPooling2D((2, 2)) (c1)
c2 = Conv2D(16, (3, 3), activation='relu', padding='same') (p1)
c2 = Conv2D(16, (3, 3), activation='relu', padding='same') (c2)
p2 = MaxPooling2D((2, 2)) (c2)
c3 = Conv2D(32, (3, 3), activation='relu', padding='same') (p2)
c3 = Conv2D(32, (3, 3), activation='relu', padding='same') (c3)
p3 = MaxPooling2D((2, 2)) (c3)
c4 = Conv2D(64, (3, 3), activation='relu', padding='same') (p3)
c4 = Conv2D(64, (3, 3), activation='relu', padding='same') (c4)
u5 = Conv2DTranspose(64, (2, 2), strides=(2, 2), padding='same') (c4)
u5 = concatenate([u5, c3])
c6 = Conv2D(32, (3, 3), activation='relu', padding='same') (u5)
c6 = Conv2D(32, (3, 3), activation='relu', padding='same') (c6)
u7 = Conv2DTranspose(32, (2, 2), strides=(2, 2), padding='same') (c6)
u7 = concatenate([u7, c2])
c7 = Conv2D(16, (3, 3), activation='relu', padding='same') (u7)
c7 = Conv2D(16, (3, 3), activation='relu', padding='same') (c7)
u8 = Conv2DTranspose(16, (2, 2), strides=(2, 2), padding='same') (c7)
u8 = concatenate([u8, c1])
c8 = Conv2D(8, (3, 3), activation='relu', padding='same') (u8)
c8 = Conv2D(8, (3, 3), activation='relu', padding='same') (c8)
outputs = Conv2D(1, (1, 1), activation='sigmoid') (c8)
model = Model(inputs=[input_img], outputs=[outputs])
model.compile(optimizer='adam', loss='binary_crossentropy')
model.summary()Model: "model"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
img (InputLayer) [(None, 600, 800, 3) 0
__________________________________________________________________________________________________
conv2d (Conv2D) (None, 600, 800, 8) 224 img[0][0]
__________________________________________________________________________________________________
conv2d_1 (Conv2D) (None, 600, 800, 8) 584 conv2d[0][0]
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 300, 400, 8) 0 conv2d_1[0][0]
__________________________________________________________________________________________________
conv2d_2 (Conv2D) (None, 300, 400, 16) 1168 max_pooling2d[0][0]
__________________________________________________________________________________________________
conv2d_3 (Conv2D) (None, 300, 400, 16) 2320 conv2d_2[0][0]
__________________________________________________________________________________________________
max_pooling2d_1 (MaxPooling2D) (None, 150, 200, 16) 0 conv2d_3[0][0]
__________________________________________________________________________________________________
conv2d_4 (Conv2D) (None, 150, 200, 32) 4640 max_pooling2d_1[0][0]
__________________________________________________________________________________________________
conv2d_5 (Conv2D) (None, 150, 200, 32) 9248 conv2d_4[0][0]
__________________________________________________________________________________________________
max_pooling2d_2 (MaxPooling2D) (None, 75, 100, 32) 0 conv2d_5[0][0]
__________________________________________________________________________________________________
conv2d_6 (Conv2D) (None, 75, 100, 64) 18496 max_pooling2d_2[0][0]
__________________________________________________________________________________________________
conv2d_7 (Conv2D) (None, 75, 100, 64) 36928 conv2d_6[0][0]
__________________________________________________________________________________________________
conv2d_transpose (Conv2DTranspo (None, 150, 200, 64) 16448 conv2d_7[0][0]
__________________________________________________________________________________________________
concatenate (Concatenate) (None, 150, 200, 96) 0 conv2d_transpose[0][0]
conv2d_5[0][0]
__________________________________________________________________________________________________
conv2d_8 (Conv2D) (None, 150, 200, 32) 27680 concatenate[0][0]
__________________________________________________________________________________________________
conv2d_9 (Conv2D) (None, 150, 200, 32) 9248 conv2d_8[0][0]
__________________________________________________________________________________________________
conv2d_transpose_1 (Conv2DTrans (None, 300, 400, 32) 4128 conv2d_9[0][0]
__________________________________________________________________________________________________
concatenate_1 (Concatenate) (None, 300, 400, 48) 0 conv2d_transpose_1[0][0]
conv2d_3[0][0]
__________________________________________________________________________________________________
conv2d_10 (Conv2D) (None, 300, 400, 16) 6928 concatenate_1[0][0]
__________________________________________________________________________________________________
conv2d_11 (Conv2D) (None, 300, 400, 16) 2320 conv2d_10[0][0]
__________________________________________________________________________________________________
conv2d_transpose_2 (Conv2DTrans (None, 600, 800, 16) 1040 conv2d_11[0][0]
__________________________________________________________________________________________________
concatenate_2 (Concatenate) (None, 600, 800, 24) 0 conv2d_transpose_2[0][0]
conv2d_1[0][0]
__________________________________________________________________________________________________
conv2d_12 (Conv2D) (None, 600, 800, 8) 1736 concatenate_2[0][0]
__________________________________________________________________________________________________
conv2d_13 (Conv2D) (None, 600, 800, 8) 584 conv2d_12[0][0]
__________________________________________________________________________________________________
conv2d_14 (Conv2D) (None, 600, 800, 1) 9 conv2d_13[0][0]
==================================================================================================
Total params: 143,729
Trainable params: 143,729
Non-trainable params: 0
__________________________________________________________________________________________________Transposed Convolution: Input 이미지 크기의 이미지를 얻기 위해 Convolution 과정을 반대로 수행한다.(Transposed Convolution output의 size=Convolution input의 size) (but, Transposed Convolution output 값과 Convolution input 값이 다를 수 있다.)
padding=’same’일 때 Output height=Input height*stride, Output width=Input width*stride다.
예를 들어 u5 = Conv2DTranspose(64, (2, 2), strides=(2, 2), padding='same') (c4)일 때, Input size는 C4의 output size인 (75,100,64)고, stride=(2,2)다. 위 식에 대입하면 75*2=150, 100*2=200로 Transpose 후 output size는 (150,200,64)다.
Concatenate: Convolution을 많이 진행하면 이미지 size가 작아져서 하나의 이미지에 많은 데이터가 들어있다. 데이터가 디테일이 부족하다(데이터가 뭉개져있다).
이미지에 더 많은 데이터를 담기 위해 같은 사이즈의 2개의 이미지를 concatenate한다. size는 유지되고, channel 수는 concatenate한 두 개의 합이다.
예를 들어 u5 = concatenate([u5, c3])를 했을 때 u5=(150,200,64), c3=(150,200,32)이면, concatenate([u5, c3])=(150,200,96)다.
callbacks = [
EarlyStopping(patience=12, verbose=1),
ReduceLROnPlateau(patience=3, verbose=1),
ModelCheckpoint('model-sdc-seg-v2.h5', verbose=1, save_best_only=True)]
results = model.fit(train_images, train_masks, batch_size=16, epochs=100, callbacks=callbacks,
validation_data=(val_images, val_masks))Epoch 1/100
50/50 [==============================] - 106s 1s/step - loss: 0.6687 - val_loss: 0.3795
Epoch 00001: val_loss improved from inf to 0.37952, saving model to model-sdc-seg-v2.h5
Epoch 2/100
50/50 [==============================] - 63s 1s/step - loss: 0.4001 - val_loss: 0.3468
Epoch 00002: val_loss improved from 0.37952 to 0.34679, saving model to model-sdc-seg-v2.h5
Epoch 3/100
50/50 [==============================] - 63s 1s/step - loss: 0.3820 - val_loss: 0.3331
Epoch 00003: val_loss improved from 0.34679 to 0.33310, saving model to model-sdc-seg-v2.h5
Epoch 4/100
50/50 [==============================] - 63s 1s/step - loss: 0.3725 - val_loss: 0.3084
Epoch 00004: val_loss improved from 0.33310 to 0.30843, saving model to model-sdc-seg-v2.h5
Epoch 5/100
50/50 [==============================] - 63s 1s/step - loss: 0.3566 - val_loss: 0.3266
Epoch 00005: val_loss did not improve from 0.30843
Epoch 6/100
50/50 [==============================] - 63s 1s/step - loss: 0.3150 - val_loss: 0.3464
Epoch 00006: val_loss did not improve from 0.30843
Epoch 7/100
50/50 [==============================] - 63s 1s/step - loss: 0.3240 - val_loss: 0.2882
Epoch 00007: val_loss improved from 0.30843 to 0.28818, saving model to model-sdc-seg-v2.h5
Epoch 8/100
50/50 [==============================] - 63s 1s/step - loss: 0.2920 - val_loss: 0.2670
Epoch 00008: val_loss improved from 0.28818 to 0.26698, saving model to model-sdc-seg-v2.h5
Epoch 9/100
50/50 [==============================] - 63s 1s/step - loss: 0.2765 - val_loss: 0.2508
Epoch 00009: val_loss improved from 0.26698 to 0.25082, saving model to model-sdc-seg-v2.h5
Epoch 10/100
50/50 [==============================] - 63s 1s/step - loss: 0.2331 - val_loss: 0.2302
Epoch 00010: val_loss improved from 0.25082 to 0.23016, saving model to model-sdc-seg-v2.h5
Epoch 11/100
50/50 [==============================] - 63s 1s/step - loss: 0.2254 - val_loss: 0.1592
Epoch 00011: val_loss improved from 0.23016 to 0.15921, saving model to model-sdc-seg-v2.h5
Epoch 12/100
50/50 [==============================] - 63s 1s/step - loss: 0.1859 - val_loss: 0.1425
Epoch 00012: val_loss improved from 0.15921 to 0.14249, saving model to model-sdc-seg-v2.h5
Epoch 13/100
50/50 [==============================] - 63s 1s/step - loss: 0.1564 - val_loss: 0.1421
Epoch 00013: val_loss improved from 0.14249 to 0.14208, saving model to model-sdc-seg-v2.h5
Epoch 14/100
50/50 [==============================] - 63s 1s/step - loss: 0.1328 - val_loss: 0.3322
Epoch 00014: val_loss did not improve from 0.14208
Epoch 15/100
50/50 [==============================] - 63s 1s/step - loss: 0.1838 - val_loss: 0.1185
Epoch 00015: val_loss improved from 0.14208 to 0.11849, saving model to model-sdc-seg-v2.h5
Epoch 16/100
50/50 [==============================] - 63s 1s/step - loss: 0.1254 - val_loss: 0.1029
Epoch 00016: val_loss improved from 0.11849 to 0.10286, saving model to model-sdc-seg-v2.h5
Epoch 17/100
50/50 [==============================] - 63s 1s/step - loss: 0.1017 - val_loss: 0.0828
Epoch 00017: val_loss improved from 0.10286 to 0.08276, saving model to model-sdc-seg-v2.h5
Epoch 18/100
50/50 [==============================] - 63s 1s/step - loss: 0.0894 - val_loss: 0.0720
Epoch 00018: val_loss improved from 0.08276 to 0.07201, saving model to model-sdc-seg-v2.h5
Epoch 19/100
50/50 [==============================] - 62s 1s/step - loss: 0.1218 - val_loss: 0.1446
Epoch 00019: val_loss did not improve from 0.07201
Epoch 20/100
50/50 [==============================] - 63s 1s/step - loss: 0.1152 - val_loss: 0.0810
Epoch 00020: val_loss did not improve from 0.07201
Epoch 21/100
50/50 [==============================] - 63s 1s/step - loss: 0.1120 - val_loss: 0.0754
Epoch 00021: ReduceLROnPlateau reducing learning rate to 0.00010000000474974513.
Epoch 00021: val_loss did not improve from 0.07201
Epoch 22/100
50/50 [==============================] - 63s 1s/step - loss: 0.0784 - val_loss: 0.0668
Epoch 00022: val_loss improved from 0.07201 to 0.06682, saving model to model-sdc-seg-v2.h5
Epoch 23/100
50/50 [==============================] - 63s 1s/step - loss: 0.0747 - val_loss: 0.0651
Epoch 00023: val_loss improved from 0.06682 to 0.06513, saving model to model-sdc-seg-v2.h5
Epoch 24/100
50/50 [==============================] - 63s 1s/step - loss: 0.0726 - val_loss: 0.0634
Epoch 00024: val_loss improved from 0.06513 to 0.06338, saving model to model-sdc-seg-v2.h5
Epoch 25/100
50/50 [==============================] - 63s 1s/step - loss: 0.0712 - val_loss: 0.0637
Epoch 00025: val_loss did not improve from 0.06338
Epoch 26/100
50/50 [==============================] - 63s 1s/step - loss: 0.0695 - val_loss: 0.0623
Epoch 00026: val_loss improved from 0.06338 to 0.06232, saving model to model-sdc-seg-v2.h5
Epoch 27/100
50/50 [==============================] - 63s 1s/step - loss: 0.0685 - val_loss: 0.0604
Epoch 00027: val_loss improved from 0.06232 to 0.06037, saving model to model-sdc-seg-v2.h5
Epoch 28/100
50/50 [==============================] - 63s 1s/step - loss: 0.0673 - val_loss: 0.0608
Epoch 00028: val_loss did not improve from 0.06037
Epoch 29/100
50/50 [==============================] - 63s 1s/step - loss: 0.0671 - val_loss: 0.0582
Epoch 00029: val_loss improved from 0.06037 to 0.05822, saving model to model-sdc-seg-v2.h5
Epoch 30/100
50/50 [==============================] - 63s 1s/step - loss: 0.0653 - val_loss: 0.0576
Epoch 00030: val_loss improved from 0.05822 to 0.05763, saving model to model-sdc-seg-v2.h5
Epoch 31/100
50/50 [==============================] - 62s 1s/step - loss: 0.0645 - val_loss: 0.0563
Epoch 00031: val_loss improved from 0.05763 to 0.05634, saving model to model-sdc-seg-v2.h5
Epoch 32/100
50/50 [==============================] - 62s 1s/step - loss: 0.0631 - val_loss: 0.0564
Epoch 00032: val_loss did not improve from 0.05634
Epoch 33/100
50/50 [==============================] - 62s 1s/step - loss: 0.0623 - val_loss: 0.0566
Epoch 00033: val_loss did not improve from 0.05634
Epoch 34/100
50/50 [==============================] - 62s 1s/step - loss: 0.0617 - val_loss: 0.0547
Epoch 00034: val_loss improved from 0.05634 to 0.05468, saving model to model-sdc-seg-v2.h5
Epoch 35/100
50/50 [==============================] - 62s 1s/step - loss: 0.0610 - val_loss: 0.0545
Epoch 00035: val_loss improved from 0.05468 to 0.05449, saving model to model-sdc-seg-v2.h5
Epoch 36/100
50/50 [==============================] - 62s 1s/step - loss: 0.0597 - val_loss: 0.0536
Epoch 00036: val_loss improved from 0.05449 to 0.05357, saving model to model-sdc-seg-v2.h5
Epoch 37/100
50/50 [==============================] - 62s 1s/step - loss: 0.0592 - val_loss: 0.0528
Epoch 00037: val_loss improved from 0.05357 to 0.05282, saving model to model-sdc-seg-v2.h5
Epoch 38/100
50/50 [==============================] - 62s 1s/step - loss: 0.0585 - val_loss: 0.0533
Epoch 00038: val_loss did not improve from 0.05282
Epoch 39/100
50/50 [==============================] - 62s 1s/step - loss: 0.0582 - val_loss: 0.0508
Epoch 00039: val_loss improved from 0.05282 to 0.05076, saving model to model-sdc-seg-v2.h5
Epoch 40/100
50/50 [==============================] - 62s 1s/step - loss: 0.0573 - val_loss: 0.0534
Epoch 00040: val_loss did not improve from 0.05076
Epoch 41/100
50/50 [==============================] - 62s 1s/step - loss: 0.0566 - val_loss: 0.0505
Epoch 00041: val_loss improved from 0.05076 to 0.05054, saving model to model-sdc-seg-v2.h5
Epoch 42/100
50/50 [==============================] - 62s 1s/step - loss: 0.0553 - val_loss: 0.0510
Epoch 00042: val_loss did not improve from 0.05054
Epoch 43/100
50/50 [==============================] - 62s 1s/step - loss: 0.0551 - val_loss: 0.0499
Epoch 00043: val_loss improved from 0.05054 to 0.04987, saving model to model-sdc-seg-v2.h5
Epoch 44/100
50/50 [==============================] - 62s 1s/step - loss: 0.0539 - val_loss: 0.0493
Epoch 00044: val_loss improved from 0.04987 to 0.04926, saving model to model-sdc-seg-v2.h5
Epoch 45/100
50/50 [==============================] - 62s 1s/step - loss: 0.0540 - val_loss: 0.0486
Epoch 00045: val_loss improved from 0.04926 to 0.04857, saving model to model-sdc-seg-v2.h5
Epoch 46/100
50/50 [==============================] - 62s 1s/step - loss: 0.0527 - val_loss: 0.0491
Epoch 00046: val_loss did not improve from 0.04857
Epoch 47/100
50/50 [==============================] - 62s 1s/step - loss: 0.0523 - val_loss: 0.0478
Epoch 00047: val_loss improved from 0.04857 to 0.04779, saving model to model-sdc-seg-v2.h5
Epoch 48/100
50/50 [==============================] - 62s 1s/step - loss: 0.0521 - val_loss: 0.0473
Epoch 00048: val_loss improved from 0.04779 to 0.04727, saving model to model-sdc-seg-v2.h5
Epoch 49/100
50/50 [==============================] - 62s 1s/step - loss: 0.0511 - val_loss: 0.0462
Epoch 00049: val_loss improved from 0.04727 to 0.04618, saving model to model-sdc-seg-v2.h5
Epoch 50/100
50/50 [==============================] - 62s 1s/step - loss: 0.0509 - val_loss: 0.0497
Epoch 00050: val_loss did not improve from 0.04618
Epoch 51/100
50/50 [==============================] - 62s 1s/step - loss: 0.0508 - val_loss: 0.0450
Epoch 00051: val_loss improved from 0.04618 to 0.04496, saving model to model-sdc-seg-v2.h5
Epoch 52/100
50/50 [==============================] - 62s 1s/step - loss: 0.0497 - val_loss: 0.0443
Epoch 00052: val_loss improved from 0.04496 to 0.04427, saving model to model-sdc-seg-v2.h5
Epoch 53/100
50/50 [==============================] - 62s 1s/step - loss: 0.0490 - val_loss: 0.0452
Epoch 00053: val_loss did not improve from 0.04427
Epoch 54/100
50/50 [==============================] - 62s 1s/step - loss: 0.0486 - val_loss: 0.0440
Epoch 00054: val_loss improved from 0.04427 to 0.04400, saving model to model-sdc-seg-v2.h5
Epoch 55/100
50/50 [==============================] - 62s 1s/step - loss: 0.0478 - val_loss: 0.0437
Epoch 00055: val_loss improved from 0.04400 to 0.04366, saving model to model-sdc-seg-v2.h5
Epoch 56/100
50/50 [==============================] - 62s 1s/step - loss: 0.0476 - val_loss: 0.0445
Epoch 00056: val_loss did not improve from 0.04366
Epoch 57/100
50/50 [==============================] - 62s 1s/step - loss: 0.0471 - val_loss: 0.0420
Epoch 00057: val_loss improved from 0.04366 to 0.04202, saving model to model-sdc-seg-v2.h5
Epoch 58/100
50/50 [==============================] - 62s 1s/step - loss: 0.0463 - val_loss: 0.0423
Epoch 00058: val_loss did not improve from 0.04202
Epoch 59/100
50/50 [==============================] - 62s 1s/step - loss: 0.0472 - val_loss: 0.0420
Epoch 00059: val_loss improved from 0.04202 to 0.04198, saving model to model-sdc-seg-v2.h5
Epoch 60/100
50/50 [==============================] - 62s 1s/step - loss: 0.0458 - val_loss: 0.0429
Epoch 00060: ReduceLROnPlateau reducing learning rate to 1.0000000474974514e-05.
Epoch 00060: val_loss did not improve from 0.04198
Epoch 61/100
50/50 [==============================] - 62s 1s/step - loss: 0.0445 - val_loss: 0.0415
Epoch 00061: val_loss improved from 0.04198 to 0.04149, saving model to model-sdc-seg-v2.h5
Epoch 62/100
50/50 [==============================] - 62s 1s/step - loss: 0.0440 - val_loss: 0.0412
Epoch 00062: val_loss improved from 0.04149 to 0.04121, saving model to model-sdc-seg-v2.h5
Epoch 63/100
50/50 [==============================] - 62s 1s/step - loss: 0.0439 - val_loss: 0.0412
Epoch 00063: val_loss did not improve from 0.04121
Epoch 64/100
50/50 [==============================] - 62s 1s/step - loss: 0.0439 - val_loss: 0.0410
Epoch 00064: val_loss improved from 0.04121 to 0.04102, saving model to model-sdc-seg-v2.h5
Epoch 65/100
50/50 [==============================] - 62s 1s/step - loss: 0.0438 - val_loss: 0.0410
Epoch 00065: val_loss improved from 0.04102 to 0.04100, saving model to model-sdc-seg-v2.h5
Epoch 66/100
50/50 [==============================] - 62s 1s/step - loss: 0.0437 - val_loss: 0.0412
Epoch 00066: val_loss did not improve from 0.04100
Epoch 67/100
50/50 [==============================] - 62s 1s/step - loss: 0.0437 - val_loss: 0.0409
Epoch 00067: ReduceLROnPlateau reducing learning rate to 1.0000000656873453e-06.
Epoch 00067: val_loss improved from 0.04100 to 0.04093, saving model to model-sdc-seg-v2.h5
Epoch 68/100
50/50 [==============================] - 62s 1s/step - loss: 0.0436 - val_loss: 0.0409
Epoch 00068: val_loss improved from 0.04093 to 0.04090, saving model to model-sdc-seg-v2.h5
Epoch 69/100
50/50 [==============================] - 62s 1s/step - loss: 0.0435 - val_loss: 0.0409
Epoch 00069: val_loss improved from 0.04090 to 0.04087, saving model to model-sdc-seg-v2.h5
Epoch 70/100
50/50 [==============================] - 62s 1s/step - loss: 0.0435 - val_loss: 0.0409
Epoch 00070: val_loss improved from 0.04087 to 0.04086, saving model to model-sdc-seg-v2.h5
Epoch 71/100
50/50 [==============================] - 62s 1s/step - loss: 0.0435 - val_loss: 0.0409
Epoch 00071: ReduceLROnPlateau reducing learning rate to 1.0000001111620805e-07.
Epoch 00071: val_loss improved from 0.04086 to 0.04085, saving model to model-sdc-seg-v2.h5
Epoch 72/100
50/50 [==============================] - 62s 1s/step - loss: 0.0435 - val_loss: 0.0409
Epoch 00072: val_loss improved from 0.04085 to 0.04085, saving model to model-sdc-seg-v2.h5
Epoch 73/100
50/50 [==============================] - 62s 1s/step - loss: 0.0435 - val_loss: 0.0409
Epoch 00073: val_loss did not improve from 0.04085
Epoch 74/100
50/50 [==============================] - 62s 1s/step - loss: 0.0435 - val_loss: 0.0409
Epoch 00074: ReduceLROnPlateau reducing learning rate to 1.000000082740371e-08.
Epoch 00074: val_loss did not improve from 0.04085
Epoch 75/100
50/50 [==============================] - 62s 1s/step - loss: 0.0435 - val_loss: 0.0409
Epoch 00075: val_loss did not improve from 0.04085
Epoch 76/100
50/50 [==============================] - 62s 1s/step - loss: 0.0435 - val_loss: 0.0409
Epoch 00076: val_loss did not improve from 0.04085
Epoch 77/100
50/50 [==============================] - 63s 1s/step - loss: 0.0435 - val_loss: 0.0409
Epoch 00077: ReduceLROnPlateau reducing learning rate to 1.000000082740371e-09.
Epoch 00077: val_loss did not improve from 0.04085
Epoch 78/100
50/50 [==============================] - 63s 1s/step - loss: 0.0435 - val_loss: 0.0409
Epoch 00078: val_loss did not improve from 0.04085
Epoch 79/100
50/50 [==============================] - 63s 1s/step - loss: 0.0435 - val_loss: 0.0409
Epoch 00079: val_loss did not improve from 0.04085
Epoch 80/100
50/50 [==============================] - 63s 1s/step - loss: 0.0435 - val_loss: 0.0409
Epoch 00080: ReduceLROnPlateau reducing learning rate to 1.000000082740371e-10.
Epoch 00080: val_loss did not improve from 0.04085
Epoch 81/100
50/50 [==============================] - 63s 1s/step - loss: 0.0435 - val_loss: 0.0409
Epoch 00081: val_loss did not improve from 0.04085
Epoch 82/100
50/50 [==============================] - 63s 1s/step - loss: 0.0435 - val_loss: 0.0409
Epoch 00082: val_loss did not improve from 0.04085
Epoch 83/100
50/50 [==============================] - 63s 1s/step - loss: 0.0435 - val_loss: 0.0409
Epoch 00083: ReduceLROnPlateau reducing learning rate to 1.000000082740371e-11.
Epoch 00083: val_loss did not improve from 0.04085
Epoch 84/100
50/50 [==============================] - 63s 1s/step - loss: 0.0435 - val_loss: 0.0409
Epoch 00084: val_loss did not improve from 0.04085
Epoch 00084: early stoppingEarlyStopping: 12번의 epoch동안 모델이 improvement 없을 때 stop하게 한다.
ReduceLROnPlateau: 3번의 epoch동안 모델 improvement가 없을 경우, learning rate를 조절해 모델의 개선을 유도한다.
from matplotlib import pyplot
pyplot.plot(results.history['loss'], label='train')
pyplot.plot(results.history['val_loss'], label='test')
pyplot.legend()
pyplot.show()
results.history를 통해 epoch이 거듭되면서 loss와 val_loss의 변화를 확인한다.
model.save('final-road-seg-model-v2.h5')NUMBER = 0
my_preds = model.predict(np.expand_dims(test_images[NUMBER], 0))
my_preds = my_preds.flatten()
my_preds = np.array([1 if i >= 0.5 else 0 for i in my_preds])
fig, ax = plt.subplots(nrows=1, ncols=2)
ax[0].imshow(my_preds.reshape(600, 800))
ax[0].set_title('Prediction')
ax[1].imshow(test_masks[NUMBER].reshape(600, 800))
ax[1].set_title('Ground truth')
NUMBER += 1
my_preds = model.predict(np.expand_dims(test_images[NUMBER], 0))
my_preds = my_preds.flatten()
my_preds = np.array([1 if i >= 0.5 else 0 for i in my_preds])
fig, ax = plt.subplots(nrows=1, ncols=2)
ax[0].imshow(my_preds.reshape(600, 800))
ax[0].set_title('Prediction')
ax[1].imshow(test_masks[NUMBER].reshape(600, 800))
ax[1].set_title('Ground truth')
위 두 코드를 통해 prediction한 것과 실제 road를 나타낸 것을 비교한다. 모델의 마지막에 sigmoid를 통해 나왔기 때문에 pixel마다 0과 1 사이 값이 나온다. my_preds를 통해 prediction을 나타내는데 if문을 사용해 i>=0.5를 1로, else를 0으로 바꾼다. 1이 segmentation하는 road를 나타내고, 0은 다른 부분들을 나타낸다.
댓글 영역