5.1 Disturbance Test
Table 1. Disturbance Test Output Terms
- Setting: A group of 2468 PNG mages with 224 * 224 size, white background color, dark body color.
- Transform: Transform images to RGB and then to tensor format.
- Disturbance: Randomly selecting n pixels to transform to all-white pixels, which is (
2.2489,2.4286, 2.6400) after normalization.
| Disturbance Test | Training Config | No Disturbance Stable. | Disturb 5 pixels (10/5 instances) | Extraction | Disturb 10 pixels (10/5 instances) | Extraction | Disturb 15 pixels (10/5 instances) | Extraction | Disturb 20 pixels (5 instances) | Extraction |
|---|---|---|---|---|---|---|---|---|---|---|
| MVCNN | Epoch-1-Shade | 0.90640193 | 0.8995138 0.899919 0.899919 0.8958671 0.897893 0.8982982 0.899919 0.8995138 0.8970826 0.8970826 | Max. 0.899919 Min. 0.8958671 Avg. 0.89850081 | 0.8873582 0.8881686 0.8885737 0.8881686 0.8885737 0.886953 0.8885737 0.89100486 0.8877634 0.8881686 | Max. 0.89100486 Min. 0.886953 Avg. 0.888330636 | 0.87236625 0.8735818 0.8768233 0.873987 0.8756078 0.8756078 0.87317663 0.8735818 0.8768233 0.8756078 | Max. 0.8768233 Min. 0.87236625 Avg. 0.874716348 | 0.8626418 0.86102104 0.86142623 0.8634522 0.86061585 | Max. 0.8634522 Min. 0.86061585 Avg. 0.861831424 |
| Epoch-1-NoShade | 0.89465153 | 0.88290113 0.88492703 0.88209075 0.8857374 0.8893841 | Max. 0.8893841 Min. 0.88209075 Avg. 0.885008082 | 0.86952996 0.8658833 0.87277144 0.87034035 0.86709887 | Max. 0.87277144 Min. 0.8658833 Avg. 0.869124784 | 0.8395462 0.8419773 0.8383306 0.8387358 0.8387358 | Max. 0.8419773 Min. 0.8383306 Avg. 0.83946514 | 0.8095624 0.8034846 0.8030794 0.80145866 0.8063209 | Max. 0.8095624 Min. 0.80145866 Avg. 0.804781232 | |
| Epoch-10-Shade | 0.9185575 | 0.9003241 0.90559155 0.90275526 0.90275526 0.90316045 | Max. 0.90559155 Min. 0.9003241 Avg. 0.902917324 | 0.89465153 0.8926256 0.89222044 0.89424634 0.8950567 | Max. 0.8950567 Min. 0.89222044 Avg. 0.893760122 | 0.88209075 0.88492703 0.8837115 0.88249594 0.8841167 | Max. 0.88492703 Min. 0.88209075 Avg. 0.883468384 | 0.87277144 0.8752026 0.8764181 0.87317663 0.8752026 | Max. 0.8764181 Min. 0.87277144 Avg. 0.874554274 | |
| Epoch-10-NoShade | 0.921799 | 0.9076175 0.90518636 0.90518636 0.90478116 0.90316045 | Max. 0.9076175 Min. 0.90316045 Avg. 0.905187086 | 0.89141005 0.8873582 0.8901945 0.8881686 0.8877634 | Max. 0.89141005 Min. 0.8873582 Avg. 0.88897895 | 0.8650729 0.8666937 0.8658833 0.86709887 0.87115073 | Max. 0.87115073 Min. 0.8658833 Avg. 0.8671799 | 0.8517018 0.84846026 0.84886545 0.84764993 0.84643435 | Max. 0.8517018 Min. 0.84643435 Avg. 0.848622358 | |
| View-GCN | Epoch-1-Shade | 0.8188817 | 0.81037277 0.81158835 0.8091572 0.8071313 0.8083468 | Max. 0.81158835 Min. 0.8071313 Avg. 0.809319284 | 0.77957857 0.78038895 0.7832253 0.78119934 0.77957857 | Max. 0.7832253 Min. 0.77957857 Avg. 0.780794146 | 0.75405186 0.75972444 0.7589141 0.75648296 0.75526744 | Max. 0.75972444 Min. 0.75405186 Avg. 0.75688816 | 0.73338735 0.7386548 0.73662883 0.7350081 0.7378444 | Max. 0.7386548 Min. 0.73338735 Avg. 0.736304696 |
| Epoch-1-NoShade | 0.8091572 | 0.8067261 0.81077796 0.80429494 0.8071313 0.80186385 | Max. 0.81077796 Min. 0.80429494 Avg. 0.80615883 | 0.7888979 0.7856564 0.7888979 0.7876823 0.7832253 | Max. 0.7888979 Min. 0.7832253 Avg. 0.78687196 | 0.7581037 0.75729334 0.7593193 0.75729334 0.76012963 | Max. 0.76012963 Min. 0.75729334 Avg. 0.758427862 | 0.7285251 0.7281199 0.73095626 0.7313614 0.73257697 | Max. 0.73257697 Min. 0.7281199 Avg. 0.730307926 | |
| Epoch-10-Shade | 0.91531605 | 0.90599674 0.9076175 0.9092382 0.90599674 0.904376 | Max. 0.9092382 Min. 0.904376 Avg. 0.906645036 | 0.8950567 0.89424634 0.89384115 0.89100486 0.89384115 | Max. 0.8950567 Min. 0.89100486 Avg. 0.89359804 | 0.8808752 0.8853322 0.8841167 0.88290113 0.88006485 | Max. 0.8853322 Min. 0.88006485 Avg. 0.882658016 | 0.8764181 0.88168555 0.87925446 0.87317663 0.8776337 | Max. 0.88168555 Min. 0.87317663 Avg. 0.877633688 | |
| Epoch-10-NoShade | 0.92585087 | 0.91815233 0.91693676 0.91815233 0.9205834 0.921799 | Max. 0.921799 Min. 0.91693676 Avg. 0.919124744 | 0.9132901 0.9132901 0.9128849 0.9092382 0.9132901 | Max. 0.9132901 Min. 0.9092382 Avg. 0.91239868 | 0.90478116 0.9080227 0.90518636 0.9080227 0.9068071 | Max. 0.9080227 Min. 0.90478116 Avg. 0.906564004 | 0.90275526 0.897893 0.8995138 0.9015397 0.8995138 | Max. 0.90275526 Min. 0.897893 Avg. 0.900243112 |
Remark:
Table 2. Fluctuation
| Training Config | No Disturbance | Disturb 5 Avg. | Disturb 10 | Disturb 15 | Disturb 20 | |
|---|---|---|---|---|---|---|
| MVCNN | Ep1-S | 90.64% | - 0.79% | - 1.8% | - 3.2% | - 4.46% |
| Ep1-NoS | 89.47% | - 0.96% | - 2.55% | - 5.52% | - 8.99% | |
| Ep10-S | 91.86% | - 1.56% | - 2.48% | - 3.5% | - 4.4% | |
| Ep10-NoS | 92.18% | - 1.66% | - 3.28% | - 5.46% | - 7.32% | |
| View-GCN | Ep1-S | 81.89% | - 0.96% | - 3.8% | - 6.2% | - 8.26% |
| Ep1-NoS | 80.92% | - 0.3% | - 2.23% | - 5.07% | - 7.88% | |
| Ep10-S | 91.53% | - 0.87% | - 2.17% | - 3.27% | - 3.77% | |
| Ep10-NoS | 92.59% | - 0.67% | - 1.35% | - 1.93% | - 2.56% |
Remark:
For MVCNN, the shade/noshade influence more, getting 2 times gap in self-comparison.
For View-GCN, the gap presents in the training epochs. Training more makes it more stable.
In conclusion, View-GCN seems more stable than MVCNN.