Lesson2: The Machine Learning Workflow 마무리

1.  Classification Metrics

 

2. Cross Validation

- Bias & Variance Trade-off

 

- Cross Validation

① Holdout method

② K-fold cross validation

③ Leave-one-out validation

 

Q. Train/Validation set 나누기 전에 EDA를 해야하는 이유는?

1. The validation set might be very similar to the training set, in which case the test error is going to be biased.

2. The class distribution of a random split might be fairly different between training and validation.  

 

 

3. Model Selection

※ Keep in mind that the validation set should remain the same during this phase! 

 

 

4. Error Analysis

- Loss & Accuracy

- Sorting images based on loss

- Find patterns in error

- Understand metrics value

 

 

5. ML Workflow

1. Frame the problem

: understand the stakes, and define relevant metrics

 

2. Understand the data

: perform an EDA and extract patterns from the dataset

 

3. Iterate on the model

: create a validation set, set up baselines, and iterate on models from simper to more complex

 

 

6. Glossary

 출처: 유다시티 (udacity.com)