# CHO-FAM Hackathon Kit — 설계·실행·결과

실험일: 2026-10-10. 공식 합성 항공 데이터 S6E10의 workflow 검증이며 제조 성능을 입증하지 않습니다.

Applied holdout ROC-AUC 0.9580040528901356; ideal MCC 0.847976005813268.

분할: stratified seed 20261010, 60/20/20. 후보 비교 16.354초. 보고 모듈 74개 테스트 통과. 브라우저 pixel QA, L1, 재사용률, 대회 사용 규정 검증 미완료. 재학습/holdout 재평가 없음.

Reference는 동결 후 읽었고 실행하거나 새 feature를 적용하지 않았습니다. Personal Travel Recall 0.345622, Business travel 0.939961.



## 실행 개요

처음 보는 문제에서 문제 정의·프로파일링·모델 비교·검증·보고 workflow를 재사용할 수 있는지 확인하는 실험입니다. 진행 중인 S6E10으로 전환했으며 Bosch는 보류했습니다.

합성 항공 데이터의 workflow 검증입니다. 제조 성능·생산 환경 일반화·Kaggle 제출·70% 재사용 달성을 주장하지 않습니다.

## 설계와 판단 근거

ProblemSpec·DataProfile → deterministic SolutionPlanner → Blueprint → Strategy를 실험 전에 동결했습니다. 아래 그래프는 설계이며 개별 구성요소의 실행 완료를 뜻하지 않습니다.

예상 40분은 설계 구성요소 비용입니다. 실측 fitting 시간과 작업 범위가 다릅니다. 공식 ROC-AUC는 threshold와 무관하며 ideal threshold는 별도의 MCC 최적화입니다.

## 실행과 동결

stratified seed 20261010, 60/20/20 분할: train 419,781 / validation 139,927 / holdout 139,927. train-only 전처리를 적용했고 모델·threshold는 validation에서 선택해 holdout 전에 동결했습니다. test는 선택에 사용하지 않았습니다.

Baseline 동결 뒤 후보 3개를 비교했습니다. 총 fit ceiling 4개, 후보 wall budget 1,200초에서 실제 비교는 16.354초였습니다. 공개 reference는 Baseline·Applied·holdout 동결 후 읽었습니다.

Blueprint required_guards가 비어 있는 경고를 유지합니다. adapter는 전처리 선언만으로 실행 검사를 인증하지 않습니다.

## 데이터 품질과 자원

공식 CSV 3개, 102,541,836 bytes. 17 numeric·4 categorical predictor, ID 제외. train 양성 310,339 / 음성 389,296 (양성 44.36%). ID 중복·공백·train/test 겹침 0개, test/sample ID 일치.

Arrival Delay in Minutes만 결측(train 292 / test 130). Profile 4.079초, peak RSS 474,181,632 bytes; 후보 최대 peak RSS 782,974,976 bytes. 원본 해시가 유지됐습니다.

## Baseline → Applied 결과

Centroid Baseline은 numeric-only이고 후보에는 제공된 categorical 열도 포함했습니다. 파생 feature·외부 데이터 없이 선형·tree·gradient boosting을 비교해 validation ROC-AUC로 HistGradientBoosting을 선택했습니다.

- baseline

- histgb

- logistic

- tree

LogisticRegression·DecisionTree holdout은 미평가이며 0이 아닙니다. Baseline probability는 calibration되지 않은 centroid score입니다.

## 세 운영점과 제약

Pessimistic: 최소 FP, validation Recall ≥ 10%. Optimistic: 최대 Recall, validation FPR ≤ 5%. Ideal: validation MCC 최대화. 같은 모델의 동일 probability에서 세 점을 계산합니다.

- baseline / ideal: Recall 0.750967, FPR 0.139432

- baseline / optimistic: Recall 0.543517, FPR 0.049975

- baseline / pessimistic: Recall 0.100068, FPR 0.00331368

- histgb / ideal: Recall 0.9033, FPR 0.0580793

- histgb / optimistic: Recall 0.890813, FPR 0.0499878

- histgb / pessimistic: Recall 0.101018, FPR 0.0030825

- logistic / ideal: Recall 0.823339, FPR 0.0781284

- logistic / optimistic: Recall 0.766256, FPR 0.049975

- logistic / pessimistic: Recall 0.100245, FPR 0.00301828

- tree / ideal: Recall 0.88685, FPR 0.070191

- tree / optimistic: Recall 0.840433, FPR 0.0447861

- tree / pessimistic: Recall 0.105272, FPR 0.00368615

- baseline / ideal: Recall 0.751676, FPR 0.140998

- baseline / optimistic: Recall 0.543227, FPR 0.050463

- baseline / pessimistic: Recall 0.10018, FPR 0.00327515

- histgb / ideal: Recall 0.902558, FPR 0.057039

- histgb / optimistic: Recall 0.890298, FPR 0.0492942

- histgb / pessimistic: Recall 0.100793, FPR 0.00249168

Baseline optimistic holdout FPR 5.0463%는 5%를 넘었습니다. validation 제약은 holdout에서 보장되지 않으며 threshold를 다시 조정하지 않았습니다.

## 오류·분포 분석

train/validation 최대 numeric KS 0.002943, categorical total variation 0.001382. 무작위 합성 split의 기술적 점검이며 시간 변화에 대한 증거가 아닙니다.

Applied Personal Travel Recall 34.56%, Business travel 94.00%. 전체 지표에 가려진 그룹 오류를 기록했고 분석 후 모델·threshold를 변경하지 않았습니다.

## 검증과 남은 범위

보고 모듈 실제 검증: 74 tests passed, 실패·skip 0개. 동결 JSON 해시가 생성 전후 일치했으며 재학습·holdout 재평가 없이 보고서를 생성했습니다.

- 수치·미평가·제약·DAG·escaping·SVG markup·회귀/이상탐지 fixture를 검증했습니다.

- 브라우저 pixel QA는 미완료입니다. 로컬 Chromium 부재, Cloud Browser file 접근 정책 차단을 우회하지 않았습니다.

- 60–90분 L1 실습·실측 재사용률·SK하이닉스 대회별 사용 규정 확인은 남아 있습니다. CHO-FAM 정규 학습 일정은 변경하지 않습니다.

## 재사용과 Reference

재사용률 미측정. kit/adapted/new 작업시간 근거가 없으므로 70% 달성을 추정하지 않습니다.

동결 후 KRISH RABARI의 S6E10 EDA and Stacking Pipeline에서 지연 sum/difference, Optuna, stacking을 확인했지만 실행하지 않았습니다. 처음 확인한 feature는 kit·모델에 편입하지 않았습니다. Reference 점수는 이 holdout과 비교하지 않습니다.

## 근거와 다운로드

동결 JSON의 SHA-256 근거를 제공합니다. 원본 데이터·개별 예측·모델 artifact·인증정보는 게시하지 않습니다. Markdown은 같은 상세 지표와 표를 포함합니다.

전체 Markdown 다운로드

# S6E10 — frozen blind benchmark

## Interpretation and warnings

- Primary metric: roc_auc; operating-point utility: mcc.
- Design estimates are not measured execution times. Missing values are not zero.
- No training, tuning or holdout evaluation is performed by this renderer.
- Reference not evaluated on this split.
- Reuse not measured. The 70% target is not claimed.
- Descriptive synthetic IID split audit; not evidence of real production generalization or causal fairness.
- blueprint_guards_unspecified:baseline:classification
- max_fpr_violated:baseline:holdout

## Design only: baseline:classification

```mermaid
flowchart TD
    n0["preprocessing (planned)"]
    n1["classifier (planned)"]
    n0 --> n1
    n2["imbalance_check (planned)"]
    n1 --> n2
    n3["evaluation (planned)"]
    n2 --> n3
```

## Execution evidence

| Stage | Status | Evidence |
| --- | --- | --- |
| Profile/planner | executed | planner_freeze.json |
| Baseline freeze | executed | baseline_freeze.json |
| Applied freeze | executed | applied_freeze.json |
| Single frozen holdout | executed | holdout_once.json |
| Descriptive post-freeze audit | executed | audit.json |
| Timed L1/reuse measurement | deferred | not executed |

## Guard evidence (declaration is not verification)

| Guard | Status | Evidence |
| --- | --- | --- |
| frozen_holdout | executed | holdout_once.json |
| train_only_preprocessing | planned | experiment config |

## Design estimates (not measured runtime)

| Blueprint | Estimated minutes | Tune policy | Guards |
| --- | --- | --- | --- |
| baseline:classification | 40 | tune | Unspecified |

## Strategy declarations

| Strategy | Objective | Declared metric |
| --- | --- | --- |
| pessimistic | minimize_false_positive | false_positive_rate |
| optimistic | maximize_detection | recall_or_coverage |
| ideal | optimize_problem_utility | roc_auc |

## Model results — partitions kept separate

| Model | Partition | Status | roc_auc | Fit seconds | Score seconds | Peak RSS bytes | Frozen |
| --- | --- | --- | --- | --- | --- | --- | --- |
| baseline | validation | executed | 0.873528 | 1.72601 | 1.35003 | 695590912 | True |
| baseline | holdout | executed | 0.873663 | Not measured | 1.44971 | Not measured | True |
| histgb | validation | executed | 0.957247 | 4.7231 | 0.453554 | 782938112 | True |
| histgb | holdout | executed | 0.958004 | Not measured | 0.57537 | Not measured | True |
| logistic | validation | executed | 0.921173 | 1.67819 | 0.123106 | 782839808 | True |
| logistic | holdout | not_evaluated | Not measured | Not measured | Not measured | Not measured | False |
| tree | validation | executed | 0.949604 | 2.63384 | 0.137805 | 782974976 | True |
| tree | holdout | not_evaluated | Not measured | Not measured | Not measured | Not measured | False |

## baseline / validation metrics

| Metric | Value |
| --- | --- |
| log_loss | 1.08659 |
| roc_auc | 0.873528 |

## baseline / validation operating points

| Point | Threshold | Precision | Recall | FPR | mcc | TP | FP | TN | FN |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| ideal | 0.929024 | 0.811092 | 0.750967 | 0.139432 | 0.617559 | 46611 | 10856 | 67003 | 15457 |
| optimistic | 0.99877 | 0.896587 | 0.543517 | 0.049975 | 0.553004 | 33735 | 3891 | 73968 | 28333 |
| pessimistic | 0.999999 | 0.960117 | 0.100068 | 0.00331368 | 0.228911 | 6211 | 258 | 77601 | 55857 |

## baseline / holdout metrics

| Metric | Value |
| --- | --- |
| log_loss | 1.08594 |
| roc_auc | 0.873663 |

## baseline / holdout operating points

| Point | Threshold | Precision | Recall | FPR | mcc | TP | FP | TN | FN |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| ideal | 0.929024 | 0.809519 | 0.751676 | 0.140998 | 0.616425 | 46655 | 10978 | 66881 | 15413 |
| optimistic | 0.99877 | 0.895633 | 0.543227 | 0.050463 | 0.552039 | 33717 | 3929 | 73930 | 28351 |
| pessimistic | 0.999999 | 0.960606 | 0.10018 | 0.00327515 | 0.229201 | 6218 | 255 | 77604 | 55850 |

## histgb / validation metrics

| Metric | Value |
| --- | --- |
| log_loss | 0.231054 |
| roc_auc | 0.957247 |

## histgb / validation operating points

| Point | Threshold | Precision | Recall | FPR | mcc | TP | FP | TN | FN |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| ideal | 0.49633 | 0.925365 | 0.9033 | 0.0580793 | 0.847464 | 56066 | 4522 | 73337 | 6002 |
| optimistic | 0.574575 | 0.934238 | 0.890813 | 0.0499878 | 0.845552 | 55291 | 3892 | 73967 | 6777 |
| pessimistic | 0.964861 | 0.963134 | 0.101018 | 0.0030825 | 0.231011 | 6270 | 240 | 77619 | 55798 |

## histgb / holdout metrics

| Metric | Value |
| --- | --- |
| log_loss | 0.229575 |
| roc_auc | 0.958004 |

## histgb / holdout operating points

| Point | Threshold | Precision | Recall | FPR | mcc | TP | FP | TN | FN |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| ideal | 0.49633 | 0.926548 | 0.902558 | 0.057039 | 0.847976 | 56020 | 4441 | 73418 | 6048 |
| optimistic | 0.574575 | 0.935056 | 0.890298 | 0.0492942 | 0.845896 | 55259 | 3838 | 74021 | 6809 |
| pessimistic | 0.964861 | 0.969922 | 0.100793 | 0.00249168 | 0.232897 | 6256 | 194 | 77665 | 55812 |

## logistic / validation metrics

| Metric | Value |
| --- | --- |
| log_loss | 0.345318 |
| roc_auc | 0.921173 |

## logistic / validation operating points

| Point | Threshold | Precision | Recall | FPR | mcc | TP | FP | TN | FN |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| ideal | 0.568183 | 0.893628 | 0.823339 | 0.0781284 | 0.753116 | 51103 | 6083 | 71776 | 10965 |
| optimistic | 0.699258 | 0.924375 | 0.766256 | 0.049975 | 0.73801 | 47560 | 3891 | 73968 | 14508 |
| pessimistic | 0.961886 | 0.963605 | 0.100245 | 0.00301828 | 0.230232 | 6222 | 235 | 77624 | 55846 |

## tree / validation metrics

| Metric | Value |
| --- | --- |
| log_loss | 0.257834 |
| roc_auc | 0.949604 |

## tree / validation operating points

| Point | Threshold | Precision | Recall | FPR | mcc | TP | FP | TN | FN |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| ideal | 0.48 | 0.909684 | 0.88685 | 0.070191 | 0.818952 | 55045 | 5465 | 72394 | 7023 |
| optimistic | 0.657718 | 0.937342 | 0.840433 | 0.0447861 | 0.807645 | 52164 | 3487 | 74372 | 9904 |
| pessimistic | 0.949454 | 0.957924 | 0.105272 | 0.00368615 | 0.234367 | 6534 | 287 | 77572 | 55534 |

## sample_submission.csv data profile

| Measure | Value |
| --- | --- |
| column_count | 2 |
| deep_memory_bytes | 4797636 |
| duplicate_ids | 0 |
| row_count | 299844 |
| size_bytes | 8095804 |
| target_counts | Not measured |

## sample_submission.csv missing_rate

| Field/class | Value |
| --- | --- |
| id | 0 |
| satisfaction | 0 |

## test.csv data profile

| Measure | Value |
| --- | --- |
| column_count | 22 |
| deep_memory_bytes | 113995170 |
| duplicate_ids | 0 |
| row_count | 299844 |
| size_bytes | 27200964 |
| target_counts | Not measured |

## test.csv missing_rate

| Field/class | Value |
| --- | --- |
| Age | 0 |
| Arrival Delay in Minutes | 0.000433559 |
| Baggage handling | 0 |
| Checkin service | 0 |
| Class | 0 |
| Cleanliness | 0 |
| Customer Type | 0 |
| Departure Delay in Minutes | 0 |
| Departure/Arrival time convenient | 0 |
| Ease of Online booking | 0 |
| Flight Distance | 0 |
| Food and drink | 0 |
| Gate location | 0 |
| Gender | 0 |
| Inflight entertainment | 0 |
| Inflight wifi service | 0 |
| Leg room service | 0 |
| On-board service | 0 |
| Online boarding | 0 |
| Seat comfort | 0 |
| Type of Travel | 0 |
| id | 0 |

## train.csv data profile

| Measure | Value |
| --- | --- |
| column_count | 23 |
| deep_memory_bytes | 266687521 |
| duplicate_ids | 0 |
| row_count | 699635 |
| size_bytes | 67245068 |

## train.csv missing_rate

| Field/class | Value |
| --- | --- |
| Age | 0 |
| Arrival Delay in Minutes | 0.00041736 |
| Baggage handling | 0 |
| Checkin service | 0 |
| Class | 0 |
| Cleanliness | 0 |
| Customer Type | 0 |
| Departure Delay in Minutes | 0 |
| Departure/Arrival time convenient | 0 |
| Ease of Online booking | 0 |
| Flight Distance | 0 |
| Food and drink | 0 |
| Gate location | 0 |
| Gender | 0 |
| Inflight entertainment | 0 |
| Inflight wifi service | 0 |
| Leg room service | 0 |
| On-board service | 0 |
| Online boarding | 0 |
| Seat comfort | 0 |
| Type of Travel | 0 |
| id | 0 |
| satisfaction | 0 |

## train.csv target_counts

| Field/class | Value |
| --- | --- |
| False | 389296 |
| True | 310339 |

## Drift / numeric_ks / train_vs_validation_only

| Feature | Value |
| --- | --- |
| Age | 0.00134832 |
| Arrival Delay in Minutes | 0.000493286 |
| Baggage handling | 0.000795653 |
| Checkin service | 0.000893323 |
| Cleanliness | 0.00221068 |
| Departure Delay in Minutes | 0.00126733 |
| Departure/Arrival time convenient | 0.00151746 |
| Ease of Online booking | 0.000626517 |
| Flight Distance | 0.00276096 |
| Food and drink | 0.000628899 |
| Gate location | 0.00102434 |
| Inflight entertainment | 0.00294201 |
| Inflight wifi service | 0.00144837 |
| Leg room service | 0.00222259 |
| On-board service | 0.0011101 |
| Online boarding | 0.000850443 |
| Seat comfort | 0.00153175 |

## Drift / categorical_total_variation / train_vs_validation_only

| Feature | Value |
| --- | --- |
| Class | 0.00116728 |
| Customer Type | 0.000419266 |
| Gender | 0.00103864 |
| Type of Travel | 0.00138167 |

## baseline subgroup errors / Class

| Group | N | Recall | FPR | TP | FP | TN | FN |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Business | 68410 | 0.833478 | 0.199819 | 41368 | 3752 | 15025 | 8265 |
| Eco | 65489 | 0.420047 | 0.123773 | 4618 | 6745 | 47750 | 6376 |
| Eco Plus | 6028 | 0.464261 | 0.104862 | 669 | 481 | 4106 | 772 |

## baseline subgroup errors / Customer Type

| Group | N | Recall | FPR | TP | FP | TN | FN |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Loyal Customer | 115190 | 0.786509 | 0.154745 | 44890 | 8993 | 49122 | 12185 |
| disloyal Customer | 24737 | 0.353495 | 0.100537 | 1765 | 1985 | 17759 | 3228 |

## baseline subgroup errors / Gender

| Group | N | Recall | FPR | TP | FP | TN | FN |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Female | 69504 | 0.753903 | 0.143392 | 22988 | 5594 | 33418 | 7504 |
| Male | 70423 | 0.749525 | 0.138595 | 23667 | 5384 | 33463 | 7909 |

## baseline subgroup errors / Type of Travel

| Group | N | Recall | FPR | TP | FP | TN | FN |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Business travel | 99545 | 0.781713 | 0.120291 | 45466 | 4978 | 36405 | 12696 |
| Personal Travel | 40382 | 0.304403 | 0.164492 | 1189 | 6000 | 30476 | 2717 |

## histgb subgroup errors / Class

| Group | N | Recall | FPR | TP | FP | TN | FN |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Business | 68410 | 0.971148 | 0.176599 | 48201 | 3316 | 15461 | 1432 |
| Eco | 65489 | 0.614608 | 0.0180384 | 6757 | 983 | 53512 | 4237 |
| Eco Plus | 6028 | 0.736988 | 0.0309571 | 1062 | 142 | 4445 | 379 |

## histgb subgroup errors / Customer Type

| Group | N | Recall | FPR | TP | FP | TN | FN |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Loyal Customer | 115190 | 0.91986 | 0.0637873 | 52501 | 3707 | 54408 | 4574 |
| disloyal Customer | 24737 | 0.704787 | 0.0371759 | 3519 | 734 | 19010 | 1474 |

## histgb subgroup errors / Gender

| Group | N | Recall | FPR | TP | FP | TN | FN |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Female | 69504 | 0.905779 | 0.0581872 | 27619 | 2270 | 36742 | 2873 |
| Male | 70423 | 0.899449 | 0.0558859 | 28401 | 2171 | 36676 | 3175 |

## histgb subgroup errors / Type of Travel

| Group | N | Recall | FPR | TP | FP | TN | FN |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Business travel | 99545 | 0.939961 | 0.104512 | 54670 | 4325 | 37058 | 3492 |
| Personal Travel | 40382 | 0.345622 | 0.00318017 | 1350 | 116 | 36360 | 2556 |

## Sources / SHA-256

| Source | Hash |
| --- | --- |
| applied_freeze.json | a3a8c980f7903857599c45a7ce87f34e607e119426061fcf11b4de831386fbaf |
| audit.json | c7b1c9c12e9bd02bf7207bccd9f46bbc3aef5890635d89586be16866f5dc734a |
| baseline_freeze.json | d8eb4bf340da628e54e8dbee772350902943d89644424631f8fdb9403ba39d1f |
| csv_profiles.json | b2e16bc4bc754c870503555b64a54c5057ae674a7d7d51dfcef7540808c1d337 |
| experiment_freeze.json | ab132fbb91620435b51c852e08e4ab89fd101b9b5dd775b268171c8964cebafe |
| histgb_freeze.json | 4bfe68dd9058f42ddf995bc408444f35011adc3dd547f42cefbfbc85de3745f6 |
| holdout_once.json | 29d8d4c17f91fdd1879853ec0227274305d3d967605988421670eae637fb9b5f |
| logistic_freeze.json | d4dde516905e3b4c36c70b5d00f306ebcc022cf28ba6d4140ddf78542946302b |
| planner_freeze.json | 518f49179e8ce294ee5df3bc0b57c5cb36655398b05260c1c2b221f320a3cda0 |
| tree_freeze.json | 16dbc4061a45a47de74bc001e6c7202f6c17c8a170d6b31e4c7b0b44e3d3a1c3 |