CHO-FAM | Hackathon Kit — 설계·실행·결과

실행 개요

처음 보는 문제에서 문제 정의·프로파일링·모델 비교·검증·보고 workflow를 재사용할 수 있는지 확인하는 실험입니다. 진행 중인 S6E10으로 전환했으며 Bosch는 보류했습니다.

전체 train699,635행23열 · 부분집합 아님

Applied holdout ROC-AUC0.958004동결 후 1회 평가

Applied holdout MCC0.847976동결된 ideal threshold

합성 항공 데이터의 workflow 검증입니다. 제조 성능·생산 환경 일반화·Kaggle 제출·70% 재사용 달성을 주장하지 않습니다.

설계와 판단 근거

ProblemSpec·DataProfile → deterministic SolutionPlanner → Blueprint → Strategy를 실험 전에 동결했습니다. 아래 그래프는 설계이며 개별 구성요소의 실행 완료를 뜻하지 않습니다.

Design estimates (not measured runtime)

Blueprint	Estimated minutes	Tune policy	Guards
baseline:classification	40	tune	Unspecified

Strategy declarations

Strategy	Objective	Declared metric
pessimistic	minimize_false_positive	false_positive_rate
optimistic	maximize_detection	recall_or_coverage
ideal	optimize_problem_utility	roc_auc

예상 40분은 설계 구성요소 비용입니다. 실측 fitting 시간과 작업 범위가 다릅니다. 공식 ROC-AUC는 threshold와 무관하며 ideal threshold는 별도의 MCC 최적화입니다.

실행과 동결

stratified seed 20261010, 60/20/20 분할: train 419,781 / validation 139,927 / holdout 139,927. train-only 전처리를 적용했고 모델·threshold는 validation에서 선택해 holdout 전에 동결했습니다. test는 선택에 사용하지 않았습니다.

Baseline 동결 뒤 후보 3개를 비교했습니다. 총 fit ceiling 4개, 후보 wall budget 1,200초에서 실제 비교는 16.354초였습니다. 공개 reference는 Baseline·Applied·holdout 동결 후 읽었습니다.

Execution evidence

Stage	Status	Evidence
Profile/planner	executed	planner_freeze.json
Baseline freeze	executed	baseline_freeze.json
Applied freeze	executed	applied_freeze.json
Single frozen holdout	executed	holdout_once.json
Descriptive post-freeze audit	executed	audit.json
Timed L1/reuse measurement	deferred	not executed

Guard evidence (declaration is not verification)

Guard	Status	Evidence
frozen_holdout	executed	holdout_once.json
train_only_preprocessing	planned	experiment config

Blueprint required_guards가 비어 있는 경고를 유지합니다. adapter는 전처리 선언만으로 실행 검사를 인증하지 않습니다.

데이터 품질과 자원

공식 CSV 3개, 102,541,836 bytes. 17 numeric·4 categorical predictor, ID 제외. train 양성 310,339 / 음성 389,296 (양성 44.36%). ID 중복·공백·train/test 겹침 0개, test/sample ID 일치.

Arrival Delay in Minutes만 결측(train 292 / test 130). Profile 4.079초, peak RSS 474,181,632 bytes; 후보 최대 peak RSS 782,974,976 bytes. 원본 해시가 유지됐습니다.

파일·결측률·타깃 상세

sample_submission.csv data profile

Measure	Value
column_count	2
deep_memory_bytes	4797636
duplicate_ids	0
row_count	299844
size_bytes	8095804
target_counts	Not measured

sample_submission.csv missing_rate

Field/class	Value
id	0
satisfaction	0

test.csv data profile

Measure	Value
column_count	22
deep_memory_bytes	113995170
duplicate_ids	0
row_count	299844
size_bytes	27200964
target_counts	Not measured

test.csv missing_rate

Field/class	Value
Age	0
Arrival Delay in Minutes	0.000433559
Baggage handling	0
Checkin service	0
Class	0
Cleanliness	0
Customer Type	0
Departure Delay in Minutes	0
Departure/Arrival time convenient	0
Ease of Online booking	0
Flight Distance	0
Food and drink	0
Gate location	0
Gender	0
Inflight entertainment	0
Inflight wifi service	0
Leg room service	0
On-board service	0
Online boarding	0
Seat comfort	0
Type of Travel	0
id	0

train.csv data profile

Measure	Value
column_count	23
deep_memory_bytes	266687521
duplicate_ids	0
row_count	699635
size_bytes	67245068

train.csv missing_rate

Field/class	Value
Age	0
Arrival Delay in Minutes	0.00041736
Baggage handling	0
Checkin service	0
Class	0
Cleanliness	0
Customer Type	0
Departure Delay in Minutes	0
Departure/Arrival time convenient	0
Ease of Online booking	0
Flight Distance	0
Food and drink	0
Gate location	0
Gender	0
Inflight entertainment	0
Inflight wifi service	0
Leg room service	0
On-board service	0
Online boarding	0
Seat comfort	0
Type of Travel	0
id	0
satisfaction	0

train.csv target_counts

Field/class	Value
False	389296
True	310339

Baseline → Applied 결과

Centroid Baseline은 numeric-only이고 후보에는 제공된 categorical 열도 포함했습니다. 파생 feature·외부 데이터 없이 선형·tree·gradient boosting을 비교해 validation ROC-AUC로 HistGradientBoosting을 선택했습니다.

baseline
histgb
logistic
tree

Model results — partitions kept separate

Model	Partition	Status	roc_auc	Fit seconds	Score seconds	Peak RSS bytes	Frozen
baseline	validation	executed	0.873528	1.72601	1.35003	695590912	True
baseline	holdout	executed	0.873663	Not measured	1.44971	Not measured	True
histgb	validation	executed	0.957247	4.7231	0.453554	782938112	True
histgb	holdout	executed	0.958004	Not measured	0.57537	Not measured	True
logistic	validation	executed	0.921173	1.67819	0.123106	782839808	True
logistic	holdout	not_evaluated	Not measured	Not measured	Not measured	Not measured	False
tree	validation	executed	0.949604	2.63384	0.137805	782974976	True
tree	holdout	not_evaluated	Not measured	Not measured	Not measured	Not measured	False

baseline / validation metrics

Metric	Value
log_loss	1.08659
roc_auc	0.873528

baseline / holdout metrics

Metric	Value
log_loss	1.08594
roc_auc	0.873663

histgb / validation metrics

Metric	Value
log_loss	0.231054
roc_auc	0.957247

histgb / holdout metrics

Metric	Value
log_loss	0.229575
roc_auc	0.958004

logistic / validation metrics

Metric	Value
log_loss	0.345318
roc_auc	0.921173

tree / validation metrics

Metric	Value
log_loss	0.257834
roc_auc	0.949604

LogisticRegression·DecisionTree holdout은 미평가이며 0이 아닙니다. Baseline probability는 calibration되지 않은 centroid score입니다.

세 운영점과 제약

Pessimistic: 최소 FP, validation Recall ≥ 10%. Optimistic: 최대 Recall, validation FPR ≤ 5%. Ideal: validation MCC 최대화. 같은 모델의 동일 probability에서 세 점을 계산합니다.

Dashed guides: min Recall applies to pessimistic; max FPR applies to optimistic only.

baseline / ideal: Recall 0.750967, FPR 0.139432
baseline / optimistic: Recall 0.543517, FPR 0.049975
baseline / pessimistic: Recall 0.100068, FPR 0.00331368
histgb / ideal: Recall 0.9033, FPR 0.0580793
histgb / optimistic: Recall 0.890813, FPR 0.0499878
histgb / pessimistic: Recall 0.101018, FPR 0.0030825
logistic / ideal: Recall 0.823339, FPR 0.0781284
logistic / optimistic: Recall 0.766256, FPR 0.049975
logistic / pessimistic: Recall 0.100245, FPR 0.00301828
tree / ideal: Recall 0.88685, FPR 0.070191
tree / optimistic: Recall 0.840433, FPR 0.0447861
tree / pessimistic: Recall 0.105272, FPR 0.00368615

Dashed guides: min Recall applies to pessimistic; max FPR applies to optimistic only.

baseline / ideal: Recall 0.751676, FPR 0.140998
baseline / optimistic: Recall 0.543227, FPR 0.050463
baseline / pessimistic: Recall 0.10018, FPR 0.00327515
histgb / ideal: Recall 0.902558, FPR 0.057039
histgb / optimistic: Recall 0.890298, FPR 0.0492942
histgb / pessimistic: Recall 0.100793, FPR 0.00249168

Baseline optimistic holdout FPR 5.0463%는 5%를 넘었습니다. validation 제약은 holdout에서 보장되지 않으며 threshold를 다시 조정하지 않았습니다.

baseline / validation operating points

Point	Threshold	Precision	Recall	FPR	mcc	TP	FP	TN	FN
ideal	0.929024	0.811092	0.750967	0.139432	0.617559	46611	10856	67003	15457
optimistic	0.99877	0.896587	0.543517	0.049975	0.553004	33735	3891	73968	28333
pessimistic	0.999999	0.960117	0.100068	0.00331368	0.228911	6211	258	77601	55857

baseline / holdout operating points

Point	Threshold	Precision	Recall	FPR	mcc	TP	FP	TN	FN
ideal	0.929024	0.809519	0.751676	0.140998	0.616425	46655	10978	66881	15413
optimistic	0.99877	0.895633	0.543227	0.050463	0.552039	33717	3929	73930	28351
pessimistic	0.999999	0.960606	0.10018	0.00327515	0.229201	6218	255	77604	55850

histgb / validation operating points

Point	Threshold	Precision	Recall	FPR	mcc	TP	FP	TN	FN
ideal	0.49633	0.925365	0.9033	0.0580793	0.847464	56066	4522	73337	6002
optimistic	0.574575	0.934238	0.890813	0.0499878	0.845552	55291	3892	73967	6777
pessimistic	0.964861	0.963134	0.101018	0.0030825	0.231011	6270	240	77619	55798

histgb / holdout operating points

Point	Threshold	Precision	Recall	FPR	mcc	TP	FP	TN	FN
ideal	0.49633	0.926548	0.902558	0.057039	0.847976	56020	4441	73418	6048
optimistic	0.574575	0.935056	0.890298	0.0492942	0.845896	55259	3838	74021	6809
pessimistic	0.964861	0.969922	0.100793	0.00249168	0.232897	6256	194	77665	55812

logistic / validation operating points

Point	Threshold	Precision	Recall	FPR	mcc	TP	FP	TN	FN
ideal	0.568183	0.893628	0.823339	0.0781284	0.753116	51103	6083	71776	10965
optimistic	0.699258	0.924375	0.766256	0.049975	0.73801	47560	3891	73968	14508
pessimistic	0.961886	0.963605	0.100245	0.00301828	0.230232	6222	235	77624	55846

tree / validation operating points

Point	Threshold	Precision	Recall	FPR	mcc	TP	FP	TN	FN
ideal	0.48	0.909684	0.88685	0.070191	0.818952	55045	5465	72394	7023
optimistic	0.657718	0.937342	0.840433	0.0447861	0.807645	52164	3487	74372	9904
pessimistic	0.949454	0.957924	0.105272	0.00368615	0.234367	6534	287	77572	55534

오류·분포 분석

baseline / holdout ideal

histgb / holdout ideal

train/validation 최대 numeric KS 0.002943, categorical total variation 0.001382. 무작위 합성 split의 기술적 점검이며 시간 변화에 대한 증거가 아닙니다.

Applied Personal Travel Recall 34.56%, Business travel 94.00%. 전체 지표에 가려진 그룹 오류를 기록했고 분석 후 모델·threshold를 변경하지 않았습니다.

그룹별 오류와 분포 점검값

Drift / numeric_ks / train_vs_validation_only

Feature	Value
Age	0.00134832
Arrival Delay in Minutes	0.000493286
Baggage handling	0.000795653
Checkin service	0.000893323
Cleanliness	0.00221068
Departure Delay in Minutes	0.00126733
Departure/Arrival time convenient	0.00151746
Ease of Online booking	0.000626517
Flight Distance	0.00276096
Food and drink	0.000628899
Gate location	0.00102434
Inflight entertainment	0.00294201
Inflight wifi service	0.00144837
Leg room service	0.00222259
On-board service	0.0011101
Online boarding	0.000850443
Seat comfort	0.00153175

Drift / categorical_total_variation / train_vs_validation_only

Feature	Value
Class	0.00116728
Customer Type	0.000419266
Gender	0.00103864
Type of Travel	0.00138167

baseline subgroup errors / Class

Group	N	Recall	FPR	TP	FP	TN	FN
Business	68410	0.833478	0.199819	41368	3752	15025	8265
Eco	65489	0.420047	0.123773	4618	6745	47750	6376
Eco Plus	6028	0.464261	0.104862	669	481	4106	772

baseline subgroup errors / Customer Type

Group	N	Recall	FPR	TP	FP	TN	FN
Loyal Customer	115190	0.786509	0.154745	44890	8993	49122	12185
disloyal Customer	24737	0.353495	0.100537	1765	1985	17759	3228

baseline subgroup errors / Gender

Group	N	Recall	FPR	TP	FP	TN	FN
Female	69504	0.753903	0.143392	22988	5594	33418	7504
Male	70423	0.749525	0.138595	23667	5384	33463	7909

baseline subgroup errors / Type of Travel

Group	N	Recall	FPR	TP	FP	TN	FN
Business travel	99545	0.781713	0.120291	45466	4978	36405	12696
Personal Travel	40382	0.304403	0.164492	1189	6000	30476	2717

histgb subgroup errors / Class

Group	N	Recall	FPR	TP	FP	TN	FN
Business	68410	0.971148	0.176599	48201	3316	15461	1432
Eco	65489	0.614608	0.0180384	6757	983	53512	4237
Eco Plus	6028	0.736988	0.0309571	1062	142	4445	379

histgb subgroup errors / Customer Type

Group	N	Recall	FPR	TP	FP	TN	FN
Loyal Customer	115190	0.91986	0.0637873	52501	3707	54408	4574
disloyal Customer	24737	0.704787	0.0371759	3519	734	19010	1474

histgb subgroup errors / Gender

Group	N	Recall	FPR	TP	FP	TN	FN
Female	69504	0.905779	0.0581872	27619	2270	36742	2873
Male	70423	0.899449	0.0558859	28401	2171	36676	3175

histgb subgroup errors / Type of Travel

Group	N	Recall	FPR	TP	FP	TN	FN
Business travel	99545	0.939961	0.104512	54670	4325	37058	3492
Personal Travel	40382	0.345622	0.00318017	1350	116	36360	2556

검증과 남은 범위

보고 모듈 실제 검증: 74 tests passed, 실패·skip 0개. 동결 JSON 해시가 생성 전후 일치했으며 재학습·holdout 재평가 없이 보고서를 생성했습니다.

수치·미평가·제약·DAG·escaping·SVG markup·회귀/이상탐지 fixture를 검증했습니다.
브라우저 pixel QA는 미완료입니다. 로컬 Chromium 부재, Cloud Browser file 접근 정책 차단을 우회하지 않았습니다.
60–90분 L1 실습·실측 재사용률·SK하이닉스 대회별 사용 규정 확인은 남아 있습니다. CHO-FAM 정규 학습 일정은 변경하지 않습니다.

재사용과 Reference

재사용률 미측정. kit/adapted/new 작업시간 근거가 없으므로 70% 달성을 추정하지 않습니다.

동결 후 KRISH RABARI의 S6E10 EDA and Stacking Pipeline에서 지연 sum/difference, Optuna, stacking을 확인했지만 실행하지 않았습니다. 처음 확인한 feature는 kit·모델에 편입하지 않았습니다. Reference 점수는 이 holdout과 비교하지 않습니다.

근거와 다운로드

동결 JSON의 SHA-256 근거를 제공합니다. 원본 데이터·개별 예측·모델 artifact·인증정보는 게시하지 않습니다. Markdown은 같은 상세 지표와 표를 포함합니다.

전체 Markdown 다운로드

동결 source manifest

Sources / SHA-256

Source	Hash
applied_freeze.json	a3a8c980f7903857599c45a7ce87f34e607e119426061fcf11b4de831386fbaf
audit.json	c7b1c9c12e9bd02bf7207bccd9f46bbc3aef5890635d89586be16866f5dc734a
baseline_freeze.json	d8eb4bf340da628e54e8dbee772350902943d89644424631f8fdb9403ba39d1f
csv_profiles.json	b2e16bc4bc754c870503555b64a54c5057ae674a7d7d51dfcef7540808c1d337
experiment_freeze.json	ab132fbb91620435b51c852e08e4ab89fd101b9b5dd775b268171c8964cebafe
histgb_freeze.json	4bfe68dd9058f42ddf995bc408444f35011adc3dd547f42cefbfbc85de3745f6
holdout_once.json	29d8d4c17f91fdd1879853ec0227274305d3d967605988421670eae637fb9b5f
logistic_freeze.json	d4dde516905e3b4c36c70b5d00f306ebcc022cf28ba6d4140ddf78542946302b
planner_freeze.json	518f49179e8ce294ee5df3bc0b57c5cb36655398b05260c1c2b221f320a3cda0
tree_freeze.json	16dbc4061a45a47de74bc001e6c7202f6c17c8a170d6b31e4c7b0b44e3d3a1c3