Few-shot Benchmark Report

Run ID: 20260505T181843Z | Generated UTC: 2026-05-05T18:21:48.786233+00:00

Configuration

{
  "models": [
    "opus-4-5",
    "opus-4-6",
    "sonnet-4-5",
    "sonnet-4-6"
  ],
  "fewshot_strategies": [
    "db_only",
    "db_plus_raw",
    "raw_only",
    "zero_shot"
  ],
  "chat_count": 21,
  "runs_per_chat": 1,
  "max_workers": 15,
  "raw_fewshot_count": 26,
  "raw_fewshot_labels_filter": []
}

Model + Strategy Summary

Model	Strategy	Runs	Success rate	Avg attempts	Avg elapsed (s)	Avg mismatch/expected run	Field match rate
opus-4-5	db_only	21	1.0000	1.0000	8.5433	7.6190	0.6825
opus-4-5	db_plus_raw	21	1.0000	1.0000	8.4424	8.6667	0.6546
opus-4-5	raw_only	21	1.0000	1.0000	18.7446	8.2381	0.6717
opus-4-5	zero_shot	21	1.0000	1.0000	7.2798	7.0000	0.7474
opus-4-6	db_only	21	1.0000	1.0000	7.8629	9.2381	0.6572
opus-4-6	db_plus_raw	21	1.0000	1.0000	8.7017	8.5238	0.6603
opus-4-6	raw_only	21	1.0000	1.0000	6.4485	8.0000	0.6812
opus-4-6	zero_shot	21	1.0000	1.0000	5.6740	6.1429	0.8045
sonnet-4-5	db_only	21	1.0000	1.0000	7.7986	7.7143	0.6867
sonnet-4-5	db_plus_raw	21	1.0000	1.0000	7.9757	7.9524	0.6687
sonnet-4-5	raw_only	21	1.0000	1.0000	7.0987	8.7143	0.6528
sonnet-4-5	zero_shot	21	1.0000	1.0000	6.0658	8.8095	0.7082
sonnet-4-6	db_only	21	1.0000	1.0000	6.9770	9.6190	0.6167
sonnet-4-6	db_plus_raw	21	1.0000	1.0000	9.5087	8.8571	0.6471
sonnet-4-6	raw_only	21	1.0000	1.0000	6.5341	8.3810	0.6660
sonnet-4-6	zero_shot	21	1.0000	1.0000	6.3680	8.6190	0.6943

Per-chat Breakdown

Chat	Model	Strategy	Runs	Success rate	Avg elapsed (s)	Mismatch counts
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-5	db_only	1	1.0000	7.7978	[15]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-5	db_plus_raw	1	1.0000	7.0853	[15]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-5	raw_only	1	1.0000	24.1299	[15]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-5	zero_shot	1	1.0000	5.8588	[15]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-6	db_only	1	1.0000	6.4233	[15]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-6	db_plus_raw	1	1.0000	7.0704	[15]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-6	raw_only	1	1.0000	4.9494	[14]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-6	zero_shot	1	1.0000	4.9043	[15]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-5	db_only	1	1.0000	7.7478	[15]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-5	db_plus_raw	1	1.0000	6.7182	[14]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-5	raw_only	1	1.0000	5.3144	[13]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-5	zero_shot	1	1.0000	5.2814	[15]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-6	db_only	1	1.0000	5.4053	[15]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-6	db_plus_raw	1	1.0000	11.9693	[15]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-6	raw_only	1	1.0000	4.9705	[16]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-6	zero_shot	1	1.0000	4.4847	[15]
02__2026-02-09__120363426578757754_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-5	db_only	1	1.0000	7.5812	[14]
02__2026-02-09__120363426578757754_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-5	db_plus_raw	1	1.0000	6.3277	[15]
02__2026-02-09__120363426578757754_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-5	raw_only	1	1.0000	36.8433	[14]
02__2026-02-09__120363426578757754_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-5	zero_shot	1	1.0000	5.5772	[14]
02__2026-02-09__120363426578757754_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-6	db_only	1	1.0000	6.8242	[15]
02__2026-02-09__120363426578757754_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-6	db_plus_raw	1	1.0000	7.0935	[14]
02__2026-02-09__120363426578757754_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-6	raw_only	1	1.0000	5.0888	[14]
02__2026-02-09__120363426578757754_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-6	zero_shot	1	1.0000	4.8696	[14]
02__2026-02-09__120363426578757754_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-5	db_only	1	1.0000	6.5448	[14]
02__2026-02-09__120363426578757754_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-5	db_plus_raw	1	1.0000	6.2405	[14]
02__2026-02-09__120363426578757754_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-5	raw_only	1	1.0000	5.0286	[13]
02__2026-02-09__120363426578757754_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-5	zero_shot	1	1.0000	5.7057	[14]
02__2026-02-09__120363426578757754_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-6	db_only	1	1.0000	8.1873	[14]
02__2026-02-09__120363426578757754_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-6	db_plus_raw	1	1.0000	13.1192	[15]
02__2026-02-09__120363426578757754_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-6	raw_only	1	1.0000	6.4834	[15]
02__2026-02-09__120363426578757754_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-6	zero_shot	1	1.0000	9.7462	[15]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-5	db_only	1	1.0000	10.2789	[17]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-5	db_plus_raw	1	1.0000	8.4137	[17]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-5	raw_only	1	1.0000	16.2404	[17]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-5	zero_shot	1	1.0000	9.2448	[17]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-6	db_only	1	1.0000	8.8351	[16]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-6	db_plus_raw	1	1.0000	10.5065	[17]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-6	raw_only	1	1.0000	8.9427	[17]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-6	zero_shot	1	1.0000	6.9384	[16]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-5	db_only	1	1.0000	10.6220	[17]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-5	db_plus_raw	1	1.0000	11.3037	[17]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-5	raw_only	1	1.0000	9.1418	[16]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-5	zero_shot	1	1.0000	8.8018	[17]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-6	db_only	1	1.0000	8.3721	[17]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-6	db_plus_raw	1	1.0000	10.7353	[17]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-6	raw_only	1	1.0000	9.1441	[18]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-6	zero_shot	1	1.0000	6.6795	[16]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	opus-4-5	db_only	1	1.0000	6.0217	[14]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	opus-4-5	db_plus_raw	1	1.0000	6.5016	[14]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	opus-4-5	raw_only	1	1.0000	5.6084	[14]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	opus-4-5	zero_shot	1	1.0000	4.8310	[15]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	opus-4-6	db_only	1	1.0000	6.6959	[15]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	opus-4-6	db_plus_raw	1	1.0000	6.9577	[14]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	opus-4-6	raw_only	1	1.0000	5.1176	[14]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	opus-4-6	zero_shot	1	1.0000	5.0645	[15]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	sonnet-4-5	db_only	1	1.0000	8.6198	[14]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	sonnet-4-5	db_plus_raw	1	1.0000	6.7137	[14]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	sonnet-4-5	raw_only	1	1.0000	5.4114	[12]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	sonnet-4-5	zero_shot	1	1.0000	4.8157	[16]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	sonnet-4-6	db_only	1	1.0000	6.1352	[14]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	sonnet-4-6	db_plus_raw	1	1.0000	18.4370	[15]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	sonnet-4-6	raw_only	1	1.0000	5.3292	[13]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	sonnet-4-6	zero_shot	1	1.0000	4.3817	[18]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-5	db_only	1	1.0000	6.9010	[14]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-5	db_plus_raw	1	1.0000	6.2877	[15]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-5	raw_only	1	1.0000	5.7034	[15]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-5	zero_shot	1	1.0000	5.0730	[14]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-6	db_only	1	1.0000	7.4538	[13]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-6	db_plus_raw	1	1.0000	8.8038	[14]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-6	raw_only	1	1.0000	5.5405	[15]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-6	zero_shot	1	1.0000	5.0031	[13]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-5	db_only	1	1.0000	7.4256	[14]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-5	db_plus_raw	1	1.0000	6.7737	[14]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-5	raw_only	1	1.0000	5.2603	[12]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-5	zero_shot	1	1.0000	5.3230	[13]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-6	db_only	1	1.0000	5.6681	[14]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-6	db_plus_raw	1	1.0000	6.3477	[15]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-6	raw_only	1	1.0000	5.1549	[15]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-6	zero_shot	1	1.0000	4.6234	[14]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-5	db_only	1	1.0000	6.7210	[14]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-5	db_plus_raw	1	1.0000	6.7340	[14]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-5	raw_only	1	1.0000	5.4206	[14]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-5	zero_shot	1	1.0000	4.9127	[14]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-6	db_only	1	1.0000	6.5684	[14]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-6	db_plus_raw	1	1.0000	6.9537	[14]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-6	raw_only	1	1.0000	5.2512	[13]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-6	zero_shot	1	1.0000	4.8988	[14]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-5	db_only	1	1.0000	6.0317	[14]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-5	db_plus_raw	1	1.0000	6.0662	[13]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-5	raw_only	1	1.0000	5.0284	[13]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-5	zero_shot	1	1.0000	5.1464	[14]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-6	db_only	1	1.0000	5.6211	[14]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-6	db_plus_raw	1	1.0000	9.9350	[14]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-6	raw_only	1	1.0000	4.8295	[13]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-6	zero_shot	1	1.0000	4.5296	[13]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-5	db_only	1	1.0000	6.2858	[1]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-5	db_plus_raw	1	1.0000	6.5685	[17]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-5	raw_only	1	1.0000	4.4782	[17]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-5	zero_shot	1	1.0000	2.5649	[1]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-6	db_only	1	1.0000	6.1288	[17]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-6	db_plus_raw	1	1.0000	8.9531	[19]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-6	raw_only	1	1.0000	4.6804	[17]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-6	zero_shot	1	1.0000	2.5292	[1]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-5	db_only	1	1.0000	4.4219	[1]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-5	db_plus_raw	1	1.0000	5.6431	[1]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-5	raw_only	1	1.0000	5.0054	[17]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-5	zero_shot	1	1.0000	3.0747	[1]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-6	db_only	1	1.0000	4.7528	[17]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-6	db_plus_raw	1	1.0000	6.3440	[17]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-6	raw_only	1	1.0000	4.2338	[17]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-6	zero_shot	1	1.0000	3.9791	[17]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-5	db_only	1	1.0000	6.3090	[17]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-5	db_plus_raw	1	1.0000	6.6022	[17]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-5	raw_only	1	1.0000	5.6837	[10]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-5	zero_shot	1	1.0000	5.2415	[14]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-6	db_only	1	1.0000	8.2271	[20]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-6	db_plus_raw	1	1.0000	7.0444	[15]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-6	raw_only	1	1.0000	5.7848	[11]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-6	zero_shot	1	1.0000	5.3247	[12]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-5	db_only	1	1.0000	6.7570	[15]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-5	db_plus_raw	1	1.0000	6.4338	[16]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-5	raw_only	1	1.0000	8.2128	[11]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-5	zero_shot	1	1.0000	5.3753	[19]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-6	db_only	1	1.0000	5.9355	[17]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-6	db_plus_raw	1	1.0000	7.1856	[15]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-6	raw_only	1	1.0000	5.3204	[11]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-6	zero_shot	1	1.0000	5.6384	[15]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-5	db_only	1	1.0000	6.5105	[15]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-5	db_plus_raw	1	1.0000	26.7842	[15]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-5	raw_only	1	1.0000	6.4909	[10]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-5	zero_shot	1	1.0000	5.2616	[12]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-6	db_only	1	1.0000	8.0422	[20]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-6	db_plus_raw	1	1.0000	10.7803	[15]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-6	raw_only	1	1.0000	5.5738	[11]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-6	zero_shot	1	1.0000	5.2122	[10]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-5	db_only	1	1.0000	6.2280	[15]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-5	db_plus_raw	1	1.0000	6.8198	[15]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-5	raw_only	1	1.0000	7.9559	[10]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-5	zero_shot	1	1.0000	5.5881	[17]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-6	db_only	1	1.0000	5.5759	[16]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-6	db_plus_raw	1	1.0000	6.3609	[15]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-6	raw_only	1	1.0000	6.2904	[11]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-6	zero_shot	1	1.0000	5.0627	[13]
multiple_product_multiple_shipment_complex.json	opus-4-5	db_only	1	1.0000	16.0715	[5]
multiple_product_multiple_shipment_complex.json	opus-4-5	db_plus_raw	1	1.0000	11.3479	[6]
multiple_product_multiple_shipment_complex.json	opus-4-5	raw_only	1	1.0000	12.9087	[7]
multiple_product_multiple_shipment_complex.json	opus-4-5	zero_shot	1	1.0000	12.6850	[4]
multiple_product_multiple_shipment_complex.json	opus-4-6	db_only	1	1.0000	13.2585	[5]
multiple_product_multiple_shipment_complex.json	opus-4-6	db_plus_raw	1	1.0000	12.6782	[6]
multiple_product_multiple_shipment_complex.json	opus-4-6	raw_only	1	1.0000	10.9548	[5]
multiple_product_multiple_shipment_complex.json	opus-4-6	zero_shot	1	1.0000	9.5077	[0]
multiple_product_multiple_shipment_complex.json	sonnet-4-5	db_only	1	1.0000	12.5446	[6]
multiple_product_multiple_shipment_complex.json	sonnet-4-5	db_plus_raw	1	1.0000	12.4740	[6]
multiple_product_multiple_shipment_complex.json	sonnet-4-5	raw_only	1	1.0000	12.6567	[7]
multiple_product_multiple_shipment_complex.json	sonnet-4-5	zero_shot	1	1.0000	9.7879	[21]
multiple_product_multiple_shipment_complex.json	sonnet-4-6	db_only	1	1.0000	10.5450	[5]
multiple_product_multiple_shipment_complex.json	sonnet-4-6	db_plus_raw	1	1.0000	13.4019	[7]
multiple_product_multiple_shipment_complex.json	sonnet-4-6	raw_only	1	1.0000	11.1374	[5]
multiple_product_multiple_shipment_complex.json	sonnet-4-6	zero_shot	1	1.0000	11.3678	[7]
multiple_product_multiple_shipment_medium.json	opus-4-5	db_only	1	1.0000	10.3140	[7]
multiple_product_multiple_shipment_medium.json	opus-4-5	db_plus_raw	1	1.0000	10.4044	[8]
multiple_product_multiple_shipment_medium.json	opus-4-5	raw_only	1	1.0000	34.8145	[8]
multiple_product_multiple_shipment_medium.json	opus-4-5	zero_shot	1	1.0000	7.8371	[7]
multiple_product_multiple_shipment_medium.json	opus-4-6	db_only	1	1.0000	9.2891	[7]
multiple_product_multiple_shipment_medium.json	opus-4-6	db_plus_raw	1	1.0000	10.5483	[7]
multiple_product_multiple_shipment_medium.json	opus-4-6	raw_only	1	1.0000	8.4393	[8]
multiple_product_multiple_shipment_medium.json	opus-4-6	zero_shot	1	1.0000	6.6737	[0]
multiple_product_multiple_shipment_medium.json	sonnet-4-5	db_only	1	1.0000	8.7758	[7]
multiple_product_multiple_shipment_medium.json	sonnet-4-5	db_plus_raw	1	1.0000	9.9930	[7]
multiple_product_multiple_shipment_medium.json	sonnet-4-5	raw_only	1	1.0000	8.7901	[9]
multiple_product_multiple_shipment_medium.json	sonnet-4-5	zero_shot	1	1.0000	7.8483	[8]
multiple_product_multiple_shipment_medium.json	sonnet-4-6	db_only	1	1.0000	8.9151	[7]
multiple_product_multiple_shipment_medium.json	sonnet-4-6	db_plus_raw	1	1.0000	9.5254	[7]
multiple_product_multiple_shipment_medium.json	sonnet-4-6	raw_only	1	1.0000	7.9204	[7]
multiple_product_multiple_shipment_medium.json	sonnet-4-6	zero_shot	1	1.0000	7.5363	[7]
multiple_product_multiple_shipment_simple.json	opus-4-5	db_only	1	1.0000	9.0913	[3]
multiple_product_multiple_shipment_simple.json	opus-4-5	db_plus_raw	1	1.0000	8.9143	[3]
multiple_product_multiple_shipment_simple.json	opus-4-5	raw_only	1	1.0000	25.9565	[4]
multiple_product_multiple_shipment_simple.json	opus-4-5	zero_shot	1	1.0000	10.6499	[6]
multiple_product_multiple_shipment_simple.json	opus-4-6	db_only	1	1.0000	7.5350	[5]
multiple_product_multiple_shipment_simple.json	opus-4-6	db_plus_raw	1	1.0000	8.9789	[5]
multiple_product_multiple_shipment_simple.json	opus-4-6	raw_only	1	1.0000	7.8721	[4]
multiple_product_multiple_shipment_simple.json	opus-4-6	zero_shot	1	1.0000	6.1409	[6]
multiple_product_multiple_shipment_simple.json	sonnet-4-5	db_only	1	1.0000	7.6163	[1]
multiple_product_multiple_shipment_simple.json	sonnet-4-5	db_plus_raw	1	1.0000	8.3078	[5]
multiple_product_multiple_shipment_simple.json	sonnet-4-5	raw_only	1	1.0000	8.0564	[6]
multiple_product_multiple_shipment_simple.json	sonnet-4-5	zero_shot	1	1.0000	5.9383	[7]
multiple_product_multiple_shipment_simple.json	sonnet-4-6	db_only	1	1.0000	7.8850	[6]
multiple_product_multiple_shipment_simple.json	sonnet-4-6	db_plus_raw	1	1.0000	8.3081	[3]
multiple_product_multiple_shipment_simple.json	sonnet-4-6	raw_only	1	1.0000	6.8336	[3]
multiple_product_multiple_shipment_simple.json	sonnet-4-6	zero_shot	1	1.0000	8.1147	[7]
real_world_msgs_test_v1.json	opus-4-5	db_only	1	1.0000	8.3186	[0]
real_world_msgs_test_v1.json	opus-4-5	db_plus_raw	1	1.0000	6.9379	[1]
real_world_msgs_test_v1.json	opus-4-5	raw_only	1	1.0000	25.9336	[1]
real_world_msgs_test_v1.json	opus-4-5	zero_shot	1	1.0000	5.9894	[0]
real_world_msgs_test_v1.json	opus-4-6	db_only	1	1.0000	6.2971	[0]
real_world_msgs_test_v1.json	opus-4-6	db_plus_raw	1	1.0000	6.8210	[0]
real_world_msgs_test_v1.json	opus-4-6	raw_only	1	1.0000	5.3569	[1]
real_world_msgs_test_v1.json	opus-4-6	zero_shot	1	1.0000	5.0713	[3]
real_world_msgs_test_v1.json	sonnet-4-5	db_only	1	1.0000	7.8192	[3]
real_world_msgs_test_v1.json	sonnet-4-5	db_plus_raw	1	1.0000	9.0627	[3]
real_world_msgs_test_v1.json	sonnet-4-5	raw_only	1	1.0000	7.9241	[4]
real_world_msgs_test_v1.json	sonnet-4-5	zero_shot	1	1.0000	6.1579	[3]
real_world_msgs_test_v1.json	sonnet-4-6	db_only	1	1.0000	7.8122	[3]
real_world_msgs_test_v1.json	sonnet-4-6	db_plus_raw	1	1.0000	7.3667	[3]
real_world_msgs_test_v1.json	sonnet-4-6	raw_only	1	1.0000	5.1861	[4]
real_world_msgs_test_v1.json	sonnet-4-6	zero_shot	1	1.0000	5.3736	[6]
real_world_msgs_test_v2.json	opus-4-5	db_only	1	1.0000	8.3792	[4]
real_world_msgs_test_v2.json	opus-4-5	db_plus_raw	1	1.0000	7.0715	[4]
real_world_msgs_test_v2.json	opus-4-5	raw_only	1	1.0000	22.9605	[5]
real_world_msgs_test_v2.json	opus-4-5	zero_shot	1	1.0000	12.2644	[4]
real_world_msgs_test_v2.json	opus-4-6	db_only	1	1.0000	7.8168	[4]
real_world_msgs_test_v2.json	opus-4-6	db_plus_raw	1	1.0000	8.2556	[4]
real_world_msgs_test_v2.json	opus-4-6	raw_only	1	1.0000	7.0224	[5]
real_world_msgs_test_v2.json	opus-4-6	zero_shot	1	1.0000	6.5707	[1]
real_world_msgs_test_v2.json	sonnet-4-5	db_only	1	1.0000	7.3369	[4]
real_world_msgs_test_v2.json	sonnet-4-5	db_plus_raw	1	1.0000	7.6004	[6]
real_world_msgs_test_v2.json	sonnet-4-5	raw_only	1	1.0000	7.7079	[10]
real_world_msgs_test_v2.json	sonnet-4-5	zero_shot	1	1.0000	6.7791	[4]
real_world_msgs_test_v2.json	sonnet-4-6	db_only	1	1.0000	7.4030	[6]
real_world_msgs_test_v2.json	sonnet-4-6	db_plus_raw	1	1.0000	7.3668	[6]
real_world_msgs_test_v2.json	sonnet-4-6	raw_only	1	1.0000	7.3119	[6]
real_world_msgs_test_v2.json	sonnet-4-6	zero_shot	1	1.0000	8.3625	[3]
real_world_msgs_test_v3.json	opus-4-5	db_only	1	1.0000	8.6902	[3]
real_world_msgs_test_v3.json	opus-4-5	db_plus_raw	1	1.0000	7.0972	[3]
real_world_msgs_test_v3.json	opus-4-5	raw_only	1	1.0000	23.7314	[3]
real_world_msgs_test_v3.json	opus-4-5	zero_shot	1	1.0000	10.7949	[0]
real_world_msgs_test_v3.json	opus-4-6	db_only	1	1.0000	9.1579	[3]
real_world_msgs_test_v3.json	opus-4-6	db_plus_raw	1	1.0000	8.4501	[3]
real_world_msgs_test_v3.json	opus-4-6	raw_only	1	1.0000	6.7960	[3]
real_world_msgs_test_v3.json	opus-4-6	zero_shot	1	1.0000	6.3371	[3]
real_world_msgs_test_v3.json	sonnet-4-5	db_only	1	1.0000	7.9646	[3]
real_world_msgs_test_v3.json	sonnet-4-5	db_plus_raw	1	1.0000	9.1159	[3]
real_world_msgs_test_v3.json	sonnet-4-5	raw_only	1	1.0000	8.4864	[6]
real_world_msgs_test_v3.json	sonnet-4-5	zero_shot	1	1.0000	5.5904	[0]
real_world_msgs_test_v3.json	sonnet-4-6	db_only	1	1.0000	6.9338	[3]
real_world_msgs_test_v3.json	sonnet-4-6	db_plus_raw	1	1.0000	8.4382	[3]
real_world_msgs_test_v3.json	sonnet-4-6	raw_only	1	1.0000	7.1819	[3]
real_world_msgs_test_v3.json	sonnet-4-6	zero_shot	1	1.0000	6.2591	[4]
single_product_multiple_shipment_complex.json	opus-4-5	db_only	1	1.0000	10.8782	[4]
single_product_multiple_shipment_complex.json	opus-4-5	db_plus_raw	1	1.0000	8.9548	[5]
single_product_multiple_shipment_complex.json	opus-4-5	raw_only	1	1.0000	27.7049	[5]
single_product_multiple_shipment_complex.json	opus-4-5	zero_shot	1	1.0000	9.9238	[0]
single_product_multiple_shipment_complex.json	opus-4-6	db_only	1	1.0000	8.8681	[1]
single_product_multiple_shipment_complex.json	opus-4-6	db_plus_raw	1	1.0000	10.3277	[4]
single_product_multiple_shipment_complex.json	opus-4-6	raw_only	1	1.0000	8.6917	[3]
single_product_multiple_shipment_complex.json	opus-4-6	zero_shot	1	1.0000	7.1207	[0]
single_product_multiple_shipment_complex.json	sonnet-4-5	db_only	1	1.0000	10.3388	[4]
single_product_multiple_shipment_complex.json	sonnet-4-5	db_plus_raw	1	1.0000	12.2838	[4]
single_product_multiple_shipment_complex.json	sonnet-4-5	raw_only	1	1.0000	9.9719	[6]
single_product_multiple_shipment_complex.json	sonnet-4-5	zero_shot	1	1.0000	7.4448	[1]
single_product_multiple_shipment_complex.json	sonnet-4-6	db_only	1	1.0000	8.8977	[4]
single_product_multiple_shipment_complex.json	sonnet-4-6	db_plus_raw	1	1.0000	11.9766	[4]
single_product_multiple_shipment_complex.json	sonnet-4-6	raw_only	1	1.0000	9.6924	[4]
single_product_multiple_shipment_complex.json	sonnet-4-6	zero_shot	1	1.0000	7.5174	[0]
single_product_multiple_shipment_medium.json	opus-4-5	db_only	1	1.0000	9.0966	[3]
single_product_multiple_shipment_medium.json	opus-4-5	db_plus_raw	1	1.0000	8.5394	[3]
single_product_multiple_shipment_medium.json	opus-4-5	raw_only	1	1.0000	22.3543	[3]
single_product_multiple_shipment_medium.json	opus-4-5	zero_shot	1	1.0000	9.1793	[1]
single_product_multiple_shipment_medium.json	opus-4-6	db_only	1	1.0000	8.4056	[2]
single_product_multiple_shipment_medium.json	opus-4-6	db_plus_raw	1	1.0000	10.3520	[3]
single_product_multiple_shipment_medium.json	opus-4-6	raw_only	1	1.0000	6.9889	[3]
single_product_multiple_shipment_medium.json	opus-4-6	zero_shot	1	1.0000	6.0001	[0]
single_product_multiple_shipment_medium.json	sonnet-4-5	db_only	1	1.0000	9.4677	[3]
single_product_multiple_shipment_medium.json	sonnet-4-5	db_plus_raw	1	1.0000	7.8498	[3]
single_product_multiple_shipment_medium.json	sonnet-4-5	raw_only	1	1.0000	7.2117	[5]
single_product_multiple_shipment_medium.json	sonnet-4-5	zero_shot	1	1.0000	6.6741	[1]
single_product_multiple_shipment_medium.json	sonnet-4-6	db_only	1	1.0000	6.6728	[3]
single_product_multiple_shipment_medium.json	sonnet-4-6	db_plus_raw	1	1.0000	8.3305	[3]
single_product_multiple_shipment_medium.json	sonnet-4-6	raw_only	1	1.0000	6.6118	[3]
single_product_multiple_shipment_medium.json	sonnet-4-6	zero_shot	1	1.0000	7.4008	[4]
single_product_multiple_shipment_simple.json	opus-4-5	db_only	1	1.0000	9.0824	[3]
single_product_multiple_shipment_simple.json	opus-4-5	db_plus_raw	1	1.0000	7.5320	[3]
single_product_multiple_shipment_simple.json	opus-4-5	raw_only	1	1.0000	28.1662	[4]
single_product_multiple_shipment_simple.json	opus-4-5	zero_shot	1	1.0000	7.4948	[4]
single_product_multiple_shipment_simple.json	opus-4-6	db_only	1	1.0000	8.3154	[13]
single_product_multiple_shipment_simple.json	opus-4-6	db_plus_raw	1	1.0000	10.2154	[3]
single_product_multiple_shipment_simple.json	opus-4-6	raw_only	1	1.0000	6.8777	[3]
single_product_multiple_shipment_simple.json	opus-4-6	zero_shot	1	1.0000	5.9368	[1]
single_product_multiple_shipment_simple.json	sonnet-4-5	db_only	1	1.0000	8.0143	[3]
single_product_multiple_shipment_simple.json	sonnet-4-5	db_plus_raw	1	1.0000	8.5885	[3]
single_product_multiple_shipment_simple.json	sonnet-4-5	raw_only	1	1.0000	6.7301	[3]
single_product_multiple_shipment_simple.json	sonnet-4-5	zero_shot	1	1.0000	5.9569	[1]
single_product_multiple_shipment_simple.json	sonnet-4-6	db_only	1	1.0000	7.0443	[3]
single_product_multiple_shipment_simple.json	sonnet-4-6	db_plus_raw	1	1.0000	10.1632	[3]
single_product_multiple_shipment_simple.json	sonnet-4-6	raw_only	1	1.0000	7.6634	[3]
single_product_multiple_shipment_simple.json	sonnet-4-6	zero_shot	1	1.0000	5.6089	[0]
single_product_single_shipment_complex.json	opus-4-5	db_only	1	1.0000	8.2430	[4]
single_product_single_shipment_complex.json	opus-4-5	db_plus_raw	1	1.0000	6.4775	[4]
single_product_single_shipment_complex.json	opus-4-5	raw_only	1	1.0000	20.3950	[5]
single_product_single_shipment_complex.json	opus-4-5	zero_shot	1	1.0000	6.9078	[4]
single_product_single_shipment_complex.json	opus-4-6	db_only	1	1.0000	6.7621	[5]
single_product_single_shipment_complex.json	opus-4-6	db_plus_raw	1	1.0000	7.6525	[4]
single_product_single_shipment_complex.json	opus-4-6	raw_only	1	1.0000	5.6598	[4]
single_product_single_shipment_complex.json	opus-4-6	zero_shot	1	1.0000	5.1428	[4]
single_product_single_shipment_complex.json	sonnet-4-5	db_only	1	1.0000	6.4033	[6]
single_product_single_shipment_complex.json	sonnet-4-5	db_plus_raw	1	1.0000	6.5115	[6]
single_product_single_shipment_complex.json	sonnet-4-5	raw_only	1	1.0000	5.6168	[8]
single_product_single_shipment_complex.json	sonnet-4-5	zero_shot	1	1.0000	5.7477	[6]
single_product_single_shipment_complex.json	sonnet-4-6	db_only	1	1.0000	6.6317	[6]
single_product_single_shipment_complex.json	sonnet-4-6	db_plus_raw	1	1.0000	7.1440	[6]
single_product_single_shipment_complex.json	sonnet-4-6	raw_only	1	1.0000	5.9534	[6]
single_product_single_shipment_complex.json	sonnet-4-6	zero_shot	1	1.0000	7.5932	[4]
single_product_single_shipment_medium.json	opus-4-5	db_only	1	1.0000	8.6304	[3]
single_product_single_shipment_medium.json	opus-4-5	db_plus_raw	1	1.0000	6.2610	[3]
single_product_single_shipment_medium.json	opus-4-5	raw_only	1	1.0000	19.5976	[2]
single_product_single_shipment_medium.json	opus-4-5	zero_shot	1	1.0000	4.9974	[1]
single_product_single_shipment_medium.json	opus-4-6	db_only	1	1.0000	7.8182	[4]
single_product_single_shipment_medium.json	opus-4-6	db_plus_raw	1	1.0000	7.0910	[3]
single_product_single_shipment_medium.json	opus-4-6	raw_only	1	1.0000	5.0971	[3]
single_product_single_shipment_medium.json	opus-4-6	zero_shot	1	1.0000	5.2821	[1]
single_product_single_shipment_medium.json	sonnet-4-5	db_only	1	1.0000	6.6754	[3]
single_product_single_shipment_medium.json	sonnet-4-5	db_plus_raw	1	1.0000	6.8331	[3]
single_product_single_shipment_medium.json	sonnet-4-5	raw_only	1	1.0000	5.0267	[2]
single_product_single_shipment_medium.json	sonnet-4-5	zero_shot	1	1.0000	5.3599	[4]
single_product_single_shipment_medium.json	sonnet-4-6	db_only	1	1.0000	6.2800	[3]
single_product_single_shipment_medium.json	sonnet-4-6	db_plus_raw	1	1.0000	9.1710	[3]
single_product_single_shipment_medium.json	sonnet-4-6	raw_only	1	1.0000	5.2559	[3]
single_product_single_shipment_medium.json	sonnet-4-6	zero_shot	1	1.0000	4.6565	[3]
single_product_single_shipment_simple.json	opus-4-5	db_only	1	1.0000	8.2063	[0]
single_product_single_shipment_simple.json	opus-4-5	db_plus_raw	1	1.0000	6.4471	[0]
single_product_single_shipment_simple.json	opus-4-5	raw_only	1	1.0000	18.5153	[0]
single_product_single_shipment_simple.json	opus-4-5	zero_shot	1	1.0000	5.5874	[0]
single_product_single_shipment_simple.json	opus-4-6	db_only	1	1.0000	6.3977	[0]
single_product_single_shipment_simple.json	opus-4-6	db_plus_raw	1	1.0000	7.2009	[0]
single_product_single_shipment_simple.json	opus-4-6	raw_only	1	1.0000	4.7336	[0]
single_product_single_shipment_simple.json	opus-4-6	zero_shot	1	1.0000	4.6261	[0]
single_product_single_shipment_simple.json	sonnet-4-5	db_only	1	1.0000	6.4154	[0]
single_product_single_shipment_simple.json	sonnet-4-5	db_plus_raw	1	1.0000	6.1572	[0]
single_product_single_shipment_simple.json	sonnet-4-5	raw_only	1	1.0000	4.5352	[0]
single_product_single_shipment_simple.json	sonnet-4-5	zero_shot	1	1.0000	4.9849	[3]
single_product_single_shipment_simple.json	sonnet-4-6	db_only	1	1.0000	5.8431	[15]
single_product_single_shipment_simple.json	sonnet-4-6	db_plus_raw	1	1.0000	8.0557	[0]
single_product_single_shipment_simple.json	sonnet-4-6	raw_only	1	1.0000	4.7110	[0]
single_product_single_shipment_simple.json	sonnet-4-6	zero_shot	1	1.0000	4.8118	[0]

Top Mismatches (up to 100 runs)

Chat	Model	Strategy	Run	Mismatch count	Sample mismatches
multiple_product_multiple_shipment_complex.json	sonnet-4-5	zero_shot	1	21	[ { "path": "data[0].items[0].quantity_unit", "expected": "bags", "actual": "BAGS" }, { "path": "data[0].items[0].pricing_unit", "expected": "USD/bag", "actual": "USD/BAG" }, { "path": "data[0].items[0].total", "expected": 300.0, "actual": 276.0 }, { "path": "data[0].items[1].quantity_unit", "expected": "boxes", "actual": "BOXES" }, { "path": "data[0].items[1].unit_price", "expected": 12.0, "actual": 11.04 } ]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-6	db_only	1	20	[ { "path": "data", "expected_len": 1, "actual_len": 2 }, { "path": "data[0].items[0].description", "expected": "TG - BP102", "actual": "lecithin fat powder" }, { "path": "data[0].items[0].quantity", "expected": 23000.0, "actual": 8.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 1410.0, "actual": 12.0 } ]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-6	db_only	1	20	[ { "path": "data", "expected_len": 1, "actual_len": 2 }, { "path": "data[0].items[0].description", "expected": "TG - BP102", "actual": "lecithin fat powder" }, { "path": "data[0].items[0].quantity", "expected": 46000.0, "actual": 8.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 1410.0, "actual": 12.0 } ]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-5	zero_shot	1	19	[ { "path": "data[0].items[0].description", "expected": "TG - BP102", "actual": "BP102" }, { "path": "data[0].items[0].quantity", "expected": 46000.0, "actual": 23.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Busan" } ]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-6	db_plus_raw	1	19	[ { "path": "data", "expected_len": 1, "actual_len": 2 }, { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "lecithin fat powder" }, { "path": "data[0].items[0].quantity", "expected": 10500.0, "actual": 8.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3100.0, "actual": 12.0 } ]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	sonnet-4-6	zero_shot	1	18	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 9000.0, "actual": 9.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3.25, "actual": 3250.0 }, { "path": "data[0].items[0].pricing_unit", "expected": "USD/KG", "actual": "USD/MT" } ]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-6	raw_only	1	18	[ { "path": "data", "expected_len": 1, "actual_len": 3 }, { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "Bergapur" }, { "path": "data[0].items[0].quantity", "expected": 6300.0, "actual": 10.5 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3050.0, "actual": 3.05 } ]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-5	zero_shot	1	17	[ { "path": "data[0].items[0].description", "expected": "TG - BP102", "actual": "BP102" }, { "path": "data[0].items[0].quantity", "expected": 23000.0, "actual": 23.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Busan" } ]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-6	db_only	1	17	[ { "path": "data[0].items[0].description", "expected": "TG - BP102", "actual": "BP102" }, { "path": "data[0].items[0].quantity", "expected": 46000.0, "actual": 23.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Busan" } ]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-5	db_plus_raw	1	17	[ { "path": "data[0].items[0].description", "expected": "TG - BP102", "actual": "BP102" }, { "path": "data[0].items[0].quantity", "expected": 46000.0, "actual": 23.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Busan" } ]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-5	db_only	1	17	[ { "path": "data[0].items[0].description", "expected": "TG - BP102", "actual": "BP102" }, { "path": "data[0].items[0].quantity", "expected": 46000.0, "actual": 23.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Busan" } ]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-6	zero_shot	1	17	[ { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "" }, { "path": "data[0].items[0].quantity", "expected": 10500.0, "actual": 10.5 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3100.0, "actual": null }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "" } ]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-6	raw_only	1	17	[ { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "" }, { "path": "data[0].items[0].quantity", "expected": 10500.0, "actual": 10.5 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3100.0, "actual": null }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "" } ]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-6	db_plus_raw	1	17	[ { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "" }, { "path": "data[0].items[0].quantity", "expected": 10500.0, "actual": 10.5 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3100.0, "actual": null }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "" } ]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-6	db_only	1	17	[ { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "" }, { "path": "data[0].items[0].quantity", "expected": 10500.0, "actual": 10.5 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3100.0, "actual": null }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "" } ]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-5	raw_only	1	17	[ { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "" }, { "path": "data[0].items[0].quantity", "expected": 10500.0, "actual": 10.5 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3100.0, "actual": null }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "" } ]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-6	raw_only	1	17	[ { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "" }, { "path": "data[0].items[0].quantity", "expected": 10500.0, "actual": 10.5 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3100.0, "actual": null }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "" } ]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-6	db_only	1	17	[ { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "" }, { "path": "data[0].items[0].quantity", "expected": 10500.0, "actual": 10.5 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3100.0, "actual": null }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "" } ]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-5	raw_only	1	17	[ { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "" }, { "path": "data[0].items[0].quantity", "expected": 10500.0, "actual": 10.5 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3100.0, "actual": null }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "" } ]
07__2025-12-23__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-5	db_plus_raw	1	17	[ { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "" }, { "path": "data[0].items[0].quantity", "expected": 10500.0, "actual": 10.5 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3100.0, "actual": null }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "" } ]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-6	db_plus_raw	1	17	[ { "path": "data", "expected_len": 1, "actual_len": 3 }, { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "Bergapur" }, { "path": "data[0].items[0].quantity", "expected": 6300.0, "actual": 10.5 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3050.0, "actual": 3.05 } ]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-6	db_only	1	17	[ { "path": "data", "expected_len": 1, "actual_len": 3 }, { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "Bergapur" }, { "path": "data[0].items[0].quantity", "expected": 6300.0, "actual": 10.5 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3050.0, "actual": 3.05 } ]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-5	zero_shot	1	17	[ { "path": "data", "expected_len": 1, "actual_len": 3 }, { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "Bergapur" }, { "path": "data[0].items[0].quantity", "expected": 6300.0, "actual": 10.5 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3050.0, "actual": 3.05 } ]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-5	db_plus_raw	1	17	[ { "path": "data", "expected_len": 1, "actual_len": 3 }, { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "Bergapur" }, { "path": "data[0].items[0].quantity", "expected": 6300.0, "actual": 10500.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "KG" }, { "path": "data[0].items[0].unit_price", "expected": 3050.0, "actual": 3.05 } ]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-5	db_only	1	17	[ { "path": "data", "expected_len": 1, "actual_len": 3 }, { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "Bergapur" }, { "path": "data[0].items[0].quantity", "expected": 6300.0, "actual": 10.5 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3050.0, "actual": 3.05 } ]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-6	raw_only	1	17	[ { "path": "data", "expected_len": 1, "actual_len": 3 }, { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "Bergapur" }, { "path": "data[0].items[0].quantity", "expected": 6300.0, "actual": 10500.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "KG" }, { "path": "data[0].items[0].unit_price", "expected": 3050.0, "actual": 3.05 } ]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-6	db_plus_raw	1	17	[ { "path": "data", "expected_len": 1, "actual_len": 3 }, { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "Bergapur" }, { "path": "data[0].items[0].quantity", "expected": 6300.0, "actual": 10.5 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3050.0, "actual": 3.05 } ]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-5	zero_shot	1	17	[ { "path": "data", "expected_len": 1, "actual_len": 3 }, { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "Bergapur" }, { "path": "data[0].items[0].quantity", "expected": 6300.0, "actual": 10.5 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3050.0, "actual": 3.05 } ]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-5	raw_only	1	17	[ { "path": "data", "expected_len": 1, "actual_len": 3 }, { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "Bergapur" }, { "path": "data[0].items[0].quantity", "expected": 6300.0, "actual": 10.5 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3050.0, "actual": 3.05 } ]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-5	db_plus_raw	1	17	[ { "path": "data", "expected_len": 1, "actual_len": 3 }, { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "Bergapur" }, { "path": "data[0].items[0].quantity", "expected": 6300.0, "actual": 10.5 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3050.0, "actual": 3.05 } ]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-5	db_only	1	17	[ { "path": "data", "expected_len": 1, "actual_len": 3 }, { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "Bergapur" }, { "path": "data[0].items[0].quantity", "expected": 6300.0, "actual": 10.5 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3050.0, "actual": 3.05 } ]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-6	db_only	1	16	[ { "path": "data[0].items[0].description", "expected": "TG - BP102", "actual": "BP102" }, { "path": "data[0].items[0].quantity", "expected": 23000.0, "actual": 23.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Busan" } ]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-5	db_plus_raw	1	16	[ { "path": "data[0].items[0].description", "expected": "TG - BP102", "actual": "BP102" }, { "path": "data[0].items[0].quantity", "expected": 46000.0, "actual": 23.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Busan" } ]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	sonnet-4-5	zero_shot	1	16	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 9000.0, "actual": 9.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF Jakarta", "actual": "CIF" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Jakarta" } ]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-6	zero_shot	1	16	[ { "path": "data", "expected_len": 1, "actual_len": 2 }, { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "Bergapur" }, { "path": "data[0].items[0].quantity", "expected": 6300.0, "actual": 6.3 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3050.0, "actual": 3.05 } ]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	sonnet-4-5	raw_only	1	16	[ { "path": "data", "expected_len": 1, "actual_len": 3 }, { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "Bergapur" }, { "path": "data[0].items[0].quantity", "expected": 6300.0, "actual": 10500.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "KG" }, { "path": "data[0].items[0].unit_price", "expected": 3050.0, "actual": 3.05 } ]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-6	zero_shot	1	16	[ { "path": "data", "expected_len": 1, "actual_len": 2 }, { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "Bergapur" }, { "path": "data[0].items[0].quantity", "expected": 6300.0, "actual": 6.3 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3050.0, "actual": 3.05 } ]
03__2026-01-30__120363403074656566_g_us__8f477a8f-2a60-4e0a-bf0e-8cc3cdf1dc9f.json	opus-4-6	db_only	1	16	[ { "path": "data", "expected_len": 1, "actual_len": 2 }, { "path": "data[0].items[0].description", "expected": "BergaPur", "actual": "Bergapur" }, { "path": "data[0].items[0].quantity", "expected": 6300.0, "actual": 6.3 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3050.0, "actual": 3.05 } ]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-6	raw_only	1	16	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3250.0, "actual": 3.25 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
single_product_single_shipment_simple.json	sonnet-4-6	db_only	1	15	[ { "path": "data[0].items[0].description", "expected": "KNM Coffee", "actual": "BP102" }, { "path": "data[0].items[0].quantity", "expected": 5.0, "actual": 23.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "BAGS", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 25.0, "actual": 1410.0 }, { "path": "data[0].items[0].pricing_unit", "expected": "USD/BAG", "actual": "USD/MT" } ]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-6	db_plus_raw	1	15	[ { "path": "data[0].items[0].description", "expected": "TG - BP102", "actual": "BP102" }, { "path": "data[0].items[0].quantity", "expected": 23000.0, "actual": 23.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Busan" } ]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-5	db_plus_raw	1	15	[ { "path": "data[0].items[0].description", "expected": "TG - BP102", "actual": "BP102" }, { "path": "data[0].items[0].quantity", "expected": 23000.0, "actual": 23.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Busan" } ]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-5	db_only	1	15	[ { "path": "data[0].items[0].description", "expected": "TG - BP102", "actual": "BP102" }, { "path": "data[0].items[0].quantity", "expected": 23000.0, "actual": 23.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Busan" } ]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-6	db_plus_raw	1	15	[ { "path": "data[0].items[0].description", "expected": "TG - BP102", "actual": "BP102" }, { "path": "data[0].items[0].quantity", "expected": 23000.0, "actual": 23.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Busan" } ]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-5	db_plus_raw	1	15	[ { "path": "data[0].items[0].description", "expected": "TG - BP102", "actual": "BP102" }, { "path": "data[0].items[0].quantity", "expected": 23000.0, "actual": 23.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Busan" } ]
09__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-5	db_only	1	15	[ { "path": "data[0].items[0].description", "expected": "TG - BP102", "actual": "BP102" }, { "path": "data[0].items[0].quantity", "expected": 23000.0, "actual": 23.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Busan" } ]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-6	zero_shot	1	15	[ { "path": "data[0].items[0].description", "expected": "TG - BP102", "actual": "BP102" }, { "path": "data[0].items[0].quantity", "expected": 46000.0, "actual": 23.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Busan" } ]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-6	db_plus_raw	1	15	[ { "path": "data[0].items[0].description", "expected": "TG - BP102", "actual": "BP102" }, { "path": "data[0].items[0].quantity", "expected": 46000.0, "actual": 23.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Busan" } ]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	sonnet-4-5	db_only	1	15	[ { "path": "data[0].items[0].description", "expected": "TG - BP102", "actual": "BP102" }, { "path": "data[0].items[0].quantity", "expected": 46000.0, "actual": 23.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Busan" } ]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-6	db_plus_raw	1	15	[ { "path": "data[0].items[0].description", "expected": "TG - BP102", "actual": "BP102" }, { "path": "data[0].items[0].quantity", "expected": 46000.0, "actual": 23.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Busan" } ]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-6	raw_only	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOIFNE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 26000.0, "actual": 26.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF NHAVA SHEVA", "actual": "CIF" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Nhava Sheva" } ]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-6	db_plus_raw	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOIFNE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 26000.0, "actual": 26.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF NHAVA SHEVA", "actual": "CIF" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Nhava Sheva" } ]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-6	raw_only	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 26000.0, "actual": 26.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF NHAVA SHEVA", "actual": "CIF" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Nhava Sheva" } ]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-5	raw_only	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOIFNE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 26000.0, "actual": 26.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF NHAVA SHEVA", "actual": "CIF" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Nhava Sheva" } ]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-5	db_plus_raw	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOIFNE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 26000.0, "actual": 26.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF NHAVA SHEVA", "actual": "CIF" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Nhava Sheva" } ]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	sonnet-4-6	db_plus_raw	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 9000.0, "actual": 9.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF Jakarta", "actual": "CIF" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Jakarta" } ]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	opus-4-6	zero_shot	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 9000.0, "actual": 9.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF Jakarta", "actual": "CIF" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Jakarta" } ]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	opus-4-6	db_only	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 9000.0, "actual": 9.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF Jakarta", "actual": "CIF" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Jakarta" } ]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	opus-4-5	zero_shot	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 9000.0, "actual": 9.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF Jakarta", "actual": "CIF" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Jakarta" } ]
02__2026-02-09__120363426578757754_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-6	zero_shot	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "Soya Lecithin Powder" }, { "path": "data[0].items[0].quantity", "expected": 39000.0, "actual": 39.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF Nhava Sheva", "actual": "CIF" } ]
02__2026-02-09__120363426578757754_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-6	raw_only	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "Soya Lecithin Powder" }, { "path": "data[0].items[0].quantity", "expected": 39000.0, "actual": 39.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF Nhava Sheva", "actual": "CIF" } ]
02__2026-02-09__120363426578757754_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-6	db_plus_raw	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "Soya Lecithin Powder" }, { "path": "data[0].items[0].quantity", "expected": 39000.0, "actual": 39.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF Nhava Sheva", "actual": "CIF" } ]
02__2026-02-09__120363426578757754_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-6	db_only	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "Soya Lecithin Powder" }, { "path": "data[0].items[0].quantity", "expected": 39000.0, "actual": 39.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF Nhava Sheva", "actual": "CIF" } ]
02__2026-02-09__120363426578757754_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-5	db_plus_raw	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "Soya Lecithin Powder" }, { "path": "data[0].items[0].quantity", "expected": 39000.0, "actual": 39.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF Nhava Sheva", "actual": "CIF" } ]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-6	zero_shot	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "EXW" } ]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-6	db_plus_raw	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3250.0, "actual": 3.25 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-6	db_only	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3250.0, "actual": 3.25 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-5	zero_shot	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3250.0, "actual": 3.25 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-5	db_only	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3250.0, "actual": 3.25 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-6	zero_shot	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3250.0, "actual": 3.25 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-6	db_plus_raw	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3250.0, "actual": 3.25 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-6	db_only	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3250.0, "actual": 3.25 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-5	zero_shot	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3250.0, "actual": 3.25 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-5	raw_only	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3250.0, "actual": 3.25 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-5	db_plus_raw	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3250.0, "actual": 3.25 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
01__2026-02-24__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-5	db_only	1	15	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3250.0, "actual": 3.25 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
08__2025-09-29__120363403592950429_g_us__d586d853-694c-42f9-93be-bc7ba5b2110c.json	opus-4-5	zero_shot	1	14	[ { "path": "data[0].items[0].description", "expected": "TG - BP102", "actual": "BP102" }, { "path": "data[0].items[0].quantity", "expected": 46000.0, "actual": 23.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/MT" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Busan" } ]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-6	db_plus_raw	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3500.0, "actual": 3.5 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-6	db_only	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3500.0, "actual": 3.5 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-5	zero_shot	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3500.0, "actual": 3.5 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	sonnet-4-5	db_only	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3500.0, "actual": 3.5 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-6	zero_shot	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3500.0, "actual": 3.5 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-6	db_plus_raw	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3500.0, "actual": 3.5 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-6	db_only	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3500.0, "actual": 3.5 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-5	zero_shot	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3500.0, "actual": 3.5 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-5	raw_only	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3500.0, "actual": 3.5 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-5	db_plus_raw	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3500.0, "actual": 3.5 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
06__2026-01-06__120363421131250401_g_us__e05574ec-b110-4554-9fc3-3abb4f9011a8.json	opus-4-5	db_only	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 1800.0, "actual": 1.8 }, { "path": "data[0].items[0].quantity_unit", "expected": "kg", "actual": "MT" }, { "path": "data[0].items[0].unit_price", "expected": 3500.0, "actual": 3.5 }, { "path": "data[0].items[0].pricing_unit", "expected": "usd/mt", "actual": "USD/KG" } ]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-6	zero_shot	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOIFNE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 26000.0, "actual": 26.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF NHAVA SHEVA", "actual": "CIF" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Nhava Sheva" } ]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-6	db_only	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOIFNE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 26000.0, "actual": 26.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF NHAVA SHEVA", "actual": "CIF" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Nhava Sheva" } ]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-5	db_plus_raw	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOIFNE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 26000.0, "actual": 26.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF NHAVA SHEVA", "actual": "CIF" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Nhava Sheva" } ]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	sonnet-4-5	db_only	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOIFNE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 26000.0, "actual": 26.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF NHAVA SHEVA", "actual": "CIF" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Nhava Sheva" } ]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-6	db_plus_raw	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOIFNE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 26000.0, "actual": 26.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF NHAVA SHEVA", "actual": "CIF" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Nhava Sheva" } ]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-5	zero_shot	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOIFNE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 26000.0, "actual": 26.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF NHAVA SHEVA", "actual": "CIF" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Nhava Sheva" } ]
05__2026-01-20__120363407382355715_g_us__12a4f3a7-d506-4d32-ae06-3f76508c6abd.json	opus-4-5	db_only	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOIFNE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 26000.0, "actual": 26.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF NHAVA SHEVA", "actual": "CIF" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Nhava Sheva" } ]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	sonnet-4-6	db_only	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 9000.0, "actual": 9.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF Jakarta", "actual": "CIF" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Jakarta" } ]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	sonnet-4-5	db_plus_raw	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 9000.0, "actual": 9.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF Jakarta", "actual": "CIF" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Jakarta" } ]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	sonnet-4-5	db_only	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 9000.0, "actual": 9.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF Jakarta", "actual": "CIF" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Jakarta" } ]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	opus-4-6	raw_only	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 9000.0, "actual": 9.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF Jakarta", "actual": "CIF" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Jakarta" } ]
04__2026-01-29__120363408498669191_g_us__4b9c2faa-94dd-4236-abcc-398807051f21.json	opus-4-6	db_plus_raw	1	14	[ { "path": "data[0].items[0].description", "expected": "GIIOFINE - P - S", "actual": "GIIOFINE-P-S" }, { "path": "data[0].items[0].quantity", "expected": 9000.0, "actual": 9.0 }, { "path": "data[0].items[0].quantity_unit", "expected": "KG", "actual": "MT" }, { "path": "data[0].items[0].ship_term", "expected": "CIF Jakarta", "actual": "CIF" }, { "path": "data[0].items[0].delivery_terms", "expected": "", "actual": "CIF Jakarta" } ]