GripProbe CLI-Agent Compatibility Matrix

This report measures real CLI-agent × model compatibility by executed outcomes (not text-only responses). Rows are agent/version + model/hash + format groups; cells link to measured cases, with color-coded outcomes, normalized Score, Typical Time (median), and Outliers.

Project repository

Test descriptions

One row per cli-agent-version/model/hash/format group. Test cells link to a concrete case detail page. Group links open the source run summary.

Model CLI Agent Show partial (sanity-only) runs in addition to extended test runs

CLI Agent	Model	Format	Score	Typical Time	Outliers	Patch File	Patch File Codex Apply Patch	Python File Simple	Shell Date	Shell PWD	JSON Rank From File	Patch File From Prepared Patch	Python File Quoted	Python File Quoted DE	Python File Quoted RU	Save File	Shell Date DE	Shell Date RU	Shell File	Web Fetch JSON Raw	Web Nonce Proof	Web Search JSON Ranked	Weekly Plan Next Week	Runs
aider0.86.2	local/MFDoom/deepseek-r1-tool-calling:8b3aa3d24	markdown	25.0%	19.4s	0/4	PASS16.3s	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 11:00
aider0.86.2	local/gemma3:12bf4031aa	markdown	25.0%	8.6s	0/4	PASS9.5s	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 11:10
aider0.86.2	local/gemma4:e2b7fbdbf8	markdown	0.0%	28.7s	2/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 11:14
aider0.86.2	local/gemma4:e4bc6eb396	markdown	0.0%	16.2s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 11:25
aider0.86.2	local/gpt-oss:20b17052f9	markdown	25.0%	42.7s	3/4	PASS83.6s	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 11:30
aider0.86.2	local/granite4:3b89962fc	markdown	25.0%	7.9s	0/4	PASS7.6s	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 11:04
aider0.86.2	local/ministral-3:8b1922acc	markdown	0.0%	7.4s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 11:58
aider0.86.2	local/mistral-nemo:12be7e06d1	markdown	25.0%	7.9s	0/4	PASS8.6s	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 11:54
aider0.86.2	local/mistral-small:24b8039dd9	markdown	25.0%	11.8s	0/4	PASS14.9s	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 11:43
aider0.86.2	local/nemotron-3-nano:4b6cc467f	markdown	25.0%	8.9s	0/4	PASS9.6s	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 12:01
aider0.86.2	local/orieg/gemma3-tools:12b-ft-v213652b1	markdown	0.0%	8.0s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 12:03
aider0.86.2	local/phi3.5:latest61819fb	markdown	0.0%	24.1s	2/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 11:06
aider0.86.2	local/pleasecech/qwen3.6-plus:latest3f477a7	markdown	25.0%	24.1s	1/4	PASS14.2s	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-11 12:09
aider0.86.2	local/qwen2.5:7b845dbda	markdown	25.0%	7.4s	0/4	PASS10.1s	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 10:52
aider0.86.2	local/qwen3:8b500a1f0	markdown	50.0%	17.1s	0/4	PASS14.4s	-	PASS19.7s	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 10:55
codex0.128.0	local/aravhawk/qwen3.5-opus-4.6-text:9b74348a6	tool	89.4%	20.4s	2/18	FAIL	PASS68.5s	PASS21.9s	PASS17.6s	PASS7.9s	PASS21.5s	FAIL	PASS7.7s	PASS10.6s	PASS7.5s	PASS25.6s	PASS9.0s	PASS11.0s	PASS7.0s	PASS19.3s	PASS45.2s	PASS47.8s	PASS40.5s	2026-05-10 12:58, 2026-05-10 21:16, 2026-05-10 22:00
codex0.128.0	local/cryptidbleh/gemma4-claude-opus-4.6:latest0961fd3	tool	49.4%	6.6s	0/18	FAIL	FAIL	PASS6.3s	PASS6.1s	PASS5.8s	FAIL	FAIL	PASS5.7s	PASS6.8s	PASS6.2s	FAIL	PASS6.0s	PASS5.5s	PASS6.4s	FAIL	FAIL	FAIL	FAIL	2026-05-02 15:33, 2026-05-03 06:05, 2026-05-10 20:43
codex0.128.0	local/pleasecech/qwen3.6-plus:latest3f477a7	tool	0.0%	0.7s	0/5	FAIL	FAIL	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-11 11:12
codex0.128.0	local/qwen3.5:9b6488c96	tool	0.0%	120.0s	15/18	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	FAIL	2026-05-10 13:09, 2026-05-10 20:47
continue-cli1.5.45	local/MFDoom/deepseek-r1-tool-calling:8b3aa3d24	tool	0.0%	4.8s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-21 16:44
continue-cli1.5.45	local/aravhawk/qwen3.5-opus-4.6-text:9b74348a6	markdown	100.0%	12.0s	0/17	PASS19.1s	-	PASS15.5s	PASS10.9s	PASS11.8s	PASS29.9s	PASS12.0s	PASS11.2s	PASS11.7s	PASS11.5s	PASS15.1s	PASS10.6s	PASS9.4s	PASS8.7s	PASS26.5s	PASS31.3s	PASS50.4s	PASS18.5s	2026-04-26 08:16, 2026-04-26 08:45, 2026-04-26 10:45, 2026-05-01 07:39
continue-cli1.5.45	local/aravhawk/qwen3.5-opus-4.6-text:9b74348a6	tool	100.0%	15.0s	2/17	PASS95.5s	-	PASS12.7s	PASS18.3s	PASS12.2s	PASS23.5s	PASS63.9s	PASS12.3s	PASS13.3s	PASS11.5s	PASS15.0s	PASS11.7s	PASS8.8s	PASS13.7s	PASS38.4s	PASS57.1s	PASS55.9s	PASS22.2s	2026-04-24 10:52, 2026-04-24 15:14, 2026-04-26 10:45, 2026-04-26 11:14, 2026-05-01 07:39, 2026-05-01 07:45
continue-cli1.5.45	local/cryptidbleh/gemma4-claude-opus-4.6:latest0961fd3	markdown	58.0%	36.4s	3/17	FAIL	-	PASS8.3s	PASS7.2s	PASS7.0s	FAIL	FAIL	PASS36.7s	PASS36.4s	PASS36.7s	PASS35.6s	PASS35.1s	PASS35.8s	PASS35.3s	FAIL	FAIL	FAIL	FAIL	2026-05-01 22:02, 2026-05-02 12:52
continue-cli1.5.45	local/cryptidbleh/gemma4-claude-opus-4.6:latest0961fd3	tool	75.0%	7.7s	0/4	FAIL	-	PASS7.5s	PASS7.9s	PASS7.5s	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-02 08:37
continue-cli1.5.45	local/fredrezones55/qwen3.5-opus:27b914789b	markdown	0.0%	600.0s	4/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-01 20:42
continue-cli1.5.45	local/fredrezones55/qwen3.5-opus:27b914789b	tool	0.0%	600.0s	4/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-02 07:17
continue-cli1.5.45	local/fredrezones55/qwen3.5-opus:9b8d16063	markdown	100.0%	13.5s	1/4	PASS42.1s	-	PASS14.6s	PASS12.4s	PASS11.9s	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-01 20:38
continue-cli1.5.45	local/fredrezones55/qwen3.5-opus:9b8d16063	tool	100.0%	17.0s	1/4	PASS54.0s	-	PASS18.9s	PASS11.6s	PASS15.1s	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-02 07:13
continue-cli1.5.45	local/gemma3:12bf4031aa	markdown	0.0%	2.4s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-21 17:45
continue-cli1.5.45	local/gemma3:12bf4031aa	tool	0.0%	3.2s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-02 08:40
continue-cli1.5.45	local/gemma4:e2b7fbdbf8	markdown	75.3%	9.8s	0/17	PASS10.1s	-	PASS7.1s	PASS9.4s	PASS9.8s	FAIL	PASS16.5s	PASS9.4s	PASS8.1s	PASS7.6s	FAIL	PASS9.3s	PASS7.1s	PASS7.0s	PASS13.8s	FAIL	FAIL	PASS27.9s	2026-04-23 13:03, 2026-04-23 15:34, 2026-04-24 08:46, 2026-04-25 09:44, 2026-04-25 21:40
continue-cli1.5.45	local/gemma4:e2b7fbdbf8	tool	100.0%	8.3s	0/4	PASS10.6s	-	PASS8.6s	PASS6.7s	PASS7.9s	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-02 08:41
continue-cli1.5.45	local/gemma4:e4bc6eb396	markdown	75.3%	15.0s	0/17	PASS9.0s	-	PASS13.8s	PASS11.7s	PASS5.8s	PASS136.9s	FAIL	PASS15.0s	PASS27.8s	PASS14.7s	PASS25.1s	PASS14.4s	PASS13.8s	PASS14.8s	FAIL	PASS43.3s	FAIL	FAIL	2026-04-23 13:06, 2026-04-23 15:38, 2026-04-24 08:47, 2026-04-25 09:47, 2026-04-26 05:09
continue-cli1.5.45	local/gemma4:e4bc6eb396	tool	100.0%	14.3s	1/4	PASS38.6s	-	PASS15.7s	PASS10.6s	PASS12.9s	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-02 08:43
continue-cli1.5.45	local/gpt-oss:20b17052f9	markdown	87.7%	186.7s	17/17	PASS210.5s	-	PASS164.6s	PASS112.6s	PASS138.0s	PASS298.9s	FAIL	PASS162.7s	PASS186.7s	PASS306.5s	PASS122.5s	PASS168.6s	PASS127.8s	PASS136.3s	FAIL	PASS430.9s	PASS533.6s	PASS365.3s	2026-04-23 13:12, 2026-04-23 15:50, 2026-04-24 08:51, 2026-04-25 09:57, 2026-04-26 05:30
continue-cli1.5.45	local/gpt-oss:20b17052f9	tool	75.0%	218.5s	4/4	PASS292.8s	-	PASS144.2s	PASS124.5s	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-02 08:48
continue-cli1.5.45	local/granite3-dense:8b-instruct-q6_Kec258d5	markdown	0.0%	8.5s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-21 16:55
continue-cli1.5.45	local/granite4:3b89962fc	tool	75.0%	4.8s	0/4	PASS5.4s	-	FAIL	PASS4.4s	PASS4.8s	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-02 08:39
continue-cli1.5.45	local/ministral-3:8b1922acc	tool	75.3%	6.1s	0/17	PASS7.6s	-	PASS4.6s	PASS4.3s	PASS4.0s	FAIL	FAIL	PASS5.5s	PASS5.3s	PASS6.1s	FAIL	PASS4.8s	PASS5.0s	PASS5.3s	PASS11.0s	FAIL	PASS16.0s	PASS8.2s	2026-04-23 14:29, 2026-04-24 09:13, 2026-04-25 10:25, 2026-04-26 06:18
continue-cli1.5.45	local/mistral-nemo:12be7e06d1	markdown	0.0%	11.9s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-21 19:19
continue-cli1.5.45	local/mistral-nemo:12be7e06d1	tool	0.0%	12.2s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-21 19:13
continue-cli1.5.45	local/mistral-small:24b8039dd9	tool	0.0%	584.0s	4/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-02 09:25
continue-cli1.5.45	local/nemotron-3-nano:4b6cc467f	tool	75.3%	10.2s	0/17	PASS24.6s	-	PASS8.3s	PASS7.0s	PASS6.9s	PASS18.5s	PASS18.4s	PASS10.2s	PASS9.0s	PASS10.2s	FAIL	PASS7.9s	PASS12.5s	PASS8.6s	FAIL	PASS20.1s	FAIL	FAIL	2026-04-23 14:32, 2026-04-24 09:15, 2026-04-25 10:27, 2026-04-26 06:20
continue-cli1.5.45	local/orieg/gemma3-tools:12b-ft-v213652b1	markdown	50.0%	133.7s	4/4	FAIL	-	FAIL	PASS133.5s	PASS133.9s	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-21 17:50
continue-cli1.5.45	local/orieg/gemma3-tools:12b-ft-v213652b1	tool	0.0%	600.0s	4/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-02 10:36
continue-cli1.5.45	local/phi3.5:latest61819fb	tool	0.0%	2.7s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-02 08:40
continue-cli1.5.45	local/pleasecech/qwen3.6-plus:latest3f477a7	markdown	0.0%	4.4s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-11 12:19
continue-cli1.5.45	local/pleasecech/qwen3.6-plus:latest3f477a7	tool	0.0%	4.3s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-11 11:12
continue-cli1.5.45	local/qwen2.5:7b845dbda	markdown	56.8%	5.8s	0/17	PASS8.0s	-	PASS5.2s	PASS4.9s	PASS4.5s	FAIL	FAIL	FAIL	PASS6.2s	PASS5.8s	PASS6.2s	PASS5.7s	PASS4.9s	PASS5.8s	FAIL	FAIL	FAIL	FAIL	2026-04-23 12:53, 2026-04-23 15:18, 2026-04-24 08:40, 2026-04-25 09:39, 2026-04-25 21:07
continue-cli1.5.45	local/qwen2.5:7b845dbda	tool	75.0%	5.2s	0/4	FAIL	-	PASS5.2s	PASS5.3s	PASS5.1s	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-02 06:10
continue-cli1.5.45	local/qwen3.5:9b6488c96	markdown	82.7%	20.2s	1/17	FAIL	-	PASS14.1s	PASS10.9s	PASS11.1s	PASS42.4s	PASS44.8s	PASS14.5s	PASS15.7s	PASS20.2s	PASS21.9s	PASS15.0s	PASS12.2s	PASS12.4s	FAIL	PASS69.0s	PASS255.3s	FAIL	2026-04-25 21:20, 2026-04-27 12:41, 2026-05-01 07:37
continue-cli1.5.45	local/qwen3.5:9b6488c96	tool	93.8%	16.1s	3/17	PASS59.2s	-	PASS15.5s	PASS17.2s	PASS9.7s	PASS202.6s	PASS38.8s	PASS12.6s	PASS13.1s	PASS13.3s	PASS16.1s	PASS13.8s	PASS12.7s	PASS13.1s	FAIL	PASS56.6s	PASS117.1s	PASS52.6s	2026-04-24 10:49, 2026-04-24 15:07, 2026-05-01 08:20, 2026-05-01 20:35
continue-cli1.5.45	local/qwen3:14bbdbd181	markdown	75.0%	229.3s	4/4	FAIL	-	PASS200.0s	PASS177.3s	PASS258.7s	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-01 19:52
continue-cli1.5.45	local/qwen3:14bbdbd181	tool	50.0%	477.9s	4/4	FAIL	-	FAIL	PASS355.7s	PASS186.2s	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-02 06:17
continue-cli1.5.45	local/qwen3:8b500a1f0	markdown	81.5%	25.8s	4/17	PASS26.3s	-	PASS27.4s	PASS17.2s	PASS13.7s	FAIL	PASS62.9s	PASS19.4s	PASS16.3s	PASS24.3s	PASS20.0s	PASS25.0s	PASS25.0s	PASS25.8s	PASS234.9s	FAIL	FAIL	PASS88.6s	2026-04-23 12:55, 2026-04-23 15:20, 2026-04-24 08:41, 2026-04-25 09:40, 2026-04-25 21:08
continue-cli1.5.45	local/qwen3:8b500a1f0	tool	100.0%	30.8s	1/4	PASS26.8s	-	PASS34.7s	PASS14.3s	PASS39.2s	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-02 06:11
continue-cli1.5.45	local/yi-coder:9b-chat-q5_K_Md4b375d	markdown	0.0%	2.1s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-21 16:53
continue-cli1.5.45	local/yi-coder:9b-chat-q5_K_Md4b375d	tool	0.0%	2.1s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-21 16:51
gptmeunknown	local/aravhawk/qwen3.5-opus-4.6-text:9b74348a6	tool	76.9%	62.4s	7/13	-	-	-	-	-	PASS85.2s	FAIL	PASS57.8s	PASS47.8s	PASS62.4s	PASS43.1s	PASS70.0s	PASS50.2s	PASS46.4s	PASS85.2s	FAIL	FAIL	PASS113.6s	2026-04-24 15:43, 2026-04-26 07:17, 2026-05-01 14:45
gptmeunknown	local/cryptidbleh/gemma4-claude-opus-4.6:latest0961fd3	tool	64.2%	18.4s	0/17	FAIL	-	PASS18.6s	PASS17.9s	PASS19.1s	PASS68.7s	FAIL	PASS17.0s	FAIL	PASS17.9s	PASS16.6s	PASS19.8s	PASS18.4s	PASS17.2s	PASS42.5s	FAIL	FAIL	FAIL	2026-05-02 12:32, 2026-05-02 13:12
gptmeunknown	local/fredrezones55/qwen3.5-opus:27b914789b	tool	0.0%	120.0s	3/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-02 12:18
gptmeunknown	local/fredrezones55/qwen3.5-opus:9b8d16063	tool	75.0%	40.7s	3/4	FAIL	-	PASS42.4s	PASS39.0s	PASS44.7s	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-02 12:13
gptmeunknown	local/gemma4:e2b7fbdbf8	tool	25.0%	56.5s	0/4	-	-	-	-	-	FAIL	-	-	-	-	-	-	-	-	PASS38.8s	-	FAIL	FAIL	2026-04-25 10:29, 2026-04-26 06:22
gptmeunknown	local/gemma4:e4bc6eb396	tool	0.0%	120.0s	1/4	-	-	-	-	-	FAIL	-	-	-	-	-	-	-	-	FAIL	-	FAIL	FAIL	2026-04-25 10:33, 2026-04-26 06:29
gptmeunknown	local/granite4:3b89962fc	tool	50.0%	10.3s	0/4	FAIL	-	FAIL	PASS9.4s	PASS10.4s	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-02 12:35
gptmeunknown	local/ministral-3:8b1922acc	tool	75.0%	18.7s	0/4	-	-	-	-	-	PASS18.0s	-	-	-	-	-	-	-	-	PASS14.9s	-	PASS51.1s	FAIL	2026-04-25 10:40, 2026-04-26 06:41
gptmeunknown	local/mistral-nemo:12be7e06d1	tool	36.1%	48.3s	4/8	FAIL	-	FAIL	PASS48.8s	PASS43.5s	FAIL	-	-	-	-	-	-	-	-	PASS37.9s	-	FAIL	FAIL	2026-04-25 10:44, 2026-04-26 06:44, 2026-05-01 22:05, 2026-05-02 04:47
gptmeunknown	local/mistral-nemo:12bunknown	tool	0.0%	0.0s	0/1	-	-	-	-	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-01 22:05
gptmeunknown	local/orieg/gemma3-tools:12b-ft-v213652b1	tool	0.0%	120.0s	4/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-21 21:43
gptmeunknown	local/pleasecech/qwen3.6-plus:latest3f477a7	markdown	0.0%	120.0s	4/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-11 12:20
gptmeunknown	local/pleasecech/qwen3.6-plus:latest3f477a7	tool	0.0%	5.8s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-11 11:13
gptmeunknown	local/qwen2.5:7b845dbda	tool	44.4%	16.2s	0/8	PASS12.9s	-	PASS14.0s	PASS14.6s	PASS10.0s	FAIL	-	-	-	-	-	-	-	-	FAIL	-	FAIL	FAIL	2026-04-25 10:51, 2026-04-26 06:50, 2026-05-02 04:54
gptmeunknown	local/qwen3.5:9b6488c96	tool	76.9%	76.2s	4/13	-	-	-	-	-	PASS76.2s	FAIL	PASS108.6s	FAIL	PASS43.8s	PASS29.9s	FAIL	PASS32.3s	PASS32.8s	PASS87.2s	PASS117.6s	PASS112.0s	PASS56.1s	2026-04-24 15:22, 2026-04-26 07:07, 2026-05-01 14:13
gptmeunknown	local/qwen3:14bbdbd181	tool	0.0%	120.0s	4/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-02 11:57
gptmeunknown	local/qwen3:8b500a1f0	tool	53.1%	59.9s	4/7	PASS59.9s	-	PASS40.1s	PASS50.1s	-	FAIL	-	-	-	-	-	-	-	-	PASS57.8s	-	FAIL	FAIL	2026-04-25 10:55, 2026-04-26 06:57, 2026-05-02 04:57
gptmeunknown	local/qwen3:8bunknown	tool	0.0%	0.0s	0/1	-	-	-	-	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-02 04:57
gptmev0.31.0+unknown	local/MFDoom/deepseek-r1-tool-calling:8b3aa3d24	tool	0.0%	21.7s	1/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-21 20:46
gptmev0.31.0+unknown	local/aravhawk/qwen3.5-opus-4.6-text:9b74348a6	tool	75.0%	37.6s	2/4	FAIL	-	PASS53.6s	PASS50.4s	PASS24.7s	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-24 11:06
gptmev0.31.0+unknown	local/gemma3:12bf4031aa	tool	0.0%	2.6s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-21 20:54
gptmev0.31.0+unknown	local/gemma4:e2b7fbdbf8	tool	77.0%	22.0s	0/13	FAIL	-	PASS28.1s	PASS18.9s	PASS24.0s	-	FAIL	PASS23.4s	PASS22.3s	PASS21.6s	PASS24.3s	PASS21.0s	PASS20.3s	PASS22.0s	-	FAIL	-	-	2026-04-23 18:23, 2026-04-24 09:16
gptmev0.31.0+unknown	local/gemma4:e4bc6eb396	tool	77.0%	27.2s	3/13	FAIL	-	PASS25.2s	PASS25.1s	PASS31.9s	-	FAIL	PASS34.4s	PASS27.2s	PASS47.3s	PASS22.6s	PASS28.3s	PASS35.7s	PASS26.0s	-	FAIL	-	-	2026-04-23 18:31, 2026-04-24 09:20
gptmev0.31.0+unknown	local/gpt-oss:20b17052f9	tool	0.0%	120.0s	4/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-21 21:02
gptmev0.31.0+unknown	local/granite3-dense:8b-instruct-q6_Kec258d5	tool	0.0%	24.7s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-21 20:49
gptmev0.31.0+unknown	local/ministral-3:8b1922acc	tool	85.2%	10.3s	0/13	FAIL	-	PASS11.1s	PASS9.4s	PASS8.8s	-	FAIL	PASS12.2s	PASS13.1s	PASS11.4s	PASS8.8s	PASS10.3s	PASS10.2s	PASS10.4s	-	PASS18.2s	-	-	2026-04-23 18:43, 2026-04-24 09:24
gptmev0.31.0+unknown	local/mistral-nemo:12be7e06d1	tool	60.7%	34.9s	5/13	FAIL	-	PASS27.7s	PASS32.3s	PASS34.7s	-	FAIL	PASS28.0s	FAIL	PASS22.2s	FAIL	PASS76.2s	PASS38.0s	PASS34.9s	-	FAIL	-	-	2026-04-21 21:35, 2026-04-23 18:47
gptmev0.31.0+unknown	local/mistral-small:24b8039dd9	tool	0.0%	120.0s	4/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-21 21:18
gptmev0.31.0+unknown	local/nemotron-3-nano:4b6cc467f	tool	0.0%	11.1s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-21 21:41
gptmev0.31.0+unknown	local/phi3.5:latest61819fb	tool	0.0%	2.1s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-21 20:54
gptmev0.31.0+unknown	local/qwen2.5:7b845dbda	tool	83.6%	11.1s	0/13	PASS6.9s	-	PASS11.1s	PASS11.2s	PASS7.5s	-	FAIL	PASS16.1s	PASS12.9s	PASS10.9s	PASS9.7s	PASS12.9s	PASS11.0s	PASS10.9s	-	FAIL	-	-	2026-04-21 20:36, 2026-04-23 19:00
gptmev0.31.0+unknown	local/qwen3.5:9b6488c96	tool	25.0%	75.9s	3/4	FAIL	-	FAIL	PASS31.8s	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-24 10:56
gptmev0.31.0+unknown	local/qwen3:8b500a1f0	tool	77.0%	49.3s	11/13	FAIL	-	PASS42.6s	PASS36.8s	PASS52.2s	-	PASS42.9s	PASS91.5s	FAIL	PASS54.1s	PASS49.3s	PASS45.5s	PASS47.5s	PASS48.7s	-	FAIL	-	-	2026-04-21 20:37, 2026-04-23 19:07
gptmev0.31.0+unknown	local/yi-coder:9b-chat-q5_K_Md4b375d	tool	0.0%	2.4s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-21 20:54
opencode1.14.20	local/MFDoom/deepseek-r1-tool-calling:8b3aa3d24	tool	0.0%	13.7s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 12:49
opencode1.14.20	local/aravhawk/qwen3.5-opus-4.6-text:9b74348a6	tool	50.0%	114.8s	4/4	FAIL	-	FAIL	PASS82.3s	PASS109.5s	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-24 11:27
opencode1.14.20	local/gemma3:12bf4031aa	tool	0.0%	2.7s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 12:53
opencode1.14.20	local/gemma4:e2b7fbdbf8	tool	0.0%	14.3s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 12:53
opencode1.14.20	local/gemma4:e4bc6eb396	tool	0.0%	42.6s	3/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 12:56
opencode1.14.20	local/gpt-oss:20b17052f9	tool	0.0%	120.0s	4/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 13:02
opencode1.14.20	local/granite4:3b89962fc	tool	0.0%	5.5s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 12:51
opencode1.14.20	local/ministral-3:8b1922acc	tool	0.0%	9.3s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 13:46
opencode1.14.20	local/mistral-nemo:12be7e06d1	tool	0.0%	95.6s	3/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 13:34
opencode1.14.20	local/mistral-small:24b8039dd9	tool	0.0%	120.0s	4/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 13:18
opencode1.14.20	local/nemotron-3-nano:4b6cc467f	tool	0.0%	26.9s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 13:48
opencode1.14.20	local/orieg/gemma3-tools:12b-ft-v213652b1	tool	0.0%	120.0s	4/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 13:52
opencode1.14.20	local/phi3.5:latest61819fb	tool	0.0%	2.7s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 12:52
opencode1.14.20	local/qwen2.5:7b845dbda	tool	0.0%	9.6s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 12:42
opencode1.14.20	local/qwen3.5:9b6488c96	tool	0.0%	120.0s	4/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-24 11:12
opencode1.14.20	local/qwen3:8b500a1f0	tool	0.0%	22.8s	1/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-04-22 12:44
opencodeunknown	local/pleasecech/qwen3.6-plus:latest3f477a7	markdown	0.0%	3.0s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-11 12:36
opencodeunknown	local/pleasecech/qwen3.6-plus:latest3f477a7	tool	0.0%	3.1s	0/4	FAIL	-	FAIL	FAIL	FAIL	-	-	-	-	-	-	-	-	-	-	-	-	-	2026-05-11 11:14

generated at 2026-05-13 04:15:32 UTC | git commit 487f06d | markdown summary