Text Generation
Transformers
Safetensors
granite
code
qiskit
conversational
Files changed (1) hide show
  1. README.md +61 -0
README.md CHANGED
@@ -70,6 +70,67 @@ for i in output:
70
  print(i)
71
  ```
72
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
73
  ## Training Data
74
 
75
  - **Data Collection and Filtering:** Our code data is sourced from a combination of publicly available datasets (e.g., Code available on <https://github.com>), and additional synthetic data generated at IBM Quantum. We exclude code that is older than 2023.
 
70
  print(i)
71
  ```
72
 
73
+ ### Comparison of Qiskit models across benchmarks
74
+
75
+ <table
76
+ style="
77
+ display: inline-table;
78
+ border-collapse: separate;
79
+ border-spacing: 0;
80
+ font-family: Inter, -apple-system, 'Segoe UI', Roboto, 'Helvetica Neue', Arial, sans-serif;
81
+ box-shadow: 0 6px 18px rgba(12, 20, 29, 0.06);
82
+ border-radius: 12px;
83
+ overflow: hidden;
84
+ table-layout: auto;
85
+ box-sizing: border-box;
86
+ margin: 16px 0;
87
+ "
88
+ >
89
+ <thead>
90
+ <tr>
91
+ <th style="text-align:left; padding:12px 16px; background:linear-gradient(90deg,#f6f8fb,#eef3f9); color:#0b1220; font-weight:700; border-bottom:1px solid rgba(15,23,42,0.06);">
92
+ Model
93
+ </th>
94
+ <th style="text-align:center; padding:12px 16px; background:linear-gradient(90deg,#f6f8fb,#eef3f9); color:#0b1220; font-weight:700; border-bottom:1px solid rgba(15,23,42,0.06);">
95
+ QiskitHumanEval
96
+ </th>
97
+ <th style="text-align:center; padding:12px 16px; background:linear-gradient(90deg,#f6f8fb,#eef3f9); color:#0b1220; font-weight:700; border-bottom:1px solid rgba(15,23,42,0.06);">
98
+ QiskitHumanEval-Hard
99
+ </th>
100
+ <th style="text-align:center; padding:12px 16px; background:linear-gradient(90deg,#f6f8fb,#eef3f9); color:#0b1220; font-weight:700; border-bottom:1px solid rgba(15,23,42,0.06);">
101
+ HumanEval
102
+ </th>
103
+ </tr>
104
+ </thead>
105
+ <tbody>
106
+ <tr style="background:#ffffff;">
107
+ <td style="padding:12px 16px; color:#0f172a;">mistral-small-3.2-24b-qiskit</td>
108
+ <td style="padding:12px 16px; text-align:center; font-family:ui-monospace, SFMono-Regular, Menlo, Monaco, monospace;">20.53</td>
109
+ <td style="padding:12px 16px; text-align:center; font-family:ui-monospace, SFMono-Regular, Menlo, Monaco, monospace;">40.39</td>
110
+ <td style="padding:12px 16px; text-align:center; font-family:ui-monospace, SFMono-Regular, Menlo, Monaco, monospace;">77.49</td>
111
+ </tr>
112
+ <tr style="background:#fbfdff;">
113
+ <td style="padding:12px 16px; color:#0f172a;">granite-3.2-8b-qiskit</td>
114
+ <td style="padding:12px 16px; text-align:center; font-family:ui-monospace, SFMono-Regular, Menlo, Monaco, monospace;">9.93</td>
115
+ <td style="padding:12px 16px; text-align:center; font-family:ui-monospace, SFMono-Regular, Menlo, Monaco, monospace;">24.50</td>
116
+ <td style="padding:12px 16px; text-align:center; font-family:ui-monospace, SFMono-Regular, Menlo, Monaco, monospace;">57.31</td>
117
+ </tr>
118
+ <tr style="background:#ffffff;">
119
+ <td style="padding:12px 16px; color:#0f172a;">granite-3.3-8b-qiskit</td>
120
+ <td style="padding:12px 16px; text-align:center; font-family:ui-monospace, SFMono-Regular, Menlo, Monaco, monospace;">14.56</td>
121
+ <td style="padding:12px 16px; text-align:center; font-family:ui-monospace, SFMono-Regular, Menlo, Monaco, monospace;">27.15</td>
122
+ <td style="padding:12px 16px; text-align:center; font-family:ui-monospace, SFMono-Regular, Menlo, Monaco, monospace;">62.80</td>
123
+ </tr>
124
+ <tr style="background:#f7fafc;">
125
+ <td style="padding:12px 16px; font-weight:700; color:#07102a;">Qwen2.5-Coder-14B-Qiskit</td>
126
+ <td style="padding:12px 16px; text-align:center; font-weight:700; font-family:ui-monospace, SFMono-Regular, Menlo, Monaco, monospace;">25.16</td>
127
+ <td style="padding:12px 16px; text-align:center; font-weight:700; font-family:ui-monospace, SFMono-Regular, Menlo, Monaco, monospace;">49.01</td>
128
+ <td style="padding:12px 16px; text-align:center; font-weight:700; font-family:ui-monospace, SFMono-Regular, Menlo, Monaco, monospace;">91.46</td>
129
+ </tr>
130
+ </tbody>
131
+ </table>
132
+
133
+
134
  ## Training Data
135
 
136
  - **Data Collection and Filtering:** Our code data is sourced from a combination of publicly available datasets (e.g., Code available on <https://github.com>), and additional synthetic data generated at IBM Quantum. We exclude code that is older than 2023.