Gemma on AI Studio and Vertex: is it production-viable?
Neither option is viable for production chat.
- AI Studio Gemma 4 31B (free): the quota is fixed at 30 requests and 16k input tokens per minute per project, even on our paid Tier 3 project, and there is no paid tier to buy more. Tier 2/3 peak needs about 1.2M input tokens/min. A typical Gemma chat request (18k tokens) is rejected on its own. The model quality is the same 31B we tested: production preferred 46–9.
- Vertex Gemma 4 26B A4B (paid, $0.15/$0.60): it is cheap (about $0.002/answer) and fast when it answers. But in a 30-minute probe 50% of one-at-a-time requests got “queue full” 429s, and quality is far below production. Production was preferred 54/60, 35 of 60 answers had a major or critical issue, it skipped search on 37 of 60 and cited invented passages in 16.
- Vertex Gemma 4 31B is not offered serverless. It would mean dedicated GPU endpoints, which aren't free. Not tested.
If we want Gemma 31B, third-party hosts on OpenRouter remain the only serverless option with real capacity.
Access options
| AI Studio · Gemma 4 31B | Vertex · Gemma 4 26B A4B | |
|---|---|---|
| Price | Free (no paid option exists) | $0.15 / $0.60 per 1M tokens |
| Quota on our project | 30 req/min · 16k input tok/min · 14,400 req/day (same on Tier 3) | Shared pool, no per-project quota shown |
| Status | No SLA | “Experimental”, global only, no SLA |
| Our data used for training? | No, on a billing-enabled project (Paid Services terms) | No (Vertex terms) |
| Thinking control | Only minimal (off) or default (on); no low/medium | OpenAI-style reasoning_effort |
| Tool calling | Works | Works, but often skipped (see quality) |
AI Studio quota vs our tier 2/3 traffic
| Tier 2/3 peak need | AI Studio 31B quota | |
|---|---|---|
| Chat answers / min | ~53 | — |
| Model requests / min | ~106+ (2+ steps per answer) | 30 |
| Input tokens / min | ~1.2M | 16k (~1% of need) |
| Requests / day | ~92k | 14.4k |
| Largest single request (Gemma, median / p90) | 18k / 34k tokens | anything over 16k is rejected outright |
Traffic is Langfuse chat-message traces tagged tier 2/3 on Oct 27: 48,706/day, peak hour 3,395, 94% reached the model. We confirmed the quota live: a 25k-token request got 429 on paid_tier_3_input_token_count after an idle minute, and a 40-request burst got 429s on paid_tier_3_requests after 30.
Latency
| First token | Decode | Errors | |
|---|---|---|---|
| AI Studio 31B, default thinking | 17 s | 39 tok/s | 429 above 30 req/min or 16k tok/min |
| AI Studio 31B, thinking minimal | 1.2 s | 37 tok/s | same quota |
| Vertex 26B (30-min probe, 1 req / 15 s) | 0.5 s (max 29 s) | 48 tok/s | 60/120 = 50% “request queue is full” 429s |
Vertex 26B handled a 20k-token prompt with its first token in 1.4 s. In the chat replay through OpenRouter's Vertex route, retries hid most 429s (1 of 60 failed), but slow turns pushed p90 to 103 s.
Quality on 60 real tier 2/3 chats
| Production (Gemini 3.7 Flash) | Gemma 4 31B | Vertex Gemma 4 26B | |
|---|---|---|---|
| Blind preference vs production | — | prod 46 · tie 5 · Gemma 9 | prod 54 · tie 4 · Gemma 2 |
| Correctness (1–5) | 4.02 | 4.05 | 3.75 |
| Completeness (1–5) | 4.4 | 4.2 | 3.43 |
| Answers with a major/critical issue | 23/60 | 17/60 | 35/60 |
| …of which critical | 1 | 3 | 6 |
| Claims unsupported by cited passage | 15% | 12% | 43% |
| Answers citing passages never retrieved | 0 | 5 | 16 |
| Answered without any search | 8 (greetings etc.) | 12 | 37 |
| Model cost per answer | $0.0171 | $0.0102 | $0.0024 |
| Median time to full answer | 23 s | 40 s | 18 s |
Both Gemma arms used the gemma-v5 prompt, thinking low and Tier 1 retrieval. The 31B column is the OpenRouter run from the previous report. The AI Studio 31B serves the same weights, but its quota made a full replay impossible. Judge: Claude Opus 5.5, grounding against cited passages plus a blind pairwise comparison.