---
title: "Tech Tip #16: Use the smallest AI model that does the job"
description: "You're using a giant AI model for a tiny job."
url: https://greencode.ai/tech-tips/16-smallest-model-that-does-the-job/
---

# Tech Tip #16: Use the smallest AI model that does the job

**You're using a giant AI model for a tiny job.**

[Watch the video](https://media.greencode.ai/shorts/smallest-model-that-does-the-job-80f446f5084a.mp4)

Video created by [explainapaper.com](https://explainapaper.com/?utm_source=greencode.ai&utm_medium=film&utm_campaign=made-by)

### The problem

General-purpose generative models get used for classification, extraction and routing that smaller or task-specific models do well. A systematic comparison found multi-purpose generative models orders of magnitude more expensive to run than task-specific ones for the same tasks.

### The fix you can use today

- Start with the smallest model, or a classical method, that meets the quality bar.
- Route requests: small model by default, large model only for the hard cases.
- Measure energy per request for each option before you choose.

A UNESCO report found that small models made for one task can cut energy use by up to 90%. GreenCode's own AI uses small, tuned models for the same reason.

### Tools that help

- [AI Energy Score](https://huggingface.github.io/AIEnergyScore/?utm_source=greencode.ai&utm_medium=referral&utm_campaign=greencode) (Hugging Face): comparable energy ratings for AI models on common tasks
- [CodeCarbon](https://github.com/mlco2/codecarbon?utm_source=greencode.ai&utm_medium=referral&utm_campaign=greencode): a Python package that estimates the energy and CO2 of a run from hardware power and regional grid intensity
- [nvidia-smi](https://developer.nvidia.com/system-management-interface?utm_source=greencode.ai&utm_medium=referral&utm_campaign=greencode): reports the power draw of NVIDIA GPUs from the card's own sensors

### How GreenCode helps

GreenCode helps tackle this by routing each request to the smallest component that can handle it, using small fine-tuned models. Retrieval means a model reads only the data it needs. [More on this](https://greencode.ai/feature/green-optimised-generative-ai/).

### Tags

#llm#model-efficiency#inference

### Sources

- [What policy makers can do (Luccioni, Jernite, Strubell, FAccT 2024)](https://greencode.ai/news/policy-software-and-ai-energy/)
- [GreenCode State of the Art Review, D2.1 (2025), PDF p. 53: small-first model selection and routing; reserve larger models for genuinely hard cases](https://greencode.ai/publications/d2-1-state-of-the-art-review/)
- [UNESCO, small task-specific models can cut energy use by up to 90%](https://www.unesco.org/en/articles/ai-large-language-models-new-report-shows-small-changes-can-reduce-energy-use-90?utm_source=greencode.ai&utm_medium=referral&utm_campaign=greencode)
- [Green Software Foundation, Green Software Patterns: Right-sized, energy-efficient AI models](https://github.com/Green-Software-Foundation/patterns/blob/main/docs/development/right-sized-energy-efficient-ai-models.md?utm_source=greencode.ai&utm_medium=referral&utm_campaign=greencode)

[All GreenCode Tech Tips](https://greencode.ai/tech-tips/)

#### Key Facts

- Tip **#16**
- For **CTOs and technical leads**
- Category **AI and machine learning**
- Format **short video**
![GreenCode](https://greencode.ai/wp-content/uploads/2024/12/greencode-logo-website-small-colours-2.webp)

Want this checked on your own system? Talk to the GreenCode team.

[Get in touch](https://greencode.ai/contact/)
