We can view LLMs and SLMs as two ends of a spectrum with overlap in between. Overall SLMs distinguish themselves from LLMs in one or more of the following ways. SLMs are more fine-tuned because vendors or companies train them on detailed, domain-specific data, for example to assist complex data engineering tasks. They enrich user prompts, for example by injecting domain-specific data into a user’s question to make the response more accurate. Data pipeline vendors are building SLMs with these capabilities now, often alongside LLMs, to help companies tackle specialized data engineering problems with better governance. This will help data teams boost productivity while reducing risks related to data quality, fairness, and explainability. We should get ready for a boom of small language models in data engineering and many other fields.

- Kevin Petrie in Should AI Bots Build Your Data Pipelines? Part III: The Emergence of Small Language Models for Data Engineering

June 21, 2023 (Blog)