← Index

Multilingual Chat Moderation

Research Engineer, Singapore Institute of Technology · 2026

AWS Bedrock · RAG · prompt caching · Python · load testing · Little's Law

F1 with RAG
0.91
F1 without RAG
0.66
Input-token cost
-92%
p95 latency
< 4 s
Throughput
~2 msg/s per container

Problem

Chat moderation has to work in the language people actually type. Singlish, Chinese, Malay, and Tamil carry slang, code-switching, and context that a standard classifier misses. The baseline scored 0.66 F1 on our RabakBench test set — good enough for a demo, not for a system that has to make a call on every message.

Constraint

Three constraints fought each other: accuracy across four languages, cost per message at real volume, and a provider with a hard throughput ceiling. Fine-tuning one model per language would have been expensive to build and expensive to serve. Scaling blindly would have hit the provider limit before we knew where our own system stopped being correct.

Decision

I built a retrieval-augmented pipeline on AWS Bedrock: instead of retraining, the prompt is enriched with relevant labelled examples at inference time, so the model sees the slang patterns it is being asked to judge. Stable prompt prefixes are cached to avoid paying for the same tokens on every call.

Concurrency was treated as a measured quantity. I ran staged load tests from 1 to 16 parallel clients and set the usable operating point where p95 stays under 4 seconds — about 2 messages per second per container. The provider ceiling of roughly 115 messages per second was established through AWS quota checks and Little’s Law, not by pushing until something broke.

Verification

  • F1 on RabakBench: 0.91 with RAG, 0.66 without.
  • Input-token cost reduced by about 92% with prompt caching.
  • p95 under 4 s at the declared concurrency, verified across staged load runs.
  • Provider ceiling derived independently from quota data and queueing theory, so the capacity claim has two sources.

Result

A multilingual moderation pipeline that is accurate, cheap enough to run continuously, and has a documented operating envelope. Co-authored paper under review at ICEED.

The transferable lesson: the RAG layer was the easy part. Knowing where the system stops being correct and where the provider stops being able to serve it was the engineering.