See Goodhart’s law: “When a measure becomes a target, it ceases to be a good measure”.

If LLMs are judged by some specific metric meant to track their negative impact, they’ll adapt to make the metric look good without improving the underlying impact.