We use NGINX’s 444 on every LLM crawler we see.

Caddy has a similar “close connection” option called “abort” as part of the static response.

HAProxy has the “silent-drop” option which also closes the TCP connection silently.

I’ve found crawling attempts end more quickly using this option - especially attacks - but my sample size is relatively small.

Edit: we do this because too often we’ve seen them ignore robots.txt. They believe all data is theirs. I do not.