

Exactly! As far as I’m concerned, robots.txt should be enough: I tell your bot to stay the hell away, or not, and your bot obeys. What it scrapes for doesn’t matter, IMO.
We don’t need more standards and rules for assholes to ignore, we need assholes to adhere to the rules.
Running the stuff on someone else’s computer still requires a dedicated team for “something serious”, unless you stuff everything in specific “serverless” platforms, in which case you’re still paying for admins, just not yours.