<snip>
I've started putting high bandwidth items exclusively under Gopher and
making
the Gopher proxy extremely strict. This has the nice side effect of
increasing,
slowly, the installed base of Gopher clients. There are a few Gopher bots
out
there but they're usually single people, easily tracked down, and typically
apologetic when they realize it went out of control.
The AI bots do none of that, so I have no compunction about blackholing
them in
huge netblocks. If they were willing to work with sites, they wouldn't
constantly toss out new IPs and user agents like germophobes and wet wipes.
that's an interesting approach for sure.
although gopher is probably not a long-term solution, I suppose it does not
matter as sooner or later the WWW will be regional webs, protected by
corps, governments and netizens themselves. Parts of it will be AI swamps
where seldom one ventures and returns unaffected