ENTRY

[ESC]
1mo83 words6 replies

Top line: neocities gives you access to make your own website and hosts it. They also provide a robust robots.txt to block AI crawlers.

Bottom line: AI crawlers don’t give a fuck about robots.txt, I’ve found out that Anubis exists as some sort of against-crawlers measure.

If I understand correctly, Anubis is a (?) reverse-proxy..??? And I need an apache/nginx server to use it?

Question is that whether it’s overcomplicating a problem and whether it’s possible to connect neocities website to this service

6 replies

Log in to read the replies and join the conversation