Voittaa Ltd reposted this
Are you feeding AI with free content 🍟 🍔 🍼 😋 Is your site being used to answer queries on Ai systems? 🤷♂️ Is your IP being used to line the pockets 💰 of the owners of the big AI systems? A simple solution... tell them to bugger off... 🖕 We are having problems with a major e-commerce site we manage where, (usually tied with the addition of new products and therefore new content) the load on the server goes through the roof. All down to one or to IP addresses pinging every page, filter and function on the site, at a speed that has a detrimental effect on the site. Turns out it is AI bots (potentially not listed below but that is a start) that do not index sites graciously like the Google Bot does, but rip content as fast as they can. Remember, this content is YOURS, it is your IP or if not you have been given permission by the IP owner to use it. Why give that to the AI bots just because they need to learn about Trains (in our case)? 🛑 Block them. 🛑 🕸 All websites and web systems should already be using a robots.txt adding these lines to that file will tell those systems they should not index your site. User-agent: CCBot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: GPTBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: Google-CloudVertexBot Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: anthropic-ai Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Omgilibot Disallow: / User-agent: Omgili Disallow: / User-agent: FacebookBot Disallow: / User-agent: Diffbot Disallow: / User-agent: DuckAssistBot Disallow: / User-agent: AI2Bot Disallow: / User-agent: Bytespider Disallow: / User-agent: Kangaroo Bot Disallow: / User-agent: ImagesiftBot Disallow: / User-agent: PerplexityBot Disallow: / User-agent: cohere-ai Disallow: / User-agent: Meta-ExternalAgent Disallow: / User-agent: Meta-ExternalFetcher Disallow: / User-agent: Timpibot Disallow: / User-agent: Webzio-Extended Disallow: / User -agent: YouBot Disallow: / 🔐 We are also looking to implement systems that automatically detect this behaviour and ban those bots from ripping the site. If you need help on this, do not hesitate to contact me. There is also a question about a new form of SEO which involves using the AI systems to your advantage, but I have not finished work on that yet. #Ai #IP #copyright #content #bots #webdevelopment ________ I am Matt, at Veribli we offer smart RFID technology and a platform that allows you to control and automate your production, track your assets, create smart products and build, direct long-lasting relationships with your customers and enabling a circular ecosystem. At Axsi Systems Ltd we track health & safety tasks & inspections for cleanliness, HACCP, fire doors, construction sites and much more. At Voittaa Ltd we do bespoke software and app development, industrial systems, business automation, web software and web design. 📧 info@veribli.com 📲 01482 455855