stopthatgirl7@kbin.social to Technology@kbin.social · 1 year agoNow you can block OpenAI’s web crawlerwww.theverge.comexternal-linkmessage-square4fedilinkarrow-up110arrow-down10file-text
arrow-up110arrow-down1external-linkNow you can block OpenAI’s web crawlerwww.theverge.comstopthatgirl7@kbin.social to Technology@kbin.social · 1 year agomessage-square4fedilinkfile-text
minus-squareNaich@kbin.sociallinkfedilinkarrow-up5·1 year agoIn /robots.txt - User-agent: GPTBot Disallow: / If you don’t want to read the article. Presumably they are doing this now because they have scraped everyone already.
minus-squarebionicjoey@lemmy.calinkfedilinkarrow-up1·1 year agoBetter to ask for forgiveness than to beg for permission
In /robots.txt -
User-agent: GPTBot Disallow: /If you don’t want to read the article. Presumably they are doing this now because they have scraped everyone already.
Better to ask for forgiveness than to beg for permission