↳
In-reply-to
»
I just built a poc search engine / crawler for Twtxt. I managed to crawl this pod (twtxt.net) and a couple of others (sorry @etux and @xuu I used your pods in the tests too!). So far so good. I might keep going with this and see what happens 😀
⤋ Read More
@prologic@twtxt.net @etux@twt.u53.us @xuu@txt.sour.is Now I want to remove the “domain” restriction, add a rate-limit and try to crawl as much of the Twtxt wider network as I can and see how deep it goes 🤔