↳
In-reply-to
»
I just built a poc search engine / crawler for Twtxt. I managed to crawl this pod (twtxt.net) and a couple of others (sorry @etux and @xuu I used your pods in the tests too!). So far so good. I might keep going with this and see what happens 😀
⤋ Read More
As a quick experiment, I modified my code to remove the domain restrictions and low and behold:
All done!
Crawled 516 feeds
Found 52464 twts
Found 736 feeds
The Twtxt network is larger than I thought. A significant no. of feeds no longer work obviously, but that’s okay, we can prune dead feeds out.