In-reply-to » I just built a poc search engine / crawler for Twtxt. I managed to crawl this pod (twtxt.net) and a couple of others (sorry @etux and @xuu I used your pods in the tests too!). So far so good. I might keep going with this and see what happens 😀

As a quick experiment, I modified my code to remove the domain restrictions and low and behold:

All done!
Crawled 516 feeds
Found 52464 twts
Found 736 feeds

The Twtxt network is larger than I thought. A significant no. of feeds no longer work obviously, but that’s okay, we can prune dead feeds out.

⤋ Read More