prologic

Making Twtxt Better

Making Twtxt Better

This post is about how we at Twt.social and [joinTwt[(https://jointwt.org) have attempted to extend the [Twtxt](https://twtxt.readthedocs.org format in ways that we believe are a non-breaking, backwards compatible improvement.

For more information on the project please Get in touch or visit our Github Org.

What is Twtxt anyway?

Twtxt was a format and specification for a decentralised alternative to Twitter™ created by Felix (known as @buckket) who wrote a reference client implementation called twtxt.

At the time Twtxt was touted as:

Twtxt is a decentralised, minimalist microblogging service for hackers.

However we feel that an open decentralised social media platform built on simple concepts should not be relegated to an “elite” group of folks. So when I discovered Twtxt in August of 2020 I immediately saw a lot of potential and set about building what started out as a simple client in the form of a Web App, but quickly grew into a decentralised multi-user Twtxt platform with an API, multiple clients as well as Mobile Apps.

What we did?

There hasn’t been a lot of activity from the original developer that created the format and spec back in ~2016 (it’s not really clear why), and there were some issues that needed to be resolved which can be found at the original project’s issues page for the reference client.

Some of the problems like Discover we have improved upon a little but are still issues that need to be worked on, designed carefully and solved. But for now here is a list of things we have done to either extend the format / spec or the overall behaviour without breaking the format / spec itself.

Standard configuration

Problem: Every client seems to implement their own configuration.

A client’s configuration typically includes what eh user is, what their canonical URL to their feed is and a list of other feeds they follow.

We decided that it would be best if we came up with a “standard” configuration that may clients could agree upon and so based on a combination of what other clients already define as configuration (including Twtxt.net) we came up with the following YAML Spec:

nick: prologic
url: https://twtxt.net/user/prologic/twtxt.txt
following:
  nick: url
  ...

(Note that I omitted my following list for brevity)

You can find an example of this from my own account at twtxt.net:

https://twtxt.net/user/prologic/config.yaml

User / Feed avatars

In order to improve the User eXperience around user/feed profiles we introduced the notion of “Avatars”. That is a user or feed profile having some visual way to represent themselves.

On Twtxt.net this ends up looking like this for local accounts on a Pod1:

Local User Avatar

It even works for external profiles!

So for anyone participating in the wider Twtxt network as a whole that whites to expose an Avatar for their feed may do so by simply dropping an avatar.png at the top-level of where their twtxt.txt feed is located.

For example; if you host your feed at https://mydomain.tld/twtxt.txt then by simply dropping an avatar.png at https://mydomain.tld/avatar.png clients like Twtxt.net will discover this automatically, cache it and use it to display a nice pretty image in place of the default for external feeds:

Subject and Conversations

@felixp7 once proclaimed:

(twtxt.net) Also, it occurs to me that with the parens feature added a while ago @prologic basically invented a subject line syntax for microblogs, something this medium never had before.

What he was referring to are the following features you see on Twtxt.net:

We basically did two things here:

This has the form of (as a regular expression):

^(@<.*>[, ]*)*(\(.*?\))(.*)

This lets us extract any Subject from a Twt, and Twtxt.net uses this to group multiple Twts together to form a “Conversation”. The way this works is every Twt has a hash calculated for it that contains the Timestamp, Author and Content. The hash itself is a Blake2b hash, is is constructed by hashing a string containing:

Author Feed URL
Timestamp
Content

Then this hash is converted to Base32 with no padding, lowered-cased and the last 7 characters taken as the “hash” representation and stored. This is what forms the basis of “Replies” and “Conversations” on Twtxt.net. Any client can adopt this same convention by either preserving the Subject of the Twt they ares replying to (the easiest thing to do) or if replying to a new Twt, constructing the same Hash representation above. In Go this is implemented as:

func (twt Twt) Hash() string {
	if twt.hash != "" {
		return twt.hash
	}

	payload := twt.Twter.URL + "\n" + twt.Created.String() + "\n" + twt.Text
	sum := blake2b.Sum256([]byte(payload))

	// Base32 is URL-safe, unlike Base64, and shorter than hex.
	encoding := base32.StdEncoding.WithPadding(base32.NoPadding)
	hash := strings.ToLower(encoding.EncodeToString(sum[:]))
	twt.hash = hash[len(hash)-HashLength:]

	return twt.hash
}

  1. A Pod is a single instance of Twt.social’s backend software also called twtxt.

Comments:

Recent tws in reply to this post.

You must be Logged in to comment.