Martin's corner on the web

Getting my old tweets onto my own site

X removed free read access to its API in 2023, and the RSS bridge I run to show my X feed on this site has broken two or three times a year since, every time X shuffles its internals. Each fix was a reminder that the ~1150 tweets I’d posted over the years live on someone else’s servers, one policy change from vanishing. They’re my words; I wanted them in my own database on my own domain.

So I built an importer that turns the official X archive export into native WordPress posts. The interesting part wasn’t parsing the ZIP; it was deciding what counts as “mine”: self-threads get merged into one post dated to the first tweet, replies to other people are dropped, retweets are dropped. Quote tweets stay, with the quoted text rendered inline – someone else’s words, but my choice to share them. That took 1150 tweets down to 518 posts.

Media was the fight. The archive ships every image and video as local files, so I sideload them into the media library instead of hotlinking twimg.com – video hotlinking is already referer-blocked there, and images could be next. Video-only tweets had no thumbnail at all, so ffmpeg pulls a poster frame from each one; without that, 39 posts showed up blank in the listing. And every media-only tweet with no text collapsed onto the same “Untitled” slug until titles learned to fall back to plain dates.

Threads needed a bit of care too. When I reply to my own tweet months later, sometimes contradicting it, the merged post gets a marker, or it reads as one continuous thought. That’s a bug you only notice by reading your own history back.

Text-only imported post: a question about separate IoT networks, a reply marker reading 1 day later, and the poll results
A 2019 tweet as it renders today – the question, and the poll results merged in a day later.

The archive is a one-time dump, and I keep tweeting, so I added a Chrome extension that syncs single tweets on demand: it grabs the thread data x.com already loads in the page, signs it, and pushes it to my server, which files it as a post like any other. No re-exporting the whole archive to catch up.

The two paragraphs above hide most of the actual code. “An importer” is really ten PHP classes around a SQLite state database, so every step is resumable, and re-runs cost nothing: titles and media uploads are cached, imported threads are skipped, and a second dedup check against WordPress post meta survives even losing that database. The extension is sneakier than it sounds too – it never calls X’s API. It wraps window.fetch on x.com and captures the thread data the page fetches for itself, which sidesteps the rotating query hashes and tokens that kept breaking my RSS bridge. From there, the tweet travels HMAC-signed to a small Python daemon on my server, gets parsed from GraphQL shape into the same structure the archive parser emits, and lands in the WordPress-writing code the bulk import uses. About 4,700 lines of PHP, Python, and JavaScript for what I just described in two sentences.

The bulk run: 518 posts, 0 failed, about 14 minutes, with ffmpeg as the bottleneck rather than anything clever. One delete –confirm removes exactly what it created.

The code was written by an AI agent at my terminal – I didn’t hand-write it. What’s mine is the design: the concept, the architecture, the rules for what counts as a post, the review at each step. The only AI in the finished posts is one gpt-5.4-mini call per post to turn the tweet into a title. Deciding what counts as a post took longer than the code did.

My whole X history is now browsable under the X post category, on the same WordPress that minted an uninvited admin last week. Still mine, though.

Tagged on:

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.