Walkero

Almost two years ago, I created my Ko-Fi page where I could blog about my work on personal projects I am working on in my free time. Most of them cover topics about the Amiga community and operating systems like AmigaOS and MorphOS. Ko-Fi.com is a nice platform, easy to use and express yourself.

One of the things I have been missing all these years is a way for anyone to follow my posts, even if they are not registered on that platform. The best way to do it is using RSS, but unfortunately, Ko-Fi.com doesn't provide them. I get it. They prefer people to register just to follow someone and potentially become their customers, but platforms like Ko-Fi.com are for people to communicate what they love to do. In my opinion, every website should have RSS available because it is convenient for people to get informed about the latest blog posts using the applications of their choice.

So, I had to do something about it. I was looking for a tool that could be useful to me. Different online apps grab content from websites that do not provide any RSS and create one. Some didn't work with Ko-Fi.com at all and others were quite expensive, requiring a monthly fee just to have an RSS feed.

In the end, I found an exceptional project called RSS-Bridge, which does exactly what I need. I just had to write some code. Not a big deal. We are developers, after all, right?

RSS-Bridge will generate feeds based on “bridges” that are developed for any site. Those bridges will collect data and extract all necessary information which is then converted into various feed formats like Atom or RSS.

RSS-Bridge is based on PHP and uses simplehtmldom to parse a website DOM to an object. What I needed to do was to write a bridge for Ko-Fi.com and parse the content. It sounds easier than it is. You see the content appears using JavaScript XHR requests, while it is broken into different URL paths.

But before that, I had to create a development environment where I could do the coding. I used Docker and created a docker-compose.yml file on my local system to build a container. At first, I did that on an Arm based computer and the first problem appeared. Although RSS-Bridge was working fine, I couldn't get any data, and the reason was that Ko-Fi.com uses Cloudflare CDN. This is something that a lot of people had issues with in the past. RSS-Bridge solves that problem by using a special build of curl that can impersonate the four major browsers: Chrome, Edge, Safari & Firefox. But unfortunately, that library doesn't work well on Arm-based systems, so I had to move to my trusty Intel-based Linux computer.

version: '3'
services:
  rss-bridge:
    container_name: "rssbridge_server"
    image: rssbridge/rss-bridge:latest

I used VS Code and the Docker plugins to get inside the container and do the coding I needed, so that they are reflected instantly on the browser, without the need to rebuild the container. I started reading the RSS-Bridge documentation and creating the first lines of the new bridge. All done pretty easily and sooner than later I had the basic code working. So now I had to parse the pages, which worked fine, having the DOM data available for me. But the main Ko-Fi.com page doesn't have all the posts I needed. I had to look deeper on the website and the HTTP requests it uses. Gladly found one that returns the latest blog posts, a few words from the intro and the main image. That's a good start, and after a few minutes fighting with the DOM I had the first RSS working. But there were some issues.

Firstly, the intro text was tiny, just a few words. And secondly, the post dates were all relative, in a way that after a few months in the past they were all the same, without being able to distinguish when exactly the articles were posted. Every post had a date like "3 months ago". Unfortunately, that information isn't available anywhere than the main article page. I decided to try and parse that one as well, and get a more accurate date and the first paragraph of the whole text. That worked great but made the parser a little bit slower because instead of parsing one page, it had to parse 30. So, I limited it to the last 10 posts to keep the parsing fast, and now it was ready for production.

I set up a Docker container on one of my servers. But how would I push that bridge inside the container? I checked the documentation and the official RSS_Bridge dockerfile and I discovered that there is an entrypoint script that checks the /config folder and gets any configuration and bridge file that it finds, copying them to the proper place. And that happens when the container is created. So I created a volume and put those files in a config folder in the host system. And since I use Traefik for reverse proxy, I added its labels at the end. I do not care for that to work with SSL, so I skipped those labels.

I rebuilt the container and everything worked flawlessly.

version: '3'
services:
  rss-bridge:
    container_name: "rssbridge_server"
    image: rssbridge/rss-bridge:latest
    volumes:
      - ./config:/config
    restart: unless-stopped
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.rssbridge_nginx.rule=Host(`rssbridge.walkero.gr`)"

But I couldn't stop there. Some kind of caching should be set so that it won't scan the Ko-Fi.com website on every RSS request and risk of getting my server IP banned. RSS-Bridge supports different ways of caching and I choose to use memcache which I set up in the docker-compose.yml file.

  memcached:
    container_name: "rssbridge_memcached"
    image: wodby/memcached
    restart: unless-stopped

Since everything worked as intended, I created a pull request to the RSS-Bridge repository and after a few hours my code was merged and now every public server will have a new Ko-Fi bridge for everyone to use. And since this is something I will use for myself, I will make sure to maintain it as long as possible.

RSS-Bridge is a great tool that anyone can use, self-host, expand and give back to the community so that we will help our online entity to be more free and expressive, reaching as much people as possible, without the need to be tied into a registration system.

If you have any ideas or proposals on how to make it even better, please contact me and I will do my best to make that happen. And if my work is useful to you, I'd love to hear about.

Oh, I forgot. My blogs' RSS feeds are:

• https://walkero.gr/.rss
• http://rssbridge.walkero.gr/?action=display&bridge=KoFiBridge&pageId=walkero&format=Atom