Building a Database Using Railway Logs as Storage

Before Railway's 2025 hackathon started, a Railway employee - Brody - gave me the idea of building a database that uses Railway's logs as a storage medium. I thought it was such a unique concept that it immediately became my hackathon project.

The Challenge: Making Logs Work as Storage

Using logs as a storage medium comes with some interesting challenges. First, logs are typically one-directional - there's no easy way of retrieving them unless you captured them ahead of time. Luckily, Railway's GQL API came to the rescue. I could retroactively fetch logs, and what's even better is that Railway's logs GQL API includes filtering features, allowing you to search logs based on specific parameters. This gave me everything I needed to build a functional database.

The second hurdle was storage limitations. Through testing, I discovered that Railway logs can hold about 80kb of data each before JSON parsing breaks. To work around this, I'd need to create a complex chunking mechanism - breaking data into smaller pieces and then reconstructing them later.

Going Dependency-Free (Because Why Not?)

I've always wanted to build a project using zero dependencies, and for some reason I figured now would be the perfect time to try that challenge. So for this hackathon, I didn't use a single external dependency - everything was built from scratch on top of NodeJS.

The Build Process

True to form, I started on Friday the 8th of August 2025 - two days after the hackathon began, because waiting 'til the last second is in my DNA.

On my first day, I built out the project structure and tackled the two most critical components: Railway's API integration and the server. I used the deploymentLogs GQL query to fetch logs with filtering. Friday went well, and I ended the day being able to store and fetch data based on an ID.

Saturday and Sunday were spent wrestling with log filtering. For some odd reason, the filters weren't working properly through the API, even though the exact same filter worked perfectly in Railway's UI search. This really scratched my brain, and I spent way too long on this problem.

On Sunday, I finally decided to switch to Railway's environmentLogs GQL query, which seemingly fixed the issue. In hindsight, I should've been using environmentLogs from the get-go because it spans across deployments, making my database persistent. Not sure why I didn't start with that.

That wrapped up Sunday for me, but knowing I had until Monday night to finish, I "speedran" my 8 hours of sleep within 4 hours and got back to work.

Adding File Uploads and Security

Monday brought the challenge of file uploads. The problem? File uploads require disk storage, but I could only store strings in logs. My solution was to convert files to base64 data, apply gzip compression, and store the compressed result. On read operations, I'd fetch the compressed data, decompress it, and convert it back to the original file before sending it to the client.

I also spent time implementing encryption - something I'd wanted from the start. After all, storing your data as logs isn't exactly the most secure storage method ever, and encryption goes a long way toward keeping data safe.

What I'd Do Differently

There are tons of features I wanted to implement, like a proper database client, but I ran out of time. If I could do anything differently in this hackathon, I'd probably have started sooner. Regardless, it was super fun building this!

Check It Out

This project has a Railway template here:

Feel free to check it out on GitHub here!

The Butterfly that is Alfred

There's a very important point in any project's life where it goes from being in a conceptual cacoon to breaking out and spreading its wings into a vibrant reality. I believe Alfred, my longest lasting project, just reached this point. If you don't know what Alfred is then I'd recommend reading this blog post where I talk about Alfred's history and goals. That blog post was made almost a year ago and since then I've been working on Alfred5 and today I'll be telling you all about it!

Truth is, I finished Alfred5 three months ago. Alfred5 released officially April 27th but since Alfred5 was such a monumental milestone for Alfred's journey, I put off writing this blog post- waiting for a time where I've grown to contain the vocabulary necassery to explain my excitement. I realized now, however, that such a time will likely never come. Alfred5 is truly an extremely important waypoint for Alfred's journey. Why you may ask? Let me show you.

The Importance of this Butterfly

It's hard to put the difference of scale between Alfred5 and Alfred4 into words. So instead, I'll just show you!

Here's Alfred4's dashboard on Railway:

and…

This is Alfred5's dashboard:

I'm sure you don't need to me to tell you that Alfred5 is in a different league from any previous Alfred version. Alfred5 is like Alfred4 except bigger, better and just a bit bolder. Alfred5 does what Alfred4 does but on another level.

In fact, I'll let Alfred5 explain the difference:

You may notice this screenshot is not from Discord and that's because it's not! Alfred5 comes fully packed with its own website. Alfred4 technically had web integrations too, although those were only utility endpoints that served only to make it easier for users to share their timezone and such with Alfred. Alfred5 however is fully independant from any third-party platform. Meaning, unlike Alfred4, you can use Alfred5 without even having a Discord account.

Ultimately, we're one step closer to Alfred's ultimate goal:

This was said by Alfred3 almost 2 years ago, I've been working hard making Alfred's wishes reality.

Of course you can still log-in with your Discord account on Alfred to use Alfred on discord. The only difference is now you can use Alfred independently from Disocrd. Alfred's website also comes with many neat features of its own, like the ability for Alfred to create Excel spreadsheets and to build inline applications via HTML, CSS and JS.

Let's go into the meat and potatoes of this though. Here's what Alfred5 can do!

The Abilities of this Butterfly

Apart from the before-mentioned abilities. Alfred can do the following:

• Google Search: Alfred can browse the internet and find up to date and accurate information from it.
• Analyze Websites: Alfred can read through articles and websites to give you in depth information on the contents of websites. Useful when paired with Google Search.
• Manipulate Memory: Alfred has a persistent memory, he can store long term and short term memories and recall these memories at later dates.
• View Images: Alfred can, naturally, also view images. This is useful when transcribing information, like asking Alfred to tablize a diagram.
• Generate Graphs: Alfred can generate many different types of graphs and diagrams for you, like pie charts, or entity relationship diagrams.
• Run Code: Alfred can run code, this is useful for complex operations or mathematical questions. Like asking Alfred to sort data or to calculate something complicated.
• Set Reminders: Alfred can remind you on Discord about anything. These reminders can be set from the website as well. You can also specify where Alfred should remind you!

Alfred5 can also support many different types of AI, like AnthropicAI, OpenAI and GoogleAI. Although GoogleAI has been removed temporarily and will be added back once it reaches Alfred5's standards. Alfred5 also supports any platform of course and a desktop / mobile app is planned!

Alfred's many functions are useful on their own, and even more useful when combined, here's an example:

The Appearance of this Butterfly

Alfred5 went through quite the metamorphosis from being in a caccoon to waking up as a beautiful butterfly. As such Alfred's appearance has changed a lot! Alfred's branding used to be old-timy. The reason for this is because I wanted Alfred to appear as a traditional butler. Although Alfred5 being as advanced as it is - I decided to put Alfred through a bit of a rebrand. Alfred's new branding involves an extremely simple robotic face with a modern minimilistic design.

The die hard fans of Alfred has shown a lot of distaste towards this change initially. Although I feel this change is necassery and shows how Alfred matured. The simplicity is meant to reflect the simplicity of using Alfred. The old timy fashion doesn't fit within my views for Alfred anymore since I don't want Alfred to represent the past but to represent the future.

The Destination for this Butterfly

Remember, the goal for Alfred5- as I mentioned in my previous blog post, is to make Alfred more flexibile in terms of where it can run. Alfred now follows a more clustered and microservice-like structure, which allows me to improve aspects of Alfred indepedently and makes Alfred more flexible in general. I'm hoping I don't have to make an Alfred6 and that I can just keep building upon Alfred5.

I'm considering making a robot body for Alfred in some point in the future. We'll see how that goes! Remember you can check Alfred5's new website out here: ButlerBot.net

The 7 Recreations Of My PaaS

About 3 months before the creation of this post, I created a blog post reflecting on my time coding my own PaaS ( I highly recommend reading that blog post if you haven't already ). At the time I only finished about half of my PaaS - the backend, I still had to do the UI and add a few other bits and pieces. However, after completing the blog post, I wasn't all too happy with the state of the PaaS, so I decided to restart from scratch. Little did I know that this restart would not be the last.

Since then I've restarted, not once, not twice, not thrice but 7 different times. You may be wondering why, I wonder that too.. during the 7th restart, I decided that I wouldn't restart again. I knew if I kept restarting then I would take this project to the grave. Luckily the 7th version wasn't all that bad I'm quite happy with the state of my PaaS right now. So I finally decided to provide an update!

Why the 7th throw stuck?

Many hands make light work

The key difference between the 7th PaaS version and all the others is that the 7th version uses Nixpacks. For those who may not know, Nixpacks is an image builder made by Railway, it takes a project's source code and automatically builds an image for it that you can then deploy.

This "image building" part is an essential part of any PaaS because it makes or breaks the deployment process. Before I started using Nixpacks I was just throwing my own Dockerfile in a project which assumes the project is a NodeJS project, and for any other type of project, I'd have to manually make a new Dockerfile. This overcomplicated things by a lot, since I use Nixpacks now - I not only have automatic image building but my code is a lot more manageable.

Less is more, keeping things simple

Another big issue with the previous version is my tendency to overcomplicate. What I usually did was implement some sort of middleware between my app and the database that automatically caches data, and while it worked very well to make my app a lot faster, it also added a lot of unneeded complexity. In the 7th version, I tried focusing more on simplicity.

This made my code a lot less difficult to navigate and maintain, which made me enjoy working on it a lot more than I did before.

Polishing the Gem

Lastly, I feel telling myself that I won't make more versions really forced me to go back and refactor code to ensure longevity and robustness. I definitely still need to go back and refactor some more but I think it's totally manageable for now.

Cutting the Dead Weight

In the previous versions, I was using Cloudflare, not only for security and performance but also for SSL/TLS. However, I later found out that my app was extremely slow for no reason, a blank HTML page took 500ms to load - from a computer 5 feet away from the server the page was loading from. I immediately thought it was Cloudflare and tried disabling CF proxy, but it didn't help my cause, after 3 days of debugging and trying to find the cause, a friend asked me to disable Cloudflare entirely. After doing that my website was blazingly fast.

What happened was Cloudflare was routing my app's requests extremely poorly, requests for my website were coming from London. I reached out to Cloudflare and they said it was my ISP's fault, but I had friends from other continents test too and they had the same issues with it being unreasonably slow. I decided I'd just drop Cloudflare, the security benefits did not outweigh the slow speed of my site. After I dropped Cloudflare, my site's response times went from 500ms to 5ms.

New features

The new version also has a lot of new features, the biggest of which is a CD. A CD ( Continuous Deployment ) is a service that allows you to easily deploy completed code automatically. This was very needed in my case since it makes updating my PaaS a lot smoother and easier to do, which makes it easier for me to work on.

Another feature is statistics with Grafana, I can now monitor everything about my app using Grafana, which I initially only deployed to help me find the issue with load times I spoke of before, but it ended up being very insightful and I love just staring at my statistics!

Future Goals for the PaaS?

I have a few awesome plans for this PaaS, firstly - I want to add a db feature in the PaaS that allows me to easily deploy a database inside of my PaaS from a predefined image. Although I'm also content with using third-party database providers since they often offer extremely generous free tiers and also make migrating my project around a lot easier since I don't have to worry about migrating my databases, so I'm not in any rush to add this.

Another thing I want to add is a docs page, currently, nothing is documented, so it'd be a good idea to document a few things before I forget them, although if I do forget them then I can always just peak at the code, so this is also not on top of my priority list.

I think for now I can finally start migrating my projects onto my PaaS. It's been a blast working on this PaaS, I've learned so much while doing it, and I highly recommend it!