How I ended up with a mini home data center using raspberry pi
The story behind the cluster of LED lights sitting under my desk.
Truth be told, what is now a hobby, was born out of a professional necessity. I was involved in a project and needed to validate an architecture design that contained a cluster of MariaDB SQL databases, and so I started to ask myself how to test this in a lab.
I thought about AWS but I was already using all of their freebies on other small personal projects and did not feel like increasing my monthly bill. I also thought about deploying a set of virtual machines in my own laptop using VirtualBox, and to be honest, for the test I wanted to do, it would have been perfectly fine; create three small machines, install MariaDB, configure the replica set and test the fail over. But then I thought that maybe I would want to keep a cluster of databases for my own personal use. For example, I thought, I could use it to keep track of my expenses in a more exciting fashion that an excel spreadsheet. So, I decided to get a couple of raspberries,connect them together and have a system where I could try things out and also host useful tools and bots for my own use.
At this point in time, I already had two raspberry pi 3 that I had used for some experiments, and decided to get two more, one to complete the SQL cluster and the second one to use as a jump server. These last two machines are raspberry pi 4, because I wanted to leverage the 4GB RAM, compared to 1GB RAM of the previous model.
I was not too concerned initially with cooling the chips to avoid overheating, but I really wanted to rack the four machines together to avoid a mess, and since the rack I bought came with four ventilators, I figured it would be much cooler (joke intended) to set it up with fans.
To connect them all together I used a five port switch. My idea initially was to use four of the five ports to connect the raspberries and the fifth one as an up link to my home router. This way, I could create a new network for my home raspberry pi data center, which is separate from the one I use to connect to the internet. Unfortunately, the router my ISP provided me cannot be configured and so this plan is on hold at the moment until I replace that router with a better one.
Here is where I got a bit creative! I wanted to created a separate network for my raspberries to communicate between each other and I also wanted for the machines to be able to connect to the internet to fetch software packages from public mirrors. Consequently, the model 4 machines, I configured with two interfaces; one connected to the home WiFi network and the other to the Ethernet network that connected to the switch. The raspberries would then communicate across the Ethernet network and I could use the WiFi interface as a point of entry into the mini data center from my personal computer and also as a point of exit towards the internet for the raspberries.
The result ended up working quite nicely. I installed squid to act as an internet proxy in one of the machines, and I configured all of the raspberries model 3 without WiFi access to connect to the internet via the squid proxy. Additionally, I had one machine which I could use as a jump server to connect to it via VNC and control the whole setup. The main advantage I found in having one such jump server, is that you can install raspbian minimal in all machines and only install a GUI in the jump server, thus saving memory in all other machines.
Cool things that can be done on this setup
Aside from the limitation that this mini home data center will not be accessible on a public IP address, it can be used for all sort of cool home applications. Right now I’m running a SQL replica set to keep track of my expenses (developing a front end & back end for it too), running a minecraft server on it and a central jupyter-notebook server.
