I don’t, and I know that I’m fucked one day.

I am fortunate enough to only manage a homelab and not an enterprise sized network. So I don’t document anything just like at work.

Lol.

Here’s my approach to documentation. It’s about habits as much as it’s about actually writing anything down:

1. Never setup anything important via naked terminal commands that you will forget you did

2. Always wrap important commands in some kind of “setup-xyz.sh” script and then run that script to see if your install worked.

3. If you need to make a change to your service, ensure you update your script and so it can be re-run without braking anything

Get into the habit of this and you are documenting as you go

I use the notes sections in proxmox preeettty heavily. Lots of links to the helper scripts, youtube videos and other resources i used to get er’ goin’.

In the near future I’m really hoping I can set up Netbox to help me document the network and equipment I’m putting in my homelab. a nice thing is that I went through a divorce a while ago and I’m getting to start from scratch. You’d be surprised at just how much you’ve learned since starting to self host and I think there’s this sunk cost fallacy that gets a lot of us to keep going with what we’ve got already set up because we’ve “already put so much work into it” and the concern of what we might lose by scrapping it and starting over.

Also, not what you asked… but if you’re still relatively new with proxmox you should check out the ProxmoxVE helper scripts. Lots of good automated scripts from doing a post-install to setting up various LXC containers and VM’s

I generally just make notes in Obsidian, mostly about switch ports, VLANs, IP assignments and that kind of thing.

Also try to save snippets of commands or config edits I needed to get something obtuse working in case I need to do it again later.

I just set up everything via ansible and comment everything religiously, I’ve gotten to the point where I have playbooks for my laptop and desktop so if I have do do a clean install I just have to run the playbook and everything is set up and installed to my liking

It’s annoying to go to my playbook to make changes and rerun it every time I want to install or remove anything from my daily PCs, but it pays off when I migrate computers or for some reason have to do a clean install

And for high level stuff and things not in ansible I keep a tiddlywiki since no matter how catastrophic the failure of my systems I can always find a way to access an HTML file for my own reference

Every time I set up anything, I do one of two things:

• If it’s container based, it gets a commented docker compose file in my custom orchestration

• If it’s on a host system, the changes are scripted and commented in a setup script, which are run on new machines. If the acrit is specific to one machine, it is configured as such

I find in-setup docs to be best for a home lab, plus if I have to replace hardware, it’s fast.

Fun fact, I do it for laptops and desktops, too.

The whole deployment is done via ansible, so the ansible source is my documentation.

I keep a very rudimentary README.md with some basic info on the services I run. I should probably set up a personal wiki to keep track of everything better

Perhaps more importantly, I also maintain a document with all the scheduled tasks that run, including what it is, how it’s defined, when it runs, and what device runs it. Really helpful for making sure cron jobs don’t run during a backup script or something

I refused to do any documentation for a long time because it made me feel stupid for not memorizing it. I learned it the hard way… Now I document everything possible with Git and Readmes.

I’ve moved to an “infrastructure as code” approach, not using any fancy tools in particular, primarily just bash shell scripts. Basically almost everything I setup or do gets documented via shell scripts, I write them as I go when I’m learning to install something new, and before I commit to something to new, I take extra care to make sure the scripts are idempotent so that when I want to do make any changes, all I need to do is add it to the appropriate script and re-run it.

The idempotent part takes some effort sometimes, but is not actually as hard as it seems, particularly if you don’t mind that it sometimes spends some wasted time doing things that have already been done, and occasionally spits out some harmless error messages because something is already done, but I also try to minimize that when I can. The consequences of doing too much by re-running are rarely serious. Yeah sometimes the scripts can break, but as long as they fail properly (set -euo pipefail) it’s usually pretty obvious how to fix it and it won’t leave too much of a mess.

Doing this has transformed my homelab from a mess of unknowable higgledy-piggledy spaghetti-services that was always teetering one small failure away from total collapse and frantic rebuilding, into something repeatable and reproducible that I can actually … wait for it … test. Just firing up a Linux ISO in a VM is all I need to test everything I’m doing in a perfect sandbox, and I can throw it away when I’m done with no regrets. Plus it makes rolling out new servers, and more importantly, decommissioning old ones, a breeze, you know exactly what’s on them and how it was set up, because it was all in your scripts. Combined with good data backups (which are also set up in the scripts) and restores (which I also test with scripts) it really takes the drama and stress out of migrations and even hardware failures.

Yeah there are probably easier ways to accomplish what I’m doing using some of the technologies like terraform, ansible and nix/flake that people have mentioned, and I’ve dabbled with those, but for me, the shell script approach strikes a nice balance of not just documenting but also learning the process myself so that I understand enough of what it’s doing to effectively debug it when something goes wrong, and it works on almost everything and in most cases requires no installation or setup. Bash is everywhere. I even have an infrastructure-as-code setup for my Steam Deck to install stuff and get it set up the way I want.

Secrets go in Keepass.

For server configs, a LibreOffice Writer file per machine (except for RPs, I only have one for those), written as a didactic manual explaining how to install and configure everything (I work on bare metal still). I started that way since diving into self-hosting was also a way to learn Linux, Sys-Admin and web-hosting. I don’t do anything without updating the relevant chapters, or creating new ones. Not gonna lie: it’s tedious. But also a life-saver, and the rationals for my choices remain available years later, which is priceless in many ways.

Once upon a time I had neat network gear running, and I mostly YOLO-ed the doc for those, relying instead on the firmware/config backups. I had to put those devices away, but when I finally get to play with them again, I’m going to suffer re-learning and re-discovering everything.

Recently, I got to hack and old console, and just did a chronological log-file with actions taken and URLs to guides, instead of writing down everything myself. It got me thinking I might add a simple log-file to track my actions, on top of my usual guides.

Or not. Having a life is nice too 😅

The Ansible playbooks I use to deploy it are the documentation.

I am the note taking king probably. I worked in the construction industry for 20 years. The rule was, ‘if you didn’t write it down, it didn’t happen.’ That has just carried over to every other aspect of my life including selfhosting. Whenever I sit down to my terminal to do anything, I open Notepad++ and a regular windows notepad session. The windows notepad session is a little script I came up with that opens windows notepad with 1000 empty lines. It’s one of the many quirks I have, but I hate having to hit the enter key to start a new line. I like to be able to click on a new line for a new line of thought and start typing.

@echo off
(for /l %%i in (1,1,1000) do echo.) > empty_lines.txt
start notepad empty_lines.txt

(Save as a bat link on desktop)

Anyways, the Notepad ++ session is for after things get worked out, I make an official entry into the Notepad++. The windows notepad session is just a scratch pad or ‘thinking paper’ from which I transfer to the Notepad ++ doc. Convoluted, no? LOL You asked, and I just pulled back the curtain for you a bit. Careful what you ask for, could stain your brain.

I try to document everything. I feel like, if I’m going to take the time to learn something, I might as well write it down. I take my Grok sessions and distill them down if I found the info relevant. I also do all of this because after my TBI which gave me a seizure condition as well as other mental/neuro issues, my memory is shit, even for someone of my age bracket. But I can stand up a server and secure it, just from my notes in a step by step manner conducive to my limited mental acuity. I’ve often wondered if anyone would be interested in my notes, like maybe some newcomer to selfhosting wouldn’t have to reinvent the wheel since I have a penchant for fucking things up.