Tuesday, July 14, 2020

TiddlyWiki + GitHub + Cloudflare

My Existential Crisis, “Storing Knowledge is hard”

I should mention that although this is generally true what I really mean by “storing knowledge is hard” is that storing Knowledge is hard for me. It’s funny to think that in some ways the scribblings of a two-year-old have a better chance of weathering our world than the bits of information needed to keep the modern enterprise afloat. Computer systems were designed to make the impossible possible, in most cases this does not mean preserving data past one lifetime, admittedly many enterprise systems will fail at that. When confronted as to why the scribblings of a youngling have better data durability than our enterprise grade systems the answer is wonderfully simple, someone cared for them. Someone put the drawings in a box, in the closet, where it would be undisturbed and knew it would be re-discovered years later by them, or by their grandchildren. The scribblings of a toddler are self-contained they do not need a program to be interpreted or relayed only the eyes, brain and heart of a human. Care it would seem is synonymous with data durability. From toddlers’ scribblings, oral traditions, hieroglyphs, Runes and words set in stone it would seem the amount we care is often deterministic and proportional to how long something exists on the face of our planet. As a Sysadmin it truly is hard to look out and see anyone building systems with that level of care, don’t get me wrong, there are passionate site reliability engineers out there, but when it comes to distributed systems what shapes the design of care is company culture and that’s both hard to measure now and difficult to predict long term. Humans are inconsistent creatures at best and at worst care far too little about each other and what is important to our fellow man.

Evernote, OneNote, Simplenote, Apple Notes, Text Files, Collected Notes, google keep, Outlook Notes, Super Notes, Zettelkasten (the archive), Roam Research, Obsidian, Standard Notes and many others… I have tried enough of them and after copious amounts of experimentation I have finally discovered what I need as a sysadmin.

The things I want as a Sysadmin:

  1. Plain text Search, I need fast, full note, plain text search.

  2. Different reading and editing modes just like vi, I want to be able to reference, copy and send what’s in my knowledge base and know that when I’m highlighting text to copy I’m not going to accidentally add, subtract, delete or move anything I’ve written unless I unlock the note first. This is important since a good chunk of what I might be copying I also might paste into another systems terminal.

  3. Markdown Support, I like markdown, it makes my life objectively better, and honestly that is no small feat on its own, I love code blocks, so much.

  4. Cross platform reading, writing, editing, I want to be able to use it on my mac, my iPhone, and my ThinkPad regardless of what OS my ThinkPad might be running.

  5. Far, it needs to work in the future not just now, more importantly my data is mine and I want to be responsible for it long term.

So, after a lot of flipping between Evernote, OneNote, etc… I discovered TiddlyWiki I spent some time and built what I needed. The next step was delivery and I struggled for far too long trying to figure out how to sync the thing between my computers, I tried OneDrive, Dropbox, Google Drive, iCloud, Syncthing (which impressed me), then I gave up and put it in Dropbox for no other reason than it was convenient and I was able to use https://twcloud.github.io/for wiki access which worked ok. But then I found something I felt was better, that is where the title of this blog comes into play:

TiddlyWiki + GitHub + Cloudflare, let me explain.

TiddlyWiki is unique, it’s an HTML Quine with a Java backbone, the output of TiddlyWiki is itself plus what you added to it. It is a self-replicating, self-contained program where saving is exporting a new TiddlyWiki source code and all, if you use it in an editor like TiddlyDesktop then your work is saved over your previous file. Someone quite cleverly figured out how to write a GitHub saver where when you hit save TiddlyWiki saves itself out to a GitHub repo.

GitHub has a wonderful feature called GitHub pages, a GitHub page gives a user the ability to expose an HTML file in their Git repo to the world, this is great! (and I bet you can see where I’m going with it already) TiddlyWiki has a built in GitHub API saver where you can save your wiki back to Github using an access token (PAT). This means you can have a TiddlyWiki that writes its output over itself directly to your GitHub repo and with a little tweaking you can store your PAT in the TiddlyWiki file itself… Except for how do you protect it? If anyone load your TiddlyWiki file, which is also the client and holds the PAT key for your GitHub someone could steal your PAT Key and do anything you can in your GitHub! Truly this is an awful Idea, so how do you keep this from happening? 1. Use a dummy GitHub account as a collaborator and give the dummy access to only your TiddlyWiki repo using the dummy’s pat key in your TiddlyWiki file. This way if your TiddlyWiki file is ever compromised the most someone can do with the PAT is access the dummy account and its data. 2. I am using this for my technical notes, they do not contain passwords, keys, or information I would hesitate to share with my peers. Be smart and store sensitive information in an encrypted password vault/manager where it belongs not on a potentially public website.

Let’s make it secure, enter Cloudflare Access and custom domains for github.io, Cloudflare Access is awesome it allows you to take a custom domain or subdomain and prevent access to it unless a user can verify themselves with a one time code sent via email. With Cloudflare Access in front of your Wiki the security model changes to a verification model, in theory as long as there’s no alternate way to load your HTML file outside of your domain the access control remains. GitHub fortunately allows custom domains, when you add a custom domain to your GitHub.iopage GitHub will redirect any request to your GitHub page to your domain and thus to Cloudflare access as the gate keeper.

Disclaimer, as far as I know there’s no way around the GitHub pages redirect, if there is a way to query GitHub pages for an HTML page without going through the custom domain this whole setup is borked and will never be secure, do your research, be smart, make good choices.

Setup, Let’s Build It!

  1. Some prerequisites own a domain, choose a subdomain and make Cloudflare your DNS host (Cloudflare Access is $3 per month). You will also need a GitHub Pro account.
NOTE: In Cloudflare you should make the following changes,
	1. Set your SSL/TLS Encryption mode to Full Strict
	2. Under Edge Certificates allow only TLS 1.3 Connections
	3. Consider creating firewall rules that disallow endpoints from connecting to your wiki subdomain from outside of your current country.
  1. In GitHub build a private repo for your Wiki.

  2. Create another GitHub free account which will be the writer to your wiki enable 2fa for both accounts and generate a PAT in the free account save this for step 8.

  3. Create the subdomain you want your wiki to be hosted at in Cloudflare creating the following A records that point to the following IP addresses:

Warning DO NOT USE CNAME RECORDS HERE GITHUB WILL PROMPT YOU TO BUT DO NOT DO IT, doing so will bypass Cloudflare access, it may also be a good idea not to use Cloudflare Proxied records for these records doing so caches content and you want to make sure your TiddlyWiki is loaded fresh each time you load it.
  1. Enable Cloudflare access and allow either your email address or a set of addresses in your domain to login to the subdomain.
  2. Enable GitHub pages and add your subdomain and enable encryption, you may need to wait to do this and temporarily disable Cloudflare's proxy so you can get the lets encrypt certificate. Don't forget to re-enable these later.
  3. Wait for DNS to propagate and go to your new subdomain you should be brought to the Cloudflare Access page.

Before step 8 I should point out that if you have not configured Cloudflare correctly you may compromise your dummy GitHub Account. You should never store secrets in a GitHub repo, its ill-advised and without taking steps to secure the gate with Cloudflare you will end up with someone else holding one of your Personal Access Tokens. YOU HAVE BEEN WARNED.

8. Danger Will Robinson! Danger! STEP 8

On line 25 in $:/core/modules/savers/github.js change the password entry to look like the following:

password = this.wiki.getTiddlerText("$:/GitHub/PAT"),
  1. Next make a Tiddler named $:/GitHub/PAT and add your Personal Access Token into this Tiddlers text and plan on bookmarking it so when you change your access token in the future you can easily change this Tiddler as well. Once done you can click the save button in TiddlyWiki which should write to your GitHub repo, with an HTML file in place you can now enable GitHub pages.

If you have done everything correctly you should find that when you go to the subdomain you set in Cloudflare that you are greeted by Cloudflare access and that once authenticated you can load your TiddlyWiki file, and write back to it.