diachron/TODO.md

high importance

• Add unit tests all over the place.

  • ⚠️ Huge task - needs breakdown before starting

• Create initial docker-compose.yml file for local development

  • include most recent stable postgres
  • include beanstalkd
  • include memcached
  • include redis
  • include mailpit

• Add first cut at database access. Remember that ORMs are not all that!

• Add middleware concept

• Add authentication

  • password
  • third party?

• Add authorization

  • for specific routes / resources / etc

• Add basic text views Partially done; see the /time route. But we need to figure out where to store templates, static files, etc.

medium importance

• Add a log viewer

  • with queries
  • convert to logfmt and is there a viewer UI we could pull in and use instead?

• figure out and add logging to disk

• Add email verification

• Update check script:

  • shellcheck on shell scripts
  • go vet on go files
  • golangci-lint on go files
  • Run go fmt on all .go files
  • Eventually, run unit tests

• write docs

  • upgrade docs
  • starting docs
  • taking over docs

low importance

• add a prometheus-style /metrics endpoint to master

• create a metrics server analogous to the logging server

  • accept various stats from the workers (TBD)

• move master-bin into a subdir like master/cmd or whatever is idiomatic for golang programs; adapt master wrapper shell script accordingly

• flesh out the sync.sh script

  • update framework-managed node
  • update framework-managed pnpm
  • update pnpm-managed deps
  • rebuild golang programs

• If the number of workers is large, then there is a long lapse between when you change a file and when the server responds

  • One solution: start and stop workers serially: stop one, restart it with new code; repeat
  • Slow start them: only start a few at first

finished

• Reimplement fixup.sh

  • run shfmt on all shell scripts (and the files they source)
  • Run go fmt on all .go files
  • Run prettier biome on all .ts files and maybe others

• Adapt master program so that it reads configuration from command line args instead of from environment variables

  • Should have sane defaults
  • Adding new arguments should be easy and obvious

• Add wrapper script to run master program (so that various assumptions related to relative paths are safer)

• Add logging service

  • New golang program, in the same directory as master
    • Intended to be started by master
    • Listens to a port specified command line arg
    • Accepts POSTed (or possibly PUT) json messages, currently in a to-be-defined format. We will work on this format later.
    • Keeps the most recent N messages in memory. N can be a fairly large number; let's start by assuming 1 million.

• Log to logging service from the express backend

  • Fill out types and functions in express/logging.ts