Houyhnhnm Computing
🐴
Video: https://vimeo.com/155517248
(note: click Ø in the menu on the lower-right to disable slideshow mode)
| Title: | Computing Systems vs Computer Systems |
|---|---|
| Subtitle: | Houyhnhnm Computing |
| Author: | François-René Rideau, TUNES Project |
| Meeting: | LispNYC, 2016-02-10 |
| URL: | http://fare.tunes.org/computing/ngnghm-slides.html |
Interactions around, not objects below.
Point of view, not techniques.
Ideas have consequences.
Using same techniques differently.
A glorious present awaits.
Ngnghm's Story: Alien point of view
Orthogonal Persistence
Going Meta: First-class Computing
The way forward
Installing overtone on Windows for my friend N
Dependencies: Clojure, JVM, Emacs, SuperCollider...
Issues: hunting files, configuration, homedir, ABI.
Solution? Use Linux (Ubuntu)
Issues on Linux: jackd vs pulseaudio!
Linux: not overall easier — but more reproducible.
How can two programs disagree on ABI? On homedir?
Why configure at all? No sane defaults?
What are files?
Why redo everything on every machine?
Why four development platforms? Can't the bare system do it?
Why care about it at all? Can't it all be automated?
Gulliver's Travel, book 4, by Jonathan Swift
Houyhnhnms & Yahoos
Ngnghm the Skeptic (also a Computing Scientist)
Shipwrecked, enslaved, almost knacked
Escaped, rescued, hiding
Learns Human Computing, comparative cultural study
No supernatural magic powers
Just a different point of view
Computing Systems vs Computer Systems
Human precedent: Cybernetics
Natural magic!
Computers as means, not ends
Computer Science is no more about computers than astronomy is about telescopes.
—E. W. Dijkstra
Dijkstra: computing as exploring pure maths
Houyhnhnms: maths yes, but foremost, meat
N: Can you show me a simple Human Computing System?
printf("hello, world\n");
C programming language, simple and well formalized.
N: It's full of holes & not even Universal! ("Turing-equiv…")
Filesystem, editor, compiler, make, shell, console, kernel, etc.
N: That's extremely complex!
Smalltalk world
Alan Kay's VPRI projects: entire system in 20 KLoC
FORTH, but...
Lower-level makes simpler computers, not simpler computing
Learning concepts
Reasoning about systems
Secure Bootstrapping
Debugging
Serious refactoring
Houyhnhnms have no keyboards. N loves watching me type.
What is that secret mantra I type all the time?
Enter? All the time, but easy to figure out.
Ctrl-Alt-Del? Quite spectacular, and rare.
C-x C-s (Windows: Ctrl-S. Mac: Cmd-S)
Save-File
If you don't save, you lose.
N: Why?
Bad click, closed app, software crash, power loss...
Device loss, theft, death → also need backup
Predictable catastrophes, automatable solutions!
Historical reasons: big files, small disks
Architectural reasons: that's how it's done
Epistemic reasons: can't auto decide which version to save
Personal reasons: been bitten too often
Efficiency: saving neurons by saving file at every pause.
N: Oh, it was automated. In meatware!
Sacred Motto of the Guild of Houyhnhnm Programmers:
"I object to doing things that computers can do."
—Olin Shivers
Automate in Meatware? Miss the whole point of Computing!
The highest goal of computer science is to automate that which can be automated.
—Derek L. VerLee
Wisdom: know to accept defeat
Foolishness: surrender when victory is at hand
Everything is always saved:
"If it's not worth saving, it's not worth computing"
You don't manually spill registers to caches to RAM...
Why manually spill RAM to disk, disk to remote backup?
Latency: if connected to power grid and internet, seconds
If disconnected, low-power: minutes latency, no replica
Space: more than enough for everything you type, download
High-level events. Strict determinism. Discard the irrelevant.
Privacy: everything encrypted by default.
Optionally, share with others, or triple encrypt.
No one size fits all: latency, replication, privacy...
Programmers can't possibly know.
Let end-users decide policy tradeoffs.
Define domains of persistence. Make it part of the UI.
Use competing storage providers, world-wide
No data loss until the End of Civilization
When designing interaction for data entry, configuration, etc:
Each can specify what they know, none need specify what they don't
Modern Apps may superficially save for you
But it's not reliable, replicated, etc.
Underneath, it's transience at every level
Very complex protocols for serializing, saving, transacting
The end-programmer can't handle all the configurations
Responsibility pushed back to the disempowered end-user
Joke on me for compulsively typing C-x C-s ?
Joke on N!
Since rescue, use Microsoft Paint to take notes
Her travels and travails, marvels and ordeals... in verse
How many Homeric Sagas have been lost?
Millions of hours of mental power wasted world-wide?
Never lose your session to a crash
Not just tabs, but all session state
Infinite Undo/Redo... for free!
Eternal instant replay
Others can't delete data on you
You own (search, analyze) your past data
Deletion is costly
Proportionally to amount of data since fact to delete
Irreversible: requires a lot of testing
Beware: Other people keep their copies
De-indexing is almost free!
Note: immutable facts, not mutable records, are primitive
Write at least 30% less code
No "save" primitives — Yes "Don't save" primitives
Experiment without fear
Bugs 100% reproducible by default
Think in terms of your abstractions, not of bits
Don't implement from "the bottom"
Yahoo programs: specify "everything from the bottom (OS)"
Yahoo metaprograms: compile-time only or leaky at runtime
Leaky Abstraction: the program can see what's underneath
Full Abstraction: program can't see what's underneath
Houyhnhnm metaprograms: what is underneath
Houyhnhnm programs: defined from as high-level as possible
Move computation to another system without restarting.
Change policy: I/O devices (console, sound...), persistence, speed, latency, robustness, "optimization level", privacy, trust, logging, etc.
Replay buggy interaction with Omniscient Debugging
Garbage Collection as migrating graph to another space
Endless possibilities...
No impermanence (but cleanup discipline)
No byte sequences (but object boundaries)
No saving files (but release management)
No filenames (but data indexing)
No startup scripts (but config diffs)
No finite access rights (but domain capabilities)
Yahoos: byte sequences, each with explicit encoding
Yahoos: text files as source, or worse
Yahoos: either unstructured or structured editing
Houyhnhnms: objects with system-managed encoding
Houyhnhnms: semantics as source, auto encoding migration
Houyhnhnms: editable views, both unstructured or structured
What if you hose your system? Grave errors persist!
OOM, fork bomb, deep metabug, persistence broken, etc.
Try not to!
Keep snapshots, drop irrelevant data, use automatic monitors...
Yet what if you do? (Old Eumel story...)
Escape to the meta-system!
The monitor: a simple system, with reserved memory
Can merge in parts of the system being debugged
Hence why build complex systems from simple ones
What about bugs while debugging?
Virtualize it! (Thus reserve virtual memory, too.)
You can dichotomize where the catastrophe happened.
A metabug that breaks persistence can wreak havoc
You can never completely avoid all bugs
Or can: prove it correct, test it like SQLite
Abstraction: most programs don't reimplement persistence
At least: not worse than for Yahoos
Fallback: religiously keep (and test) old backups
Entire system under git — but live, not dead
Virtualization is branching
Undoing bad bug is rebasing (orwellian retcon)
Limits: can't unprint the data, unlaunch missiles
Hardware acceleration welcome, not needed
What is needed: Full Abstraction
Higher-levels of abstraction can't see implementation specifics
An essential difference between Houyhnhnms and Yahoos
A property of the system: not hardware, not language
Leaky abstractions are not just top priority bugs:
They are security issues — call cops on abstraction breakers
First-class Computing Systems
Not just programming language
Includes runtime execution state
Not just a One True Computing System
Abstract over incremental computing system diffs
Virtualization not just for sysadmins
Every program is an abstract computing system
Yahoo apps: rigid towers rooted in the OS
Houyhnhm components: modular stackable elements
Yahoo Operating Systems: privileged kernels
Houyhnhnm: dynamically relinking metaprograms
Each leaky abstraction is a costly liability
Error situations bubble up to the user
Manual parsing, unparsing: slow, unsecure, hard to get right
Unvalidated inputs → "Language-Theoretic Security"
Compatible? entire tower must match → find common level
Adapters above lose atomicity → Need common metalevel below
Persistence, Robustness, Upgradability, Maintainability...
Dismissed by Human programmers as "Non-functional requirements"
To Houyhnhnms, essential aspects of Computing interactions
Choosing artifacts over interactions: Sacrificing ends to means
Compression algorithms (lossless vs lossy)
New synchronization algorithm (sequential or CRDT)
Optimization passes (safe or unsafe)
Data structure representations (depending on constraints)
Solve issue once with a meta-program
Not every time manually with a "design pattern".
Infinite turtles below...
and you can change them, dynamically!
Building software down, not just up.
Not Tower of Babel, but computing algebra
Initial resource handler → Bootstrap chain
Resource access monopoly → Resource ownership transmission
Resource management → Linear Logic
Multiplexing access → Servers
Implementing services → Meta-programs
Virtualization → First-class full Abstraction
One-way communication? It's a document, not an app
Extensible Platforms (like Emacs)
Yahoos: each app reimplements its own bad file selector, etc.
Houyhnhnms: component solves a problem for everyone
Each should focus on his specialty (division of labour)
T diagram: each T translates from A to B, running in C
Lisp: great at abstraction, but leaky as hell
PLT Racket: PL construction kit
Who to pay costs of Migration?
How to keep up with Yahoos?
How to win despite network effects?
How not to fall into autism?
Where to bootstrap from?
Vision and Will come foremost
"If you want to build a ship, don’t drum up the men to gather wood, divide the work, and give orders. Instead, teach them to yearn for the vast and endless sea."
—Antoine de Saint-Exupéry (creatively misquoted)
The goal is not a system
The goal is a saner approach to software development
Metaprogramming is programming programming
Apply the knowledge. Share it with colleagues. Change the project.
Not every project can be reformed. Sometimes, restart from scratch.
Sometimes, quit the project, or even the company.
You can't save other people. But you can save yourself.
There is no path to Happiness. Happiness is the path.
Yet recognize the tree by its fruits.
Start Anywhere: no need to start "from the bare metal".
Start Simple: no "all or nothing". Systemantics.
Do your thing: division of labor
Build bridges: division of labor
Join forces: division of labor
Go down, not just up
New PoV leads to new architecture
The interaction isn't a one way command-and-control