Why literate programming

What does literate programming offer ?

I would say literate programming is a proven technology for authors and bloggers. We use it everyday, except we call it blogs and tutorials! If you are writing any book on programming, you will have to build a tool to manage all the code snippets and present the snippets out of order for the reader. Don Knuth’s first LP system was also one of the first hypertext systems. Like the Xanadu, LP focuses on two-way links - transclusion instead of hyperlinks of html. I believe a LP IDE could in theory look like a hypertext browser. I explore this approach in another project called Index. It should also be possible to build LP into an engine like MediaWiki or Jekyll or Wordpress or Python notebooks; the solution in this project is however more general and minimal so that it can use readily available web apis.

One reason why literate programming did not take off is because the previous solutions focused on providing delcarative approaches to LP intsead of procedural ones. I call the approach I have taken here as procedural LP. While the emergence of online repls, blogs, api documentation and notebooks has made LP somewhat redundant for pedagogical purposes, I believe LP becomes truly useful when you start treating it for production code. Apart from the free-from documentation, with hyperlinks, TODO’s and FIXME’s you also get a simple project management tool for free.

Criss Cross Programming

The main primitives in wheel are prcedural preprocessing and code generation instead of mangle and tangle. The wheel workflow is web friendly making full use of videos and all html5 features, you can’t get that with latex. Because of preprocessing, I believe literate programming can also directly enable the following abstractions,

  1. Snippet as a general unit of code as opposed to functions, files and folders which also enables modularity.

  2. Literate programming directly enables feature driven development once you start treating every chapter as a feature. In games this could be levels I suppose.

  3. Because each snippet is programmable, you can pipe snippets to arbitrary functions or writing build function like for say css preprocessing, allowing you to experiment with DSL’s and new syntaxes.

Literate programming allows programming to be treated as knowledge work i.e programming is more like a team creating and maintaining a wikipedia page than building a car. Its by sheer magic that in programming writing text is exactly what makes a virtual car, where the model coincides with the execution. In many ways the execution of a typical application also resembles a screenplay, especially if you consider how use cases are documented in plain documents. Can we approach programming from the prespective of showcasing code next to use-cases than thinking of programming as managing files and folders ?

By far the biggest win for literate programming will be knowledge transfer and onboarding. This is a huge win. Who, What, Where, Why, How, For Whom are important too not just “When”. Major time in programming is also spent in communication and maintaining, building is just one phase of the project the often gets too much focus.Knowledge workers spend 38% of their time searching for information, one can only imagine how time is spent in code grepping. By organising code into features and using simple hyperlinks, code becames dramatically simpler to search and navigate. All the code that is triggered for “startup” or an “exit” button events can be shown in one place.

Personal Observations

Some observations on literate programming after using it for 2 apps (~4k lines, ~1.5k lines)

  1. I can read the code in the browser, casually. Spotting bugs and repetitive code is easier.

  2. TODO’s and FIXME’s make it easy to start from where you left off. This is less taxing on the memory.

  3. Backend, tests and the frontend templates are in one place. By far my favorite of all. This makes files and folders irrelevant. Files are for the compiler.

  4. No more commit messages.

  5. No more creating unnecessary files for modularising

  6. Big functions are not a problem anymore because it can be divided into sections, keeping code screenful. Code readability is dramatically improved, which is what 90% of programming is about.

  7. No need to divide code into functions for comprehension. A function is created if its reusable. A snippet which is a smaller unit of code reuse is also available.

  8. No more source banners.

  9. Each chapter is a new feature. This is much better way to think about code organisation than as files, functions and folders.

  10. You can drop webpack for minification. The literate program acts as a glue connecting multiple build tools together. Heck if you want, spellcheck your code because you have pipes.

  11. Because of cog, even the documentation can be divided into sections and passed through external tools.

  12. Single file for everything that means no need for grep or complex refactoring tools.