(This page is very much WIP including the layout, but if you've made it this far, send me an e-mail!)

Files and file-systems


Priority: A+
This is the project I'm currently trying to push forward the most.

((ref: [[projects.upend]]#Intro,1:#*))


Life-cycle of files


Generic Out-of-Band Markup editor (/format?)

Priority: A
(opens the door to many other projects!)


If anyone wants to mark up a document it's easy to just fire up any text editor. On the other hand, there's no equivalent for out-of-band markup.

To be fair, there isn't even any common format for OOB markup - maybe start there? How to simply represent text + attached information?

Overlaps with Multi-Annotator?

Video Navigator

Priority: B+
(non-essential, but scratches an itch I have annoyingly often)

Making video as easy to navigate as text!

Concepts to utilize:

Itch spotted elsewhere:

Paper-text Quote-taker

Priority: B
(exists in some forms already but none that would really fit me god dammit)

Social annotation tools are really nice, but so is reading from paper - why is the workflow to merge the two so complicated? Imagine an app:

  1. Take a picture of a book 1.5. Select the appropriate text excerpt by "highlighting it"
  2. It is OCR'ed and stored
  3. You can attach information about the book, page number etc.

Various Experiments

Machinical a/v cut-ups // Decontextualizer

Priority: Solid B?

  1. Take a set of videos that are subtitled
  2. Sort subtitle segments by length (text, time)
  3. Filter or even modulate chosen segment length
  4. Randomly pick N segments
  5. Join them together along with their videos

There could be an intermediate format (i.e. step 4.5) - a text file that is then fed to a final video editor (ffmpeg script); and hence editable using standard tools (grep, vim, etc.).

A/V Recombinator

Priority: Solid B.

Imagine a pool of audio, a pool of video, and a set of operations. Pick a video, pick some transformations (cuts, filters, etc.), and feed the result back into the pool. (Kind of like evolves) Add some audio, let it stew, check back in a while :)

Another inspiration is the ffmpeg Glitch Art bot on mastodon.

Slide Capturer

Priority: B+
(The simplest version that's already useful should not be too difficult to implement.)

A program that extracts slides as bitmaps based on image (dis)similarity when given a video of the lecture (and possibly coordinates where the slides are in the video?)

Additionally: Just output the timestamps of slide changes, e.g. for annotating an audio track so that each segment can afterwards be paired with the slide...

YouTube annotations

RIP, another Google-gutted feature with great potential. Let's do better.

Web Annotations

Priority: C
(Stuff like popcorn.js would have been sufficient, but died - for a reason? Additionally, a lot of the work there is in UX polish and doing Google's work.)

Essentially comments with a time-stamp + spatial info (being able to annotate a certain area of the video).

From existing comments


Go through current page of comments, check for timestamps, insert them on the timeline - kind of like SoundCloud used to?


Priority: C
(Big project, lots of essential complexity. Perhaps [[UpEnd|projects.upend]] will help this along?)

Like except with native understanding of the formats

Social Networks


Priority: B
(Too interesting not to attempt.)

A collaborative spatial canvas that you can place any sort of info in (files, images, links, text), but in which every participant has a tangible presence.

A real "gathering place" for projects, planning, etc.

There's projects are trying to solve the awkwardness of video calls, but they approach the spatialization either in a completely abstract way (Wonder, or a bit too "skeuomorphic" for my tastes (Kumospace.

Why not join the natural metaphor of spatialized presence with an actual (hyper-)pinboard system - kind of like Muse does?

You could, at the same time, build a 2D multimedia mind-map collage of whatever you're talking about, while actually being casually "present" in that document.

Open-source Identity Toolkit

Priority: C
(There's people who can do it much better and faster than I can.)

A ready-to-use software package that contains:

(Think VRChat, but without the VR.)


Various Text-Messaging Experiments

Priority: C
(Not really all that important, but would be fun)

A chat where various rules can be imposed (including several rules at once), just to see what it does to communication:

Constrained chat

messaging app where everyone has a unique speech bubble style, like in comics (@tophtucker). every time you say something, it slightly but permanently perturbs your speech bubble's design

IRC thread detector

Usually, people in IRC reply back an forth to each other, with each message prefixed with the nick of the recipient. Would it be possible to automatically detect an on-going conversation and highlight/hide it, if one wants to (not) follow that one?

Modular Social Network / Discussion Board

Priority: Concept?

A modular "mix-and-match" system for building a social network.

The underlying structure of most (if not all?) discussion systems can be expressed using a differently shaped tree of messages.

This is not some grand insight, but since the base is the same, the rest can be thought of as different "modifiers". For example:

Obviously, it's a tough job to write software for just one type of a social network, let alone all of them. But for experiment's sake, this doesn't really need to scale to millions of users, and the UX doesn't need to be perfect either.

It would be interesting enough just to try out what a Twitter with optional anonymity would look like, image-only Reddit, real-time phpBB bulletin board, time-limited IRC with voting and forced anonymity (etc.), and I don't think it's impossible to create a UI that is abstract/general enough to allow even for weird use cases like these.

Hardware // Interfaces

Single-handed keyboard

Priority: C

After I got into computers by 1962... damnit I wanted a machine that would allow me to type with one hand wherever I went! And we still don't have it.
Ted Nelson - Intertwingularity: When Ideas Collide (9:37)

Haptic feedback mouse

Priority: ???

Shouldn't be too hard to prototype on the hardware side?

Personal Information Management

Multi-Calendar // Timeline

Calendar part

Personal "everything log"

Quantified-Self Trackers

Mental fatigue tracking

Priority: Literally my bachelor thesis topic
The cursor entropy part, anyway; I'd like to experiment with the rest later.

1st phase: Just see if it works and corresponds to subjective ratings? 2nd phase: Adjust UIs, color temperature based on fatigue levels?

Sleep video movement tracker

Priority: C

Point a cheap IR camera at my bed, quantify pixel differences, see if it matches up with wristwatch sleep tracking?

(Cons: Might get really creepy, but worth trying!)

Raspberry Pi

mp4museum with a web interface

Priority: B

Perspective Transformator // Mapper

Priority: C+
Mostly a workflow-optimizer rather than a novel approach, but touches on interesting topics (i.e. real-time rendering on the raspi)

Universal video-mapping solution for galleries, etc.

Take HDMI in, transform according to the web interface, output over HDMI.

(Hardware for HDMI input:

UNIXy stuff

Better shell

🚧🚧🚧 but in reverse?





Priority: B except better


MQTT-based process SIGSTOPper

VS Code Outliner Extension

Priority: Why me?
There's currently no extension that would provide an outliner editing experience but without the whole shebang (e.g. Foam, I think?).

I just want to be able to travel in lists and collapse them easily.


Other's resources

Other lists of ± project ideas that I found particularly inspiring: