Memory Resembles Covering Indexes

Lately I've started to notice that my memory behaves not like a book, but rather like an index. To specify more precisely, my mind treats memory as covering indexes for database tables.

Covering indexes store extra information solely for data retrieval, so queries can use the index alone without visiting the table. This speeds up searching and reading operations, but building a covering index for every query rarely proves a good idea. Each extra index adds write overhead to keep the index up-to-date, consumes storage, and increases cache pressure. When misused, it might actually degrade the performance.

My memory seems to have many composite indexes in the form of (topic, resource). For each topic the mind will efficiently get relevant resources and operate with their attached data (things I remember).
When the data appears insufficient or not present at all, the mind will need to see which other topics have similar resources and use their attached data. It may also hallucinate something on the fly, or just give up and go to the table.

The problem? My memory has no tables! In databases each extra index has a cost. Yet the mind seems to operate on indexes entirely, and, without tables as a single source of truth, it doesn't have to keep anything up-to-date. Until it does.

The memory needs a refresh. I remember the resource, so I just need to review it once again. That takes time. Sometimes the search process may take even more time than the reviewing itself. Here we can draw a parallel to the index updating overheads in a database. Not efficient. Also, the mind only updates the index and rarely stores the full data volume. And it keeps forgetting stuff all the time.

Notes can solve this issue by summarizing the key points of a resource or even a whole topic into juicy data ideal for those imaginary indexes. Processing some information? Write down any facts that you didn't know but want to remember. Store those notes somewhere with fast access to revisit upon forgetting. In some time you will learn them.

With too much information to learn you better stop violating the mind and use a personal knowledge management system (PKMS) like Obsidian to simulate the memory outside the mind, effectively extending it. Doing so, one will inevitably appreciate how much the mind actually does to keep things relevant and fast.

Alas, I had no intention to sell Obsidian here. In certain scenarios locality matters. Take the Rust Standard Library achieving highest tier documentation by placing it right in the code. With that in mind, often I just use a local file named what.md and dump the relevant info there. In fact, while trying to describe the what page for my website I got this article. Hope you enjoyed it 🌹