Merciven
Work · Applied AI

LegacyOS: oral knowledge, searchable

Elders' recordings transcribed with Whisper and made answerable, with every answer citing who said it and when.

Organisation
Merciven (MVP)
My role
Designed and built it
When
2025
Stack
Whisper, Claude, PostgreSQL, pgvector

Problem statement

Knowledge about crafts, farming and family history lives in elders' memories and is lost when they die. Recordings help, but nobody listens back through hours of audio to find one answer.

Objectives & goals

  • Transcribe recordings in local accents
  • Answer questions in plain language
  • Cite the speaker and timestamp for every answer
  • Say "I don't know" rather than invent

Process

  1. DiscoverSpoke with families and community archivists about what they wanted to keep.
  2. DefineChose retrieval with citations over a chatbot, and made consent a required field.
  3. BuildWhisper transcription, chunking, pgvector retrieval, Claude answers from retrieved passages only.
  4. ProveTests for retrieval, citations and refusal on uncovered questions.

Who it's for

Family member

Asks what grandma said about the family land.

Researcher

Needs exact quotes with timestamps.

Challenges

  • Models fill gaps with plausible fiction
  • Recordings need consent before they're stored
  • Long, rambling speech has to be cut into answerable chunks

What I built

Retrieval with citations

Answers name the speaker, recording and timestamp.

Refuses when unsure

Below a relevance threshold, it says the recordings don't cover the question.

Consent in the schema

A recording can't be stored without a consent reference.

pgvector

HNSW index on embeddings in Postgres.

Flow

  1. Record
  2. Whisper transcript
  3. Chunk by speaker and time
  4. Embed into pgvector
  5. Retrieve
  6. Claude answers from passages
  7. Cite speaker and timestamp

Output

$ python -m legacyos "Which colour of aso oke was for chiefs?"
... Alaari, the deep red, was for chiefs and important ceremonies.
[Elder A (fictional), Weaving recording 1 at 00:00:12]
The sample answering from a fictional transcript.

Results

Citedevery answer
RAGWhisper + pgvector + Claude

Tools & languages

  • Whisper
  • Claude
  • PostgreSQL
  • pgvector
  • Python

The code

Runs offline with a small TF-IDF embedding; uses Claude when an API key is set. Includes the pgvector schema. 3 tests.

The repository is private. I'm happy to share it with a hiring team or walk you through it on a call.

Request a code walkthrough

Samples are written with synthetic data. No client code or client data is used.