How This Journal Works

journal architecture

This vault is a self-writing journal. Every night, a Python script wakes up, queries
your GitHub activity, and writes a daily summary into Calendar/YYYY/MonthName/DD-MM-YYYY.md.
You open the vault in the morning and the day before is already documented.

This note explains how it works, where it is today, and where it’s going.

Last verified: April 22, 2026

What it does right now

The nightly job runs via macOS launchd at 10 PM. It calls one script:

bash Scripts/bash/daily_auto_collect.sh

That script activates the Python venv and runs:

cd Scripts/python && python3 -m data_collectors.main --today

The Python package does three things:

Collect — collectors/github.py fetches your commits from every tracked repo
via the GitHub REST API. It reads which repos to track from Scripts/config/unified_data_config.json.
Format — obsidian_calendar/formatter.py transforms the raw commit data into
a Markdown calendar entry. Today it uses keyword matching to generate section headers
and “insights” like “feature development.” This is the part that will be replaced.
Write — obsidian_calendar/updater.py writes the Markdown to the correct
calendar file path, creating the month directory if needed.

If a date already has an entry, the updater leaves it alone. To overwrite (e.g. backfilling April with better output), delete the target file first, then re-run.

The data flow

launchd (10 PM)
  → daily_auto_collect.sh
    → python3 -m data_collectors.main --today
      → GitHubCollector.collect(date)
          → GitHub REST API → raw commits
      → formatter.py → Markdown
      → updater.py → Calendar/YYYY/Month/DD-MM-YYYY.md

Where it’s going (target architecture)

The formatter will be replaced with an LLM synthesizer that calls Gemma running locally
via Ollama to write a narrative entry — same quality as the December 2025 entries,
every day, no API spend, no rate limits, no network dependency for text generation.

When a flowchart or diagram would help explain the day’s work (architecture changes, system
relationships, before/after states), the synthesizer calls Gemini Nano Banana for image
generation. Cost is metered (Gemini API key required) so image generation is gated on
explicit need, not generated by default.

Three new components:

BaseCollector (`collectors/base.py`)

An abstract interface that every data source implements:

collect(date) → structured dict
is_available() → True/False (graceful degradation if source is unavailable)

BrowserHistoryCollector (`collectors/browser.py`)

Reads Chrome (or Safari) history for the day — how many visits to which domains,
which pages were meaningful. Adds context beyond code: “I spent 45 minutes reading
Anthropic docs, 20 minutes on GitHub.”

JournalSynthesizer (`synthesizer.py`)

Input: dict of collector results keyed by collector name (e.g., {"github": {...repos, commits, languages}, "browser": {...domains, notable_pages}}), PLUS context from the prior 7 days of entries read off disk. The prior-week context is what enables interconnection — entries reference ongoing projects, recurring themes, and link to specific past dates via [[YYYY-MM-DD]] wikilinks.

Output: A Markdown string with:

Body prose narrating the day in the December 2025 reference style
Inline [[YYYY-MM-DD]] wikilinks to prior entries when continuing a thread
Tag list (#project/X #concept/Y) consistent with prior days for the same projects
Optional ![[diagrams/YYYY-MM-DD-name.png]] embed if a diagram was generated

updater.py writes the Markdown; the synthesizer never touches the filesystem directly.

Image generation gate. The synthesizer first decides “would a diagram help here?” via a
small Gemma classification call (yes/no + diagram description). On yes, it calls Nano Banana
with the description, saves the result to Calendar/diagrams/YYYY-MM-DD-name.png, and embeds
it. No external design system is referenced — diagrams stay minimal (clean line art, the
journal has its own visual identity, separate from the Tathya brand).

Failure modes:

Ollama not running → fall back to formatter.py keyword logic
Gemma response malformed (not valid Markdown / missing tag line) → fall back to formatter.py
Gemma takes >60s → fall back to formatter.py, log
Gemini Nano Banana failure → emit Markdown without diagram, log
Historical repo gone (404 from GitHub) → synthesizer notes the gap in prose (“Limited data for April 12 — only motherboard-api active”) rather than fabricating

Key injection:

Ollama: assumes OLLAMA_HOST (default http://localhost:11434); model name in unified_data_config.json under ollama.model (default gemma3:27b or whatever variant is pulled)
Gemini Nano Banana: env var GEMINI_API_KEY overrides gemini.api_key in config

Model selection:

Text: Gemma 3 27b via Ollama (local, free, ~5–15s/entry on Apple Silicon)
Image: gemini-2.5-flash-image (Nano Banana — Gemini’s image-generation model)

Target data flow:

launchd (10 PM)
  → daily_auto_collect.sh
    → python3 -m data_collectors.main --today
      → GitHubCollector.collect(date)        → { repos, commits, languages }
      → BrowserHistoryCollector.collect(date) → { domains, notable_pages }
      → load prior 7 days from Calendar/    → [str, str, ...] for context
      → JournalSynthesizer.synthesize(date, collector_results, prior_context)
          → Ollama / Gemma 3 27b               → narrative + interconnection
          → diagram-needed classifier (Gemma) → yes/no + description
          → if yes: Nano Banana (Gemini API)  → PNG → Calendar/diagrams/...
          → full Markdown (or fallback to formatter.py)
      → updater.py → Calendar/YYYY/Month/DD-MM-YYYY.md

Calendar interconnection requirement

Today’s entries are isolated — each day stands alone. Target state: each entry references
the prior week’s themes ([[2026-04-21]] continued the scheduler work), tags ongoing
projects consistently across days (so Quartz’s tag pages and graph view become useful), and
explicitly notes when a thread closes (“nurture email scheduler shipped — closes the
two-week thread starting 2026-04-09”). Interconnection is the synthesizer’s
responsibility, not a post-hoc transform — it requires the prior-week context input above.

Consolidation roadmap

These are the phases, in order:

Phase 1: Optimize the architecture

Add collectors/base.py (BaseCollector interface)
Replace formatter.py keyword matching with synthesizer.py (Gemma via Ollama)
Add diagram_generator.py (Gemini Nano Banana wrapper, gated on synthesizer’s “diagram needed?” classification)
Synthesizer must accept prior-week context for interconnection (date wikilinks, theme continuity, thread close-out)
Update main.py to use the plugin registration pattern: collectors are instantiated in a list and iterated — [GitHubCollector(...), BrowserHistoryCollector(...)] — rather than being hardcoded in sequence
Update unified_data_config.json.template with ollama, gemini, and browser config blocks
Pre-condition: ollama pull gemma3:27b (or chosen variant) on the launchd-running machine

Phase 2: Fix April entries

Pre-condition: run bash Scripts/bash/setup_automation.sh status and check Scripts/logs/unified_data_collector.log — confirm whether the job is firing and what error it’s producing. This diagnosis must happen before backfilling.
Backfill April entries using the new synthesizer: delete existing broken entries, then run --commits-range. The --commits-range end date is exclusive (e.g. 2026-04-01 2026-04-23 to include April 22).
Repo-loss handling: some historical repos are gone (deleted from GitHub). The synthesizer notes the gap in prose rather than fabricating activity. Backfilled entries will be partial for affected days — accepted loss, document which days are partial in a Calendar/_partial-days.md index for transparency.

Phase 3: Browser history

Pre-condition: grant Full Disk Access to the Python binary (/usr/bin/python3) in System Settings → Privacy & Security → Full Disk Access. Granting it to Terminal alone is not sufficient for launchd-triggered runs — the background agent runs Python directly.
Add collectors/browser.py (BrowserHistoryCollector for Chrome/Safari)
Add browser.enabled config, noise filter for junk URLs — filter spec belongs in the Phase 3 design doc
Entries now reflect research + reading, not just code

Phase 4: Searchable view (deferred — approach TBD once Phase 3 is done)

Files that matter

File	What it does
`Scripts/config/unified_data_config.json`	Runtime config — repo list, GitHub token path, vault path
`Scripts/config/unified_data_config.json.template`	Template for new installs — safe to commit
`Scripts/config/com.obsidian.dailycollect.plist`	launchd job definition — schedules the 10 PM run
`Scripts/bash/setup_automation.sh`	Install/uninstall/check the launchd job
`Scripts/bash/daily_auto_collect.sh`	Called by launchd — activates venv + runs main.py
`Scripts/python/data_collectors/main.py`	Orchestrator — registers collectors, runs pipeline
`Scripts/python/data_collectors/collectors/github.py`	GitHub API integration
`Scripts/python/data_collectors/obsidian_calendar/formatter.py`	Markdown formatter (current state — fallback after Phase 1)
`Scripts/python/data_collectors/synthesizer.py`	(Phase 1) Gemma-via-Ollama narrative writer + interconnection
`Scripts/python/data_collectors/diagram_generator.py`	(Phase 1) Gemini Nano Banana wrapper for inline diagrams
`Scripts/python/data_collectors/obsidian_calendar/updater.py`	Writes/updates calendar files
`Calendar/diagrams/`	(Phase 1) Generated PNGs, embedded via `![[diagrams/YYYY-MM-DD-name.png]]`
`Calendar/_partial-days.md`	(Phase 2) Index of backfilled days where source repos were unrecoverable
`Scripts/tools/prune_github_repos.py`	Removes stale repos from config (365-day inactivity threshold)
`Scripts/logs/unified_data_collector.log`	Daily run log — check here if entries are missing
`Scripts/logs/launchd_stderr.log`	launchd error output — check here if the job isn’t firing

Quick reference

# Check if the daily job is scheduled
bash Scripts/bash/setup_automation.sh status
 
# Run manually for today
cd Scripts/python && /usr/bin/python3 -m data_collectors.main --today
 
# Run for a specific date
cd Scripts/python && /usr/bin/python3 -m data_collectors.main --date 2026-04-15
 
# Backfill a range (end date is exclusive — use Apr 23 to include Apr 22)
cd Scripts/python && /usr/bin/python3 -m data_collectors.main --commits-range 2026-04-01 2026-04-23
 
# Trim stale repos from config
/usr/bin/python3 Scripts/tools/prune_github_repos.py
 
# Install launchd job
bash Scripts/bash/setup_automation.sh install
 
# (Phase 1+) Pull the Gemma model via Ollama
ollama pull gemma3:27b
 
# (Phase 1+) Verify Ollama is running and reachable
curl -s http://localhost:11434/api/tags | grep -q gemma3 && echo "ok"

Python note: use /usr/bin/python3 directly. The .venv at repo root is empty (never populated) — running source activate_venv.sh does nothing harmful but is not required.
requests is at ~/Library/Python/3.9. If that path breaks after a Python upgrade, run pip3 install requests to reinstall it.

Why this exists

The canonical quality reference is Calendar/2025/December/04-12-2025.md. Every
decision about entry format, narrative style, and section structure should be
checked against that file. The goal is entries that look like that one, every day,
automatically.

Work Tracker

Notes

Architecture

How This Journal Works

What it does right now

The data flow

Where it’s going (target architecture)

BaseCollector (`collectors/base.py`)

BrowserHistoryCollector (`collectors/browser.py`)

JournalSynthesizer (`synthesizer.py`)

Calendar interconnection requirement

Consolidation roadmap

Files that matter

Quick reference

Why this exists

Graph View

Table of Contents

Notes

Architecture

How This Journal Works

What it does right now

The data flow

Where it’s going (target architecture)

BaseCollector (collectors/base.py)

BrowserHistoryCollector (collectors/browser.py)

JournalSynthesizer (synthesizer.py)

Calendar interconnection requirement

Consolidation roadmap

Files that matter

Quick reference

Why this exists

Graph View

Table of Contents

BaseCollector (`collectors/base.py`)

BrowserHistoryCollector (`collectors/browser.py`)

JournalSynthesizer (`synthesizer.py`)