Knowledge Diffusion: A New Way to Extract Structured Knowledge from LLMs
Originally shared on my personal blog here: Shep Bryan’s blog.
-
Playing around with LLMs lately, I stumbled onto something that's been keeping me up at night - a way to pull structured knowledge out of these systems that feels fundamentally different. The patterns I'm seeing suggest we might be able to tap into how these models actually structure and connect information.
The Pattern
LLMs hold vast, interconnected webs of understanding. When we let that knowledge naturally diffuse outward from a seed topic, it creates rich networks of insights. The connections that emerge often surprise me - revealing relationships I wouldn't have thought to explore.
The Experiment
I built a quick system to test this idea, inspired by how Stable Diffusion generates images from noise. It’s an agentic digital garden generator where AI is the gardener. Starting from a seed topic, it lets the LLM expand outward organically, creating and connecting ideas.
Running it locally with Mistral NEMO 12B, I fed it "How To Make Delicious Pizza" and let it run for an hour. The results were fascinating:
62 interconnected markdown files in an Obsidian vault
Each file linked to 5 others based on semantic relevance
Unexpected connections emerged - specialized baking techniques got naturally woven in, creating paths through the knowledge I wouldn't have mapped manually
The process looks like this:
Plant a seed topic
Let it expand to semantically relevant subtopics
Have the AI reference both its knowledge and previous content
Create organic links between related concepts
Repeat until you have a rich knowledge network
Map everything in a final knowledge graph
The Implications
This opens up new possibilities for extracting and organizing the vast knowledge trapped in these models. The structures that emerge feel qualitatively different from human-organized information - and that difference might be exactly what makes them valuable.
The Next Layer
I'm particularly excited about:
How this could change research and learning
What happens when we compare knowledge structures across different LLMs
Adding human steering to guide the diffusion process
Using this for rapid domain understanding
This feels like just scratching the surface. I'm continuing to experiment with different approaches and applications. If you're working on similar ideas or see interesting possibilities here, I'd love to hear your thoughts. Reach out via our contact form or check out more of our lab's explorations here.
The experiment continues.
