How I Built a Personal AI Knowledge Assistant (And Learned AWS Along the Way)

I love learning. But between work, family, and everything else, I never seem to have enough time to read everything I want to — let alone remember it all.

So I built myself an AI assistant that reads for me, remembers what I care about, and lets me ask questions in plain English whenever I want to learn about a new topic.

And along the way, it taught me a ton about AWS.

Here’s how it works, in plain English.

---

The Problem

I follow a lot of newsletters, blogs, and research papers. There’s a huge gap between:

• What I find interesting (AI research, investing, engineering, crypto and markets…)
• What I can actually consume and remember

I needed something that could:

1. Collect articles automatically from my favorite sources
2. Filter out the noise and keep only what matters
3. Remember everything so I can find it later
4. Let me ask questions like “What has Ethan Mollick said about AI workflows?”

That’s exactly what I built.

---

What the System Does

Think of it as a personal research librarian and note-taker combined — except it never sleeps, never forgets, and fetches new reading material for me every day.

Every day, it:

1. Automatically fetches new articles from newsletters and blogs I follow (Substack, RSS feeds)
2. Filters and scores each article for relevance and quality
3. Breaks it into small chunks and embeds them into a vector database
4. Makes it searchable through a chat interface

So when I open my chat assistant and ask:

“What are the most interesting ideas about agentic AI from this month?”

It searches thousands of chunks of documents and gives me a well-grounded answer with context.

---

The Tech Stack (Explained Simply)

Here’s what’s under the hood — using everyday analogies:

1. Document Collection

The system subscribes to RSS feeds and newsletters, just like you’d subscribe in a podcast app. It checks them regularly and downloads new content automatically.

2. Filtering & Quality Scoring

Not all content is worth saving. The system scores each article based on how relevant it is to my interests and how authoritative the source is — like a spam filter that’s personalized for everything I care about.

3. Chunking & Embedding

Every article gets split into smaller pieces (chunks) and converted into a “semantic fingerprint” (embedding). This lets the system understand meaning — not just keywords.

4. Vector Database (ChromaDB)

All documents are stored in a special database optimized for semantic search — essentially a giant memory palace.

5. LLM Answering

When I ask a question, the system:

• Finds the most relevant chunks
• Feeds them to a language model along with my question (RAG - Retrieval-Augmented Generation)
• Returns an answer with real context from the articles I saved

---

The Tech I Actually Used

For those who speak tech:

Layer	Technology
Language	Python
LLM Provider	OpenAI / DeepSeek / OpenRouter (multi-provider support)
Local Models	Ollama (Gemma 4-E4B / Qwen2.5)
Embeddings	SentenceTransformers / all-MiniLM-L6-v2
Vector DB	ChromaDB
Processing Pipeline	Python (fetcher → filter → extractor → embedder)
Chat Interface	Custom Python web interface (localhost + Telegram bot)
Deployment	AWS EC2, Docker, Docker Compose
Scoring System	Custom quality, novelty, retrieval and exploration scores

The real magic isn’t picking fancy tools — it’s tying them together into a coherent, automated system.

---

Learning AWS Along the Way

There’s one part of this project that took me from zero to actually understanding cloud infrastructure: deploying the system to AWS.

Before this project, “the cloud” felt abstract. I knew words like EC2, Docker, and S3 — but I didn’t really understand how they fit together.

By actually deploying and running my RAG pipeline on AWS EC2, I learned:

• How to launch and manage EC2 instances
• Why containerization matters (Docker) and how to keep an app running continuously (Docker Compose)
• How to persist data across restarts (the elephant in the room for any database!)
• How to secure and expose an API responsibly

It was frustrating at times — deployment days never go smoothly on the first try — but nothing teaches like breaking and fixing your own infrastructure.

---

Why This Matters Even for Non-Technical People

You don’t need to be a developer to want a system like this.

Many of us have the same problem: there’s more valuable content available than we can possibly read, remember, or connect. An AI-powered personal knowledge base is like giving yourself a second brain — one that does the reading, keeps what matters, and answers your questions.

---

What’s Next

I plan to keep expanding this:

• Adding more sources across different topics
• Sharing it publicly once I’m happy with the stability
• Using it for everything from weekly research briefings to preparing for career interviews
• Continuing to explore new use cases (personal productivity, investment research, and more)

If you’re curious or want to see the technical architecture, feel free to reach out. I’m happy to share more about what I’ve learned — both about AI systems and about AWS.