Blog

Projects

I’m comfortable with services talking over the wire, partial failure, and eventual consistency. Distributed systems are messy by nature—I keep them boring on purpose.

Distributed systems

I like systems under pressure. Concurrency, queues, back-pressure, load—this is where design either holds or collapses. I build for the former.

High concurrency & load

I think in systems—how pieces fit, where boundaries should live, what breaks first. I care about clarity and durability more than cleverness.

System design

Specialties

Tech Stack

Business software in Cambodia for SMEs and larger outfits. Around since 2010.

CAS-BIZ does business software in Cambodia—from small teams to bigger orgs and NGOs. I helped build and run parts of what keeps it going.

Career education and design for business leaders. Japan Headhunter Award winner.

Concord Academy runs career education and has run classes at the University of Tokyo. I worked on the platform that supports those programs.

Marketplace for authors and readers—write, sell, buy books.

Ehon-Navi is a book marketplace for all ages. I helped build the platform so authors and readers can connect without a fuss.

Training and certifications for Cambodia's financial sector. Around since 2004.

IBF Cambodia offers training and certs for banks and microfinance. I helped with the systems that support that.

Learning platform for Cambodia's banking sector—training and capacity building.

The Institute of Banking and Finance runs this LMS for the Association of Banks in Cambodia. I worked on the platform that powers the courses.

Resources for parents—entrance exams, English, ICT, programming.

ReseMom is for parents of K–12 kids, with info on exams, English, and programming education. I worked on the product and platform.

School management system in Cambodia and Singapore. Automation for schools.

SALA helps schools manage things in one place. I worked on making the system reliable and easier to use.

API and admin backend for a game—settings and services.

I didn’t build the game itself; I built the API layer and admin tools so the game could be configured and run smoothly.

Gaming site with reviews, tips, and news for players.

ZillionGamer is a place for game reviews and industry updates. I helped build the platform so it stays fast and easy to use.


There's a moment in every product team's life. The roadmap is open, someone's had a coffee too many - going all out of their mind, and from across the table comes those **5** fateful words:

**"What if we added chat?"**

The room nods. It feels obvious. It feels *modern*. It has that new-feature smell. Nobody asks why. Nobody does the math. The ticket gets created, the estimate gets wildly undershot, and six months later a senior dev is at 2am debugging why messages arrive out of order for users in Southeast Asia.

This post is an intervention.

---

## 🎭 Chat Looks Simple. Chat Is Not Simple.

The cruel trick of chat is that the demo takes about four hours to build. WebSockets, a text input, a message list, boom, you're basically WhatsApp. Ship it.

Then your first real user shows up.

Suddenly you need to handle:

- **📬 Message delivery guarantees:** Did the message actually arrive, or did the socket drop silently? Better build acknowledgment logic. Better handle retries. Better make sure the retry doesn't deliver the same message twice. Users don't love getting "hey are you free?" four times.

- **⏱️ Message ordering:** Two users send messages at roughly the same time. Whose arrives "first"? Clocks on distributed systems disagree more than you'd expect. Congratulations, you've stumbled into the world of distributed clocks and causal ordering (vector clocks, Lamport timestamps), and you did not pack for this trip.

- **🟢 Presence and online state:** "Is this person online?" sounds like one database column. It is actually a distributed state synchronization problem that has made grown engineers weep.

- **✅ Read receipts:** Those two little checkmarks? Each one is a fan-out write to every participant in the conversation. At scale, this gets spicy fast.

- **🔔 Notification systems:** Push notifications, email fallbacks, in-app badges, do-not-disturb windows, notification preferences per conversation. This is its own product.

- **📜 Message history:** How far back do you store? How do you paginate it efficiently? What happens when someone scrolls up through 50,000 messages in a support channel? Your database will have opinions about this.

- **🛡️ Moderation:** Someone will say something terrible. What's your plan? Who sees flagged messages? Can you delete from all clients simultaneously? Can you audit it?

- **📎 Attachments:** "Can we send files?" Yes, and now you've also built a file storage system, a media preview system, and a malware scanning pipeline. Congrats.

- **🧑‍💼 The admin dashboard nobody warned you about:** Here's one that sneaks up on teams: the moment chat exists, management will want to see it. Who's messaging whom? Can we flag certain conversations? Can we pull reports? Can we suspend a user mid-conversation? What started as a feature request for your users has now spawned an entirely separate internal tool for your ops team. That's two products for the price of one sprint estimate.

And this is before you've handled any of the *product* complexity: group chats, thread replies, reactions, message editing, search, slash commands, bots...

The iceberg doesn't even have the decency to show you its bottom.

---

## 😤 It's Literally One Google Search Away

Here's what genuinely stings. This isn't obscure knowledge buried in a whitepaper or locked behind a paywall. You don't need to have built three chat systems to know what you're walking into. You just need to type "challenges of building a chat system" into a search bar and spend fifteen minutes reading.

The tradeoffs are documented. The war stories are public. Engineers have been writing about WebSocket pitfalls, message ordering nightmares, and presence system complexity for years, for free, on the internet. Stack Overflow threads, engineering blogs from Slack, Discord, Telegram, all out there, all findable in under a minute.

And yet, somehow, some leads and managers will sit in a planning meeting, confidently wave away engineering concerns, and greenlight a chat feature with a two-week estimate without doing that one search. Not because they're malicious. But because the decision *felt* obvious, and obvious decisions don't feel like they need research.

This is the part that frustrates engineers the most, and fairly so. It's not that hard problems exist. It's that the hard problems were completely avoidable with the smallest amount of due diligence. When you raise concerns in the meeting and get "I think you're overcomplicating it" from someone who has never had to implement fan-out writes at 3am, something inside you dies a little.

So if you're a lead or manager reading this: please, before the next time you greenlight a real-time feature, spend fifteen minutes googling it. Not to become an expert. Just to have a basic respect for what you're asking your team to build. Your engineers will notice, and they will appreciate you for it more than you know.

---

## 🤔 Why Teams Default to Chat Anyway

The reasons are understandable, even if they're not good:

**"Our competitors have it."** Ah yes, the product manager's favorite reasoning: copy the thing, inherit the success. Your competitors also have years of infrastructure investment, dedicated teams, and hard-won scar tissue from all the times it broke. You have a sprint and a prayer.

**"Users expect it."** Sometimes true. Usually a projection. Have you *asked* users? Or did you assume they'd want it because you use Slack?

**"It'll increase engagement."** It will increase *something*, yes. Mostly support tickets and infrastructure costs until it's done properly.

**"It can't be that hard."** Reader, it was that hard.

---

## 💡 What You Probably Actually Need

__Here's the thing:__ underneath every "let's add chat" request is a real product problem. The question is whether chat is actually the right solution to it.

**If the goal is customer support:** You don't need to build this. Intercom, Crisp, Zendesk, Freshchat. These exist, they're good, and they have entire teams working on the thing you'd be building from scratch. Integrate one in a day and ship literally anything else.

**If the goal is async team collaboration:** Are you sure that's in scope for your product? Unless communication tooling *is* your product, you're about to build a worse Slack inside your app and charge people for the privilege of using it instead of the Slack tab they already have open.

**If the goal is real-time multiplayer or collaboration** (think Google Docs-style, or a game): this is one of the few cases where building real-time infrastructure is genuinely core to the product. Go for it, but go in with eyes open and allocate real engineering time to it.

**If the goal is notifications or status updates:** You almost certainly want a notification feed, not a chat system. Much simpler data model, no two-way communication complexity, still feels interactive and alive.

---

## ❓The Three Questions Worth Asking First

Before your team writes a single line of WebSocket code, answer these honestly:

1. **What specific behavior changes for users who have chat vs. users who don't?** If you can't answer this with a concrete user story, you don't have a product reason. You have a feature hunch.

2. **What's the cheapest non-chat solution to the same problem?** Could an email thread solve it? A comment section? A shared document? A third-party embed? If something much simpler solves 80% of the problem, that's probably the right call for now.

3. **Can your product survive chat being broken for 24 hours?** If chat becomes core infrastructure and it goes down, what's the blast radius? Real-time systems fail in ways batch systems don't. Plan accordingly.

---

## ✅ When You *Should* Build Chat

To be fair: some teams absolutely should build a chat system. If real-time communication is your core value proposition, if users come to your product *because of* the messaging, then yes, invest in it properly, staff it appropriately, and build it to last.

The keyword there is *properly*. Not "let's add a websocket endpoint and see how it goes." Proper investment, proper architecture, proper on-call rotation for when things inevitably get weird at 2am.

But if chat is a side feature, a nice-to-have, a "might be cool," reconsider hard. Your engineering team's time is finite and precious. Every week spent on chat infrastructure is a week not spent on the thing that actually differentiates your product.

---

## 🧾 The Bottom Line

Chat is seductive because it *looks* like a feature and *is* a platform. You don't stumble into building Slack. You commit to it, staff for it, and treat it as a first-class technical investment.

Most teams don't need to do that. Most teams need to solve the underlying problem that made chat sound appealing in the first place, and there's usually a simpler, faster, cheaper path to solving it.

So next time someone says "what if we added chat?" don't nod along. Ask why. Poke at the real problem. Reach for the third-party embed.

And if the answer is still "no really, we need to build this ourselves," great. Now at least you know what you're signing up for.

Good luck. You'll need it around the read receipts.


A deep dive into the hidden technical and product challenges of building real-time chat systems, and why it’s more complex than most teams realize.

Blog

Posts

Why Most Teams Shouldn't Build a Chat System