Popular
Trace a Token Through TP + CP + SP + EP + DP + PP in MoE Inference
How tensor, context, sequence, expert, data and pipeline parallelism compose in MoE prefill on 32 GPUs, and why experts force DP replicas into lockstep.
5.5x Faster HuggingFace Tokenizers for a Single Sequence
Use all cores on one long input with output identical to serial: Llama 3 tokenizes 4MB in 163ms, not 805ms, cutting text only where it's proven safe.
Accelerate LLM Inference with Speculative Decoding
Verify a draft model's tokens in one forward pass without changing the output distribution, or drop the draft model for Medusa heads and tree attention.
Parallelizing Multi-Head Attention
Why multi-head attention code transposes Q, K, V to (B, heads, T, head_dim) after the reshape: so one batched matmul computes every head in parallel.
Enterprise Sales
Layering enterprise sales onto PLG: features worth a premium, the $1M ARR rule of thumb, your first sales hire, 10-80-10 outbound, and when to discount.
More Career Advice
Ask for help, build influence through thought leadership, invest in peers early, lean into conflict, apologize well, and go straight to the job you want.
Interviewing Adrien Treuille, Founder CEO of Streamlit
Streamlit's founder on the $800M Snowflake acquisition, OSS metrics, Apache 2 licensing, PLG vs sales, community contributions, and hiring leaders sooner.
Accounting Advice for Founders
Notes on a PwC partner's lecture: collect sales tax early, classify expenses so less is reported as overhead, run a 409A yearly, and guard against fraud.
Intellectual Property and Entrepreneurship
IP law notes for founders: patents, copyright, trademarks, trade secrets, NDAs, California IP assignment rules, and open-source licenses from GPL to MIT.
Software Engineering Levels and Promotion
What's expected of new grad, senior, staff, and principal engineers (L3 to L7+), and why promotion comes from consistently delivering next-level scope.
What to Talk about in Effective 1-on-1s
Bring your manager questions, not status updates: expectations at your level, gaps to your next career goal, and how happy you are with projects and team.
Advice I wish I got at the start of my career
Prioritize learning over pay, join a breakout-stage company, build prestige through writing and talks, and vet your future manager before taking an offer.
New Posts
Notes: The Lean Startup
Notes on Eric Ries's The Lean Startup: validated learning, MVPs, Build-Measure-Learn, vanity vs cohort metrics, pivots, and sticky, viral and paid growth.
Notes: Venture Deals
Notes on Venture Deals by Brad Feld and Jason Mendelson: option pools, liquidation preference, pay-to-play, anti-dilution, board control, and no-shop clauses.
Scaling Istio
Trim xDS config and metrics to stop istio-proxy OOMKills, and keep istiod alive through scale-outs with discoverySelectors, HPA tuning, and idle timeouts.
Work Around Max Count of Security Group Rules on EKS
EKS NLB Services can exhaust the 1000 security group rules allowed per ENI. Managing rules yourself with consecutive static NodePorts cuts 5 per Service to 2.
Layer-4 Load Balancer & Zero-downtime Autoscaling and Upgrade
externalTrafficPolicy: Local drops NLB traffic on scale-in and rolling upgrades. IP targets or ProxyTerminatingEndpoints, plus a preStop drain, fix it.
Kubernetes Networking From the First Principles
How Pod-to-Pod, ClusterIP, and NodePort traffic flows through network namespaces, veth pairs, the cbr0 bridge, VXLAN, and kube-proxy iptables NAT.
Never miss a story from me, subscribe to my newsletter