Giving Smarter: How Tech's Communication Fixes Inspire Charity Efficiency

From AI Labs to Charity Kitchens

You wouldn't think a talk about training giant AI models would have much to say to someone running a food bank. But stick with me. The engineers at Huawei's 2012 Lab spend their days shaving milliseconds off data transfers between computer chips. Their problems are not your problems—but their approach? That's transferable.

At the AICon conference in Shenzhen this August, Huawei's Li Baichao laid out how his team optimized communication for the Pangu model on Ascend hardware. The details are dense: AllToAll patterns, KV cache offloading, host-to-device bandwidth. But underneath the jargon is a set of principles that any nonprofit leader would recognize. Don't waste effort on the wrong bottleneck. Build for the tools you actually have. And sometimes the biggest win comes from a small, targeted tweak.

The 30% Problem Nobody Talks About

In large language models, communication between processors can eat up more than 30% of the total time during training. That's like a charity where a third of every donated dollar goes to processing fees, not programs. Most people would fixate on the model itself—the math, the data. But the real bottleneck is the handoff between parts.

Li's team found that in MoE (mixture-of-experts) models, a specific communication step called AllToAll was the culprit. They didn't rewrite the whole system. They zoomed in on that one transfer and made it faster. The result? A 10% improvement in performance. Ten percent might not sound huge, but when you're running millions of operations a day, it compounds.

For a charity, the equivalent is finding that one administrative task that eats up staff hours—maybe it's manual data entry or a clunky donation form—and fixing just that. You don't need a total overhaul. You need to find the 30% and attack it.

When the Old Playbook Fails

Here's where it gets tricky. The engineers tried using an existing optimization library called DeepEP. It worked fine on some platforms. But on their specific hardware, the Ascend 910A3, it fell flat. The library was designed for a different chip architecture, and forcing it just made things worse.

So they built custom communication operators tailored to the Ascend 950's network topology. It meant abandoning the universal solution for one that worked on their machine. That's a hard lesson for charities too. The best-practice playbook from a big national nonprofit might not fit your small local group. Your donor base, your volunteers, your software—they're all different. Sometimes you have to ditch the shiny tool that works elsewhere and build something that fits your context.

The Hidden Time Sink: Host to Device

As models got longer contexts—think a million tokens—a new bottleneck appeared. The time it took to move data from the host CPU to the device (the accelerator) started to dominate. This is the 'H2D' transfer, and it was killing their speed.

Li's team tackled it from two sides. On the hardware side, the Ascend 950 gave each NPU a dedicated H2D path—no more waiting in line. On the software side, they built something called Omni Cache to efficiently offload and reload KV data. Together, these changes cut time-to-first-token by over 10%. That's the difference between a donor waiting three seconds for a thank-you page and three-point-three seconds. On a large scale, it matters.

For a charity, think about your own 'host-to-device' handoffs. How long does it take to get a new volunteer from sign-up to actually helping? Or a donation from the online form to the bank account? If there's a delay, people lose interest. Streamline that transfer, and you'll see more engaged supporters.

Cover the Gaps, Not Just the Exposed Bits

One of the most elegant ideas in Li's talk was about 'overlapping'—hiding communication time behind computation. In an ideal world, the model is always computing, never waiting. The team's goal was to minimize 'uncovered' communication time—the moments where nothing else is happening.

That's not just a technical trick. It's a productivity principle. If you can make your staff's non-essential tasks happen while they're waiting for something else, you've effectively created more hours in the day. Maybe it's scheduling data entry during a slow afternoon, or using automated thank-you messages while a volunteer is on the phone. The goal is to never let a bottleneck sit idle.

What Charities Can Steal from This Playbook

So what does this all mean for a charity? Three concrete takeaways:

• Find your AllToAll. What's the one process that eats up a third of your time or budget? It could be donor communications, grant reporting, or volunteer coordination. Focus your optimization there first.
• Don't force a square peg. If a tool or strategy works for another organization but feels clunky in yours, it's okay to build something custom. A simple spreadsheet might beat a complex CRM if your team actually uses it.
• Overlap your tasks. Look for ways to do two things at once. Automate the boring stuff so your people can focus on the mission.

The Bottom Line

Li's talk was about AI infrastructure, but the underlying message is universal: efficiency isn't about doing everything perfectly. It's about finding the critical bottleneck and fixing it with the tools you have. For a charity, that might be the difference between serving a hundred families a week and serving a hundred and ten. Small tweaks, big impact.

And if you're ever tempted to chase the latest optimization trend, remember: the best solution is the one that works for your hardware—or your community.