Ingress and egress, day one

If you’re redesigning an AWS account structure, get your ingress and egress right on day one. Forget everything else.

I’ve learned this the hard way twice now. I won’t make it a third time.

The first time: five AZs

The first time, I expanded the network across five availability zones in the same region. It wasn’t for anything running. It was there so we could do A/B changes: build the new version next to the old one, then switch over.

I regret it. I over-complicated it. Those extra AZs weren’t carrying live systems, but they were still network: more subnets and routes to look after, and a lot more to unpick when the migration came round later. It was complexity I built on day one for a “just in case”, and paid for on migration day.

The second time: egress

The second time, I didn’t get egress right out of the gate, and that was my call. Some of our customers restricted access by IP, so they only let in traffic from addresses they’d allow-listed. Fixing egress properly meant going to every one of them and asking them to change their side.

I didn’t want to have that conversation. So I avoided it, and built workarounds to keep the old egress working instead.

The workarounds didn’t make the problem go away. They made it worse. Traffic wasn’t leaving the way the design said it did, so it was hard to audit, and hard to show we were meeting our compliance requirements. Every review meant explaining the exceptions all over again.

If I did it again, I’d have the customer conversation first, and put the entire project on hold until egress was sorted. Not the network workstream. The whole project.

Why it’s always the network

Almost everything in a new account structure can change later, and it’s cheap to do:

• OUs get renamed and reshuffled
• SCPs get tightened, loosened and rewritten
• tagging, naming and budgets get fixed as you go
• workloads move between accounts

The network doesn’t move like that. Once traffic is flowing, every decision about how it gets in and out is baked into something you don’t own:

• Your egress IPs are on a partner’s allow list, a SaaS vendor’s firewall, a bank’s integration
• Your CIDR ranges are in route tables, VPN tunnels and on-prem firewall rules
• Your DNS is in hard-coded config nobody remembers writing
• Your ingress paths are behind certificates, WAF rules and a load balancer someone pointed a domain at

Change any of those after go-live and it’s no longer a refactor. It’s a migration, with outside teams on the critical path and a change window you have to beg for.

Everything else in the account structure is a decision. Ingress and egress are a contract.

Keep it below the line

The network belongs below the line. It should be transparent to the people using what you build. An app team shouldn’t have to think about which way their traffic leaves, and a customer should never notice that you’ve moved anything.

That’s why I go central for ingress and egress. Central doesn’t mean there’s only one. You can have several ingress points and several egress points, across regions and for different kinds of traffic. Central means consistent: every path in and out looks the same, is built the same way, and is owned by the same team.

Central ingress and egress isn’t one door. It’s one consistent surface area.

That consistency is what makes it easy to audit, easy to explain to a customer, and easy to change without anyone above the line noticing.

Go further and own the addresses. Buy your own IP range and make everything static. If your customers allow-list IPs you own, you can rebuild everything underneath them and they never have to change a thing. I’d have skipped that awkward customer conversation entirely.

What I’d get right on day one

Before a single workload moves:

1. An IP plan. Non-overlapping CIDRs for every account, every region, and on-prem, with room to grow. Assume everything will need to talk to everything eventually.
2. Only the AZs you need. Every extra availability zone is more network to run and to migrate, even when nothing live sits in it. If you want room for A/B changes, find a simpler way than more AZs. Add them when you need them, not because you might.
3. Egress. Decide where traffic leaves, and make every exit consistent. Make every public IP static, and know which ones traffic leaves from. Better still, buy your own IP range and bring it to AWS. Then the addresses your customers allow-list belong to you, not to an account, a region or a provider, and they move with you. Find every customer and partner who allow-lists you, and agree the change with them before anything moves. Have that conversation, however awkward. A workaround is something you’ll have to explain at every audit.
4. Ingress. Decide how traffic gets in, and make every entry point consistent: where the load balancers, WAF and certificates live, and who owns them.
5. Hybrid connectivity. Direct Connect, VPNs and the Transit Gateway routes, agreed with whoever owns the other end.
6. DNS. Who owns which zones, and how names resolve between accounts and on-prem.
7. Inspection. If traffic needs to go through a firewall, design that path now. Bolting it on later means re-routing everything.

Then build the rest. Get the OUs wrong and you fix them on a Tuesday afternoon. Get the network wrong and you fix it over six months.

So, a third time?

No. Ingress and egress first, and in writing, before anyone argues about OU names.