Last week my payment processor went down for 6 hours on a Monday morning

Lost about $12k in processing fees that day because our backup was on the same cloud provider. Has anyone else dealt with single-point-of-failure issues in their fintech stack?

3 comments

3 Comments

phoenix_grant343d ago

Multiple providers is the only real answer. Ray nailed it about backups in the same region. Twelve grand lost in one morning pays for a lot of redundancy.

ray_campbell464d ago

Man that's rough. I've seen this play out way too many times where companies think they're covered but their "backup" is just a copy sitting in the same data center or even the same cloud region. You really need to spread stuff across completely different providers, like have your main on AWS and your failover on something like DigitalOcean or a bare metal setup, because if the whole region goes down you're still dead in the water. It costs more to maintain two separate live systems, but when you're looking at losing twelve grand in six hours, that math changes fast.

dianahayes4d ago

Yeah but "spread across providers" is easier said than done when you're already drowning in tech debt. Most small shops barely keep one stack running smooth, let alone two at the same time. Still, twelve grand is twelve grand, so maybe that's the kick in the pants to actually do it.