The strange world of Python, as used by big investment banks. High finance is a foreign country; they do things differently there. post
I will discuss a fictional, amalgamated, imaginary Bank Python system called "Minerva". The names of subsystems will be changed and though I'll try to be accurate I will have to stylise some details and - of course: I don't know every single detail. I might even make the odd mistake. Hopefully I get the broad strokes.
Barbara — The first thing to know about Minerva is that it is built on a global database of Python objects in a key-value store.
Dagger — One important thing that investment banks do is estimate the value of financial instruments.
Walpole — A bank-wide job runner that fetches source code from Barbara. This considerably lowers the bar for getting stuff deployed.
MnTable — Unlike hashes, tables are memory-dense and easy to spool to and from disk. They can use b-tree indices to allow efficient access by any route, support bulk operations and can make use of lazy evaluation.
Over time the divergence between Bank Python and Open Source Python grows. Technology churns on both sides, much faster outside than in of course, but they do not get closer. The rest of the world is not going to adopt any of Minerva's ideas, not least because they've never heard of them. Minerva is also not adopting many of the ideas from the outside. There is an uncharitable view (sometimes expressed internally too) that Minerva as a whole is a grand exercise in NIH syndrome.
It could be that the biggest disadvantage is professional. Every year you spend in the Minerva monoculture the skills you need interact with normal software atrophy.
.
Bank Python has spawned a couple of excellent threads where developers in the industry relate their experiences.
Oh lord, everything about this post about the custom in-house use of Python at investment banks, inject it straight into my veins. twitter
I was the person who first deployed Python at Goldman Sachs. The core engineers from GS went on to build the Athena system at JP and the Quartz platform at BAML. ycombinator