Business, tech, and life by a nerd. New every Tuesday: Splitting Light: The Prism of Growth and Discovery.
Share
Splitting Light: Season 2 - Episode 20
Published 2 days ago • 3 min read
Splitting light
Season 2 Episode 20
Sharpened sense of purpose
If you are no longer interested in the newsletter, please unsubscribe
By early June 2018, 8 months in, we were advancing quickly. All these bricks started to be assembled into something that worked. It almost felt like advancing following a lego model manual. Except we didn’t have a manual.
The hardware, the software, the integration with existing systems, the testing started to converge into something that could be used. It started to look like a product.
At that point, we were in the alpha phase. We had a working product but made no guarantee on the availability or durability. We had onboarded several internal teams to be able to test. It was one thing to hit the cluster with performance tests, it was a different thing to actually have real flows.
My lens of understanding had finally come into focus. I could now see steps and ecosystems clearly. We were continuing to raise the performance and with new people being onboarded we were starting to have actual use cases that would trigger edge case bugs. A few more iterations, we would be ready to go to beta launch.
Photo of the war room by the launch event photographer
For me more importantly, I was more comfortable in my role. I was to guide the team from the technical point of view. I had been chosen to lead the team from the tech perspective because the rest of the company had seen what I was capable of. I didn’t see it myself. I did not understand back then. Even so, I could see my impact. I could feel that what I was doing mattered.
The team had many juniors, some still in or straight out of university. Almost every team had a similar mix. A lot of my guidance was avoiding pitfalls. It was reducing the risks of the path product launch or risk inside the product. Almost like a guide with a lantern.
Some bits I would do, but most bits I would have people assigned to. I could feel the product growing and maturing. Defects were fixed. Improvements baked in. Later we would joke that I was birthing object storage.
At the time we were a crazy bunch. This maverick team who didn’t care what obstacles were in front. We would move forward. We would take the train to the Lille office and request OpenIO to meet us there on a whim. We only wanted to fix problems and move forward. I can’t imagine how frustrating it must have been for them.
At the time, I was happy. Scaleway was growing. The stated goal with the pivot was to compete head on with AWS. I remember not taking this too seriously when we were told that. Intuitively we were clearly not in the category. But now with time flowing behind me, I understand that it was something else. It was the north star. It was what we aimed for.
That’s where the team was going. With Théo we were looking at different parts of the S3 protocol. Seeing what we could work on after the launch. On the other side of the storage team, Florian was starting experimentation with block storage. Checking what had to be done. Following a similar process as we had done with object storage. Checking what were the available options to us.
Our next step after both were to be in production, was file storage. That was my personal goal. My personal dream to work on. To build a filesystem.
We were ambitious. We outgrew ourselves. We burned the energy and grew the scope to make this work. Time almost didn’t exist. We were in the flow. So in the flow that I ended up doing an all-nighter.
If you have missed it, you can read the previous episode here
Splitting light Season 2 Episode 19 Bandwidth waves If you are no longer interested in the newsletter, please unsubscribe At every step we would test the performance. Crude methods at first. Sowing together scripts would enable us to get more kick out of the performance testing. The more performance we wanted to extract, the harder it was to do the tests. At first one powerful machine was enough to generate the request and traffic. Then we needed two of them. Then twenty… Then a hundred. We...
Splitting light Season 2 Episode 18 Controlling latency If you are no longer interested in the newsletter, please unsubscribe We didn't only increase our scope in iteration phases to reduce risk or go faster. We also did it for customer facing metrics. One specifically required some tradeoffs; it was latency. To be more precise, time to first byte. Object storage is a generic way of storing and fetching data. The maximum data you could store in a single object at the time was 5 terabytes but...
Splitting light Season 2 Episode 17 Increasing scope If you are no longer interested in the newsletter, please unsubscribe Just as we were iterating to reduce risk and gain experience. We were also increasing the scope of our work as the iterations went. We didn’t expect to have to handle the hardware. Yet we learned how to choose it. How many servers we wanted per rack and why. Measuring the actual power load at full usage. With the help of the network team, choosing the right routers and...