Splitting light Season 2 Episode 22 Too many logs If you are no longer interested in the newsletter, please unsubscribe I’ve rarely seen people talk about this effect. The effect being the amplification of requests. This effect can overwhelm your system. We had to deal with it. The object storage, at least OpenIO, was a collection of distributed services. You might call them micro services if you want. That had implications. When a request comes in, from the user perspective, it’s a single...
8 days ago • 2 min read
Splitting light Season 2 Episode 21 All nighter If you are no longer interested in the newsletter, please unsubscribe As we were moving forward, in mid June 2018, we hit a point where we needed to be able to check the logs of the cluster as a whole. The way we had done it until then was manually connecting to the machines and opening the right files to look inside. This was no longer viable. One of the main office rooms (1) Scaleway’s monitoring team had done a metric stack which we already...
15 days ago • 2 min read
Splitting light Season 2 Episode 20 Sharpened sense of purpose If you are no longer interested in the newsletter, please unsubscribe By early June 2018, 8 months in, we were advancing quickly. All these bricks started to be assembled into something that worked. It almost felt like advancing following a lego model manual. Except we didn’t have a manual. The hardware, the software, the integration with existing systems, the testing started to converge into something that could be used. It...
22 days ago • 3 min read
Splitting light Season 2 Episode 19 Bandwidth waves If you are no longer interested in the newsletter, please unsubscribe At every step we would test the performance. Crude methods at first. Sowing together scripts would enable us to get more kick out of the performance testing. The more performance we wanted to extract, the harder it was to do the tests. At first one powerful machine was enough to generate the request and traffic. Then we needed two of them. Then twenty… Then a hundred. We...
29 days ago • 3 min read
Splitting light Season 2 Episode 18 Controlling latency If you are no longer interested in the newsletter, please unsubscribe We didn't only increase our scope in iteration phases to reduce risk or go faster. We also did it for customer facing metrics. One specifically required some tradeoffs; it was latency. To be more precise, time to first byte. Object storage is a generic way of storing and fetching data. The maximum data you could store in a single object at the time was 5 terabytes but...
about 1 month ago • 3 min read
Splitting light Season 2 Episode 17 Increasing scope If you are no longer interested in the newsletter, please unsubscribe Just as we were iterating to reduce risk and gain experience. We were also increasing the scope of our work as the iterations went. We didn’t expect to have to handle the hardware. Yet we learned how to choose it. How many servers we wanted per rack and why. Measuring the actual power load at full usage. With the help of the network team, choosing the right routers and...
about 1 month ago • 3 min read
Splitting light Season 2 Episode 16 Iterative process If you are no longer interested in the newsletter, please unsubscribe All throughout these first few months, from December 2017 to May 2018, we did iterations on object storage. This was both necessary to pick the skills and also reduce risks. The work to bring up an object storage is a subset to making a public facing object storage product. In the first case you control everything, in the second, you control much less. Quentin working in...
about 2 months ago • 2 min read
Splitting light Season 2 Episode 15 Internal identification If you are no longer interested in the newsletter, please unsubscribe End of May 2018 had arrived. I had an appointment. After doing a tattoo on my arm, I continued to look into tattoo art. Looking at different things and identifying more symbolism that I wanted. I remember reading Revenger by Alastair Reynolds, a mix between new space opera and pirates of the caribbean. Spaceships with light sales. I finally contacted the tattoo...
about 2 months ago • 3 min read
Splitting light Season 2 Episode 14 Access key If you are no longer interested in the newsletter, please unsubscribe We were now at a stage where we had to no longer have hardcoded credentials. We needed to be plugged to Scaleway’s authentication database. Historically Scaleway, the cloud computing division, had decided to design their own API. They used a mechanism called Json Web Token (JWT) to authentify. This system only required a secret credential. On our side, S3 required two things. A...
2 months ago • 3 min read