Splitting light

Season 2 Episode 32

Where my money at

If you are no longer interested in the newsletter, please unsubscribe

End of February 2019

It’s a strange thing to think back to 2019 in 2026. Why? Because of costs. One rack of SIS was several hundred thousand dollars. We joked that each rack was the same price as a small Parisian apartment. Nowadays, a rack that contains a single machine with one GPU can cost the same price or more.

Why was it critical that we got money flowing ? Because, unlike other companies, we didn’t get free cloud credits to pay for our products. We had to buy the hardware with real cash. The talks with the IAM & Billing team led by Kevin (a), had started a few months back with Théo (b), Kevin, Robin (c) and I. We had debated on how to bill and what was the expected telemetry format. What was important. How the billing pipeline worked.

Tree made of purple ethernet cable in Scaleway's Lille office (1)

Kevin & Robin explained in detail how the pipeline worked. Object storage was unique in the Scaleway portfolio at the time because a single resource unit could grow and shrink over the course of a day.

What happens behind the scene is that you do an integral. The amount you bill is the surface under the usage curve. What you are billing are gigabytes per hour. It is simply a generalized case of resource consumption. In the other cases, the quantity stays the same for the resource.

Segmentation of usage curve. You bill the surface under the usage curve. The smaller the intervals, the more precise you are, but the more compute and storage you need to bill.

Once we had understood how it worked, hooking it in was not simple.

We decided to bill the usage and the bandwidth. I did a rough design and delegated the work to Marian (d). Marian was a fourth year student at Epitech. He worked part time as part of his studies. He was the only member of the team in the Lille Office.

Skateboard ramp room in Scaleway's Lille office (1)

I offloaded a task that would require two additional senior people to finish. I expected him to be able to do it by himself at the time but I had not understood the depth of the work. In addition to the deep complexity, I put him through a trial by fire. Not realizing that he needed help. Théo did a much better job than me on this.

Nevertheless, he and Alexandre (e) did the usage part. The system would poll the cluster at regular intervals and store the telemetry in a local database. This required querying the internal OpenIO API. Once the telemetry was collected in the database, we would emit it to the billing systems. Why did we do it this way? The data we collected was ephemeral. If we did not collect it we didn’t have it. We could not reconstruct it. By storing in an intermediary location, we decoupled the systems.