Splitting Light: Season 1 - Episode 23


Splitting light

Season 1 Episode 23

Carbon14 product launch

If you are no longer interested in the newsletter, please unsubscribe

The initial scope for the cold storage product contained the hardware, frontend, backend, user , billing and website. When I took over, my goal was to reduce as much as I could the amount of work that had to be done to have a functional product.

I was able to have the dedicated team patch their own frontend and backend to add a few elements. They also enhanced the billing pipeline to handle billing by usage which it did not support yet. Just this simplification reduced the amount of sub systems to design and code from six plus different systems to only three. Plus a couple modifications on the existing systems.

I wasn’t keen on designing new APIs or protocols, there were already a slew of existing ways to transfer data, what if we could just use those? We ended up having the data reception part that was a container and inside you had a server that supported scp, sftp, webdav or rsync. The user would open an archive, upload the data and either they would decide to archive once he had uploaded his data, or the container would automatically close down after seven days. The data would be asynchronously pulled from there and sent to the cold storage sub-system.

Out of the brainstorming sessions, someone suggested the name carbon14. I do not remember who. It was a perfect name because, in common knowledge, carbon 14 was already associated with things being old and archeology. Instinctively people associated that with long term and for a cold storage system, it was just perfect.

And just like that, June 24 2016, with a stack of pizza we launched carbon14. Because we reused so many things, it was dizzyingly fast to launch. I went from debugging the hardware, to coding the storage system, to checking the racked hardware in DC2, to doing the launch in only a few months.

It was exciting to work on this project, it was fast paced, it was adrenaline fueled. On the launch night, I stayed late and helped the customers get a hand in the product.

Carbon14 would be part of my next five years at Scaleway in a way or another. It was a pivotal point in my career. A year later, everything would catapult forward. Until then, I had to weed out the bugs and make the customers happy.

(1) https://web.archive.org/web/20160627053733/https://www.online.net/en/c14

To pair with :

  • Laily (Photographer Remix) - Aly & Fila, Karim Youssef, Photographer
  • Ghost in the Shell by Shirow Masamune

If you have missed it, you can read the previous episode here


Vincent Auclair

Connect with me on your favorite network!

Oud metha, Dubai, Dubai 00000
Unsubscribe · Preferences

Symbol Sled

Business, tech, and life by a nerd. New every Tuesday: Splitting Light: The Prism of Growth and Discovery.

Read more from Symbol Sled

Splitting light Season 2 Episode 23 Beat the cluster to a pulp If you are no longer interested in the newsletter, please unsubscribe With proper observability we could now push the cluster even further. This was the final set of tests that we would perform before wiping everything and going to beta after a new setup. We huddled and concocted a strategy. Picked up our tools and went on the field to beat the cluster to a pulp one last time. Our goal was explicitly to overwhelm the cluster as...

Splitting light Season 2 Episode 22 Too many logs If you are no longer interested in the newsletter, please unsubscribe I’ve rarely seen people talk about this effect. The effect being the amplification of requests. This effect can overwhelm your system. We had to deal with it. The object storage, at least OpenIO, was a collection of distributed services. You might call them micro services if you want. That had implications. When a request comes in, from the user perspective, it’s a single...

Splitting light Season 2 Episode 21 All nighter If you are no longer interested in the newsletter, please unsubscribe As we were moving forward, in mid June 2018, we hit a point where we needed to be able to check the logs of the cluster as a whole. The way we had done it until then was manually connecting to the machines and opening the right files to look inside. This was no longer viable. One of the main office rooms (1) Scaleway’s monitoring team had done a metric stack which we already...