Cocaine

Cocaine (Configurable Omnipotent Custom Applications Integrated Network Engine) is an open-source PaaS (platform as a service) system for creating custom cloud hosting apps that are similar to Google App Engine or Heroku.

Any library or service can be implemented as a service in Cocaine using a special API. Several indispensable services have already been implemented this way, including a service for detecting a user's region or language, a service for accessing MongoDB storage, and a URL fetcher.

Architecture

The cloud is made up of one or more independent machines that have the Cocaine server (Cocained) installed.

Users do not know anything about the location of the services they are accessing — only the address of the load balancer and the app name are available to them. User requests are sent to the load balancer, which passes them to the cloud. In the cloud, the optimal machine is selected for each request, and then the request is executed.

The infrastructure details and the app's environment settings are hidden from the developer, as well. The developer only needs to send the code to the cloud and write a special manifest for executing the code. It is not necessary to set up anything else, such as databases, key-value stores, or HTTP clients. There are services that do this, which from the programmer's point of view are native modules for the programming language the code is written in. For example, the Storage service lets users access Elliptics storage, and the Uatraits service uses data about a client's user agent and its HTTP headers to determine the characteristics of the device that sent the request.

App isolation

We have chosen the Docker project as a technology for infrastructure virtualization and app isolation. Docker is an open-source technology that provides an easy and effective way to create lightweight, portable and self-sufficient containers from any app. Once created, such containers can run in virtually any environment, from the developer's and tester's laptops to production clusters with thousands of nodes.

The Docker technology is based on the well-known Linux Containers (LXC), which, by themselves, only provide the ability to start apps in an isolated environment, due to the use of namespaces and cgroups. In contrast to full virtualization environments such as Xen and KVM, containers share a common kernel and cannot provide device emulation, but their use doesn't incur additional overhead and they start almost instantly. In addition to containerization, Docker provides tools for configuring networks and creating images of apps using a layered filesystem designed to minimize overhead on deployment and distribution of apps.

On our platform, Docker support is implemented as a plugin, which connects to the dockerd daemon and controls it using a rich REST API.

For more information about Docker, see its website. You may also want to view the presentation by Jérôme Petazzoni, senior engineer of dotCloud (the company behind Docker) given at Yet Another Conference 2013 in Moscow.

Applying the technology

Where Yandex uses Cocaine