Q&A on the Compose Specification Community

Source:-infoq.com

Docker released a community for developing the Compose specification to help developers build cloud-native applications using compose. There’ve been different implementations of Docker compose to make it work in platforms like Kubernetes or AWS ECS. But Docker wants to work with the community to provide better support and define the future of Compose. The Compose spec is to let people use compose in different platforms and make any new features as a first-class feature in Compose.

InfoQ recently talked to Justin Cormack, security lead at Docker, to learn more about the Compose specification community.

InfoQ: What is the Compose specification community? What type of problems is the spec looking to solve?

Justin Cormack: Originally, there was only one implementation, the Docker compose in Python. But there have been more implementations where they’ve looked at what Docker compose does and tried to copy as much of it as possible. But sometimes they’re doing different things. Even within Docker, we have three different implementations like Docker compose for Kubernetes or Swarm. So, all these implementations had to copy what we did and then try to make it work, but they didn’t get any real influence on how to implement Docker compose on different platforms.

The Compose spec is to let people use compose in different platforms and make any new features as a first-class feature in Compose, allowing the community to influence how the spec progress. We knew there were problems with running compose on Kubernetes because it does networks and volumes differently. And other platforms like AWS ECS has its implementation to interact with other AWS services. So we thought the best thing to do is to make an open spec to help people use compose within other platforms different than Docker Swarm or the desktop.

InfoQ: What’s the progress you’ve made so far? What’s the current state of the spec?

Cormack: It’s not a complete spec yet, it only has what set of parameters exist and what other sections you can include in the YAML file, it doesn’t say exactly what it should do. It includes which bits in compose are optional because existing implementations don’t implement them all, and there’s also a reference for implementing tests.

We’ve also found many issues that we think need to be fixed, features we don’t like, or issues that the community has brought up. We have regular meetings with the community where we discuss the current issues and what to do about them. For instance, there isn’t a set of conformance tests to define a valid implementation yet, but we’d like to have that in the future.

Also, we’re looking in the short term for ways that people can do custom extensions if they have very platform-specific features, like in the case of Kubernetes or AWS ECS. Then, we’ll work out which of these features makes sense as general cross-platform extensions later.

InfoQ: Docker compose has more a developer focus. Do you think Docker compose could evolve for production usage as well?

Cormack: There are two ways people use compose at the moment. Some people use compose for development, and it’s not what goes to production at all. They use compose for testing, and it might not spin up the same services, but it’s similar. We also know people who have a workflow where the developers write compose files, and then another team writes another YAML file that corresponds, but it’s not an automatic version and might have different things.

We’re not trying to force people how and where to use compose. One of the things that we looked at when we were defining the spec was how many people are using compose. Only in GitHub, there are like 700,000 Docker compose files, and I’d say that most of them are not currently using it for production things. Usually, these files only define an example of how to spin up the system, so it’s easy to get started, but it’s not necessarily how you want to run the system in production.

So my feeling is that we should keep compose as things that developers care about. And then things that operations care about in regards to how to run systems in production. So your compose file wouldn’t be the whole description of production; it would just be the developer-facing pieces. It’s a sort of separation of concerns piece.

InfoQ: What’s in the roadmap for the specification? What would you like to see?

Cormack: We have some ideas, but we don’t have a road map as such. Part of the aim is to let the community drive where they want to go and fix people’s problems that they’ve got early. There are a few things that I would like to do, like not having to specify a specific version number in the compose file. Instead, I’d guide to a feature-driven approach where we look at the features you’re using and say, “Does that work on this platform?” And if not, return a helpful message saying, “This feature is not supported on this platform.”