Over the last few months, we have been rewriting the current codebase of dpl, our deployment tooling, and the result is a new major version: dpl v2.
Almost every line of code has been touched, code quality, test coverage and test quality greatly improved, and many of the supported service providers and volunteer contributors have been involved. We are excited about this huge community effort to improve and modernize dpl and give you the best deployment experience.
The diff stat on the main pull request for this work gives a vague impression of its extend:
Out of these ~16,000 lines, less than 7,000 are implementation code, the rest is documentation, tests, etc.
Today, we are releasing dpl v2.0.0-alpha.1 as a developer preview release, and would love for you to try it out.
You can opt in by adding this to your .travis.yml file:
deploy:
- provider: [your-provider]
edge: true
...This version will not become the default until we are are able to release a stable version, but as always, early feedback is greatly appreciated. We would love to welcome you in the community forum to share yours.
During the process, close to 30 pull requests have been ported and merged to the new codebase. If you have open issues that we have not been able to address please give us a nudge.
We have implemented a maturity model for dpl providers (per service “plugins”) that will help us communicate expectations about the stability of a provider programmatically, e.g., in your build logs, our README and the help output of the command line tool (e.g. if used outside of Travis CI).
The model includes 4 levels:
dev – the provider is in development (initial level)alpha – the provider is fully testedbeta – the provider has been in alpha for at least a month and successful real-world production deployments have been observedstable – the provider has been in beta for at least two months and there are no open issues that qualify as critical (such as deployments failing, documented functionality broken, etc.)Given the criteria above, most providers are currently in alpha: they are fully tested ports of previous functionality, but we have not seen any real world production deployments yet. Few of them are in dev since we need an end-to-end integration test, deploying to the respective service.
You can check your favorite deployment provider in the README under supported providers.
This is how you can help us move things forward:
alpha.dev. You could help us get an actual test deployment going in an automated way.Please let us know in the community forum.
As this is a new major version it is also the (rather rare) opportunity for us to correct early decisions that have turned out not that great.
Dpl v2 contains two changes that are breaking, at least in a very technical sense. We do not expect this to actually break many deployments, but sometimes things might behave slightly differently, so we call them “breaking”.
These are:
cleanup: truekeep_history: falseskip_cleanup is now deprecated, and cleanup is false by default. The default used to be true, so you had to opt out using skip_cleanup … and has been used a lot. Cleaning up the working directory from any left over build artifacts only made sense for few providers, so we have changed this default.
If you need the Git working directory to be cleaned up after your build you now need to opt into that using:
deploy:
- provider: [your-provider]
cleanup: trueAlso, for the GitHub Pages provider the original default behavior was to git push --force a new orphan branch onto your target repository and branch (e.g. gh-pages), erasing your Git history on that branch.
As this is not usually the desired behavior, an option keep_history was added later in order to not break that default.
We have taken the opportunity to change this default: the provider pages will now keep the existing history of the target repository and branch by default.
If for whatever reason you need your deployment to erase that history, starting with dpl v2 you will need to opt into that:
deploy:
- provider: pages
keep_history: falseOur goal with this rewrite was to keep existing functionality, not change it. But there are still functional changes that are worth mentioning:
Until now failed deployments did not always fail or error the build when using certain providers. This meant that you may not have been notified if your deployment failed.
Starting with dpl v2 a failed deployment always errors the build.
In one of the very few scenarios were you do want a failed deployment to be silently ignored, you can opt out of erroring the build using:
deploy
- provider: [your-provider]
allow_failure: trueDpl is an open source project written in Ruby, started in the early days of Travis CI. It is community-driven, but maintained by Travis CI developers and it has been adapted and used by and on many other platforms, such as GitLab.
Dpl supports 40 (forty) deployment providers, many of which we only learned of when they were contributed by you over the past 6 years that this project existed.
All of this accumulated domain knowledge is worth its weight in gold. It allows you to deploy to any of these services without having to learn much about the underlying tooling, and go through the hassle of setting things up on Travis CI manually.
Hiro has done a stellar job at dealing with this constantly incoming stream of knowledge and change. We like to look at his position as sitting at the receiving end of a firehose of unicorn fairy dust chaos for years.
However, we have not always done the best job in keeping the code quality high, tests consistent and complete, and all functionality in sync with changes on the respective services.
Maintaining centralized code in a diverse community environment is extremely educating and fun, but also can be challenging. Contributions sometimes come from developers not very familiar with Ruby, and very often from developers with different styles and approaches. Will still accept them, of course, if they add onto existing or new functionality.
Quite often we have accepted pull requests in a hurry though, and not always have we had the time and resources to help refactor, clean things up and smooth over consistency.
So we decided it was time for a rewrite.
As this is the announcement of a developer preview release we thought it might be interesting to you to include some more details on a rather technical level.
Our goal with this rewrite was to keep existing functionality, but improve code quality, maintainability, and consistency.
Here’s what has changed under the hood, in a nutshell:
dpl gem, and Ruby gems required by a give provider class are now installed and required at run time (not load time) using Bundler’s inline strategy. This simplifies things a lot, and removes the need for us to build sub gems, which was our previous strategy of dealing with incompatibilities between requirements of various provider implementations.All of this is good because:
Requiring all options to be specified gives us insight into what options we actually support, in a programmatical way. That means we can now auto-generate the README, and auto-validate the deploy section in the build config format specification, as well as the documentation.
Using a rich command line parser allows reducing the implementation code significantly, and enforces consistent behavior in dealing with unknown options, missing options, malformatted values, etc.
Separating shell commands and user facing log messages from the implementation code achieves several minor improvements. It makes it easy to get an overview of the commands used by a given provider class, compare messages and spot inconsistencies in wording. It also makes the implementation a lot more focused on the logic which helps with pattern recognition.
Providing a meaningful DSL means implementors have to worry less about our specific implementation requirements, and can focus more on the logic and behavior they need to provide. This is important to us because it enforces consistency in a very diverse and somewhat chaotic community environment, and thus reduces complexity and effort in communication by enforcing standards. (It also requires much better documentation for contributors, which we hope we have addressed by putting a lot of work into the README, CONTRIBUTING guide, and rubydocs.)
We would like to give a big shoutout to those of you that have invested their time and energy to work with us on making dpl better.
A big shoutout to: