Research on Several Self-Hosted CI Tools

Among existing open source tools, I normally use azure’s pipeline (private projects) and travis-ci (public projects). Personally I prefer azure’s: you can build your own agent, and after CI finishes it can complete the deployment work along the way.

Research on Self-Hosted CI Tools

The git repository used in my department is not an open source git repo like gitlab, and doesn’t support third-party apps. Currently only webhooks are usable, so CI tool choices are much fewer; even using open source CI tools, you have to customize them yourself. Also, since I already maintain a PaaS platform and the CI tool serves it, I’ll also consider this platform’s related issues.

This post only discusses CI systems that can be self-hosted, with brief discussion of feature usage, user learning curve, customization difficulty, and deployment difficulty.

Most CI tool information comes from:

https://ligurio.github.io/awesome-ci/

https://alternativeto.net/software/travis-ci/?platform=self-hosted

Jenkins

I’ve written some Jenkins configuration files myself before. Personal view: independent large projects are quite suitable for jenkins. The Jenkinsfile is very powerful — it can start docker images, generate virtualenv environments, and customize notifications.

One bad thing about Jenkins is you have to write too much. Deployment isn’t huge for applications on a PaaS platform, but there are many of them. Having every application developer write a jenkinsfile is bad for maintenance and increases cost. For PaaS platform applications, the needed CI is just simply running a few unit test statements; at most you might need to start a mysql or redis. Using native Jenkinsfile feels like using a sledgehammer to crack a nut.

Agola

Development stack: Golang + Vue Database: etcd

https://agola.io/

Overall, I don’t like agola‘s documentation. Setting up gitea and agola per its plan may succeed, but you’ll feel lost in the fog. I strongly suggest starting with Github‘s API, and you need a public domain and server. The program has some BUGs; users may feel frustrated by inexplicable errors — be mentally prepared to hack.

I tried it quite a lot. agola uses oauth2 login for authentication and authorization. It can be used as an app of Github or Gitlab.

agola’s runtime flow:

1. Apply for a Github app, get the clientid and secret
2. Use agola remotesource create to add it as a Github app
3. User logs in, creates a project (the program sets the hook in github behind the scenes)
4. When the user modifies code and the hook triggers, the build runs

Pros and Cons Analysis

Pros

1. Open source; you can deploy it yourself
2. Fairly complete; you can make some changes within the framework

Cons:

1. The user interface isn’t very comfortable — e.g. some buttons are inconvenient to click; clicking doesn’t match the imagined function; task page tabs are too big, making content look empty
2. Although it uses yaml files, they’re a bit verbose; reading the docs I still couldn’t find preloading for services like redis or mysql
3. The program still has BUGs — when loading pages, / somehow gets escaped to %2F
4. Build-time variable settings have a page, but I still couldn’t find where to set them

If a PaaS project wants to integrate this CI tool, a fairly feasible plan is:

For the PaaS platform:

1. Implement an oauth authentication login and a rough authorization system, so agola can use the PaaS platform to log in and get some of its projects.
2. Webhooks need the platform to relay
3. Currently agola’s yaml code is fairly complex; you may need to simply implement a yaml converter

For the agola platform:

Refer to github’s oauth to add some code; also fix some BUGs (like the path problem above).

Abstruse CI

Development stack: Golang + Angular

https://github.com/bleenco/abstruse

The last release was December ‘18. After trying the docker startup, like agola it can be integrated via oauth, or use its own user system. But after integrating oauth login, I could only see projects on github, and I never figured out how to write the CI file to make it run. Gave up.

Concourse-CI

Development stack: Golang

A friend had already set it up and written a blog post; I referred to his:

https://www.boris1993.com/tools/concourse/concourse-quick-start.html

This is a relatively independent CI tool. At first I thought its features were weak, but after using it I found it actually focuses more on CI/CD functionality.

Pros and Cons Analysis

Pros

1. Fairly complete
2. Compared with complex Jenkins, its configuration is already very simple
3. The pipeline part has fairly complete documentation and example projects — much better than agola

Cons

1. The user configuration interface has almost no functionality; all operations rely on the command-line tool fly
2. The user system must be built separately, and it can’t support oauth2 etc. If you plan from the start to support Gitlab or Github projects, don’t consider it

A Fairly Feasible Integration Plan

For the Concourse-CI platform:

1. After building the platform, use OpenID as the verification tool for login
2. User group information needs maintenance

For the PaaS platform:

1. After users create projects, render a Concourse-CI pipeline and bind the pipeline to the corresponding group
2. Periodically sync user group information into the CI system

Compared with agola, modifications on the PaaS platform side may be fewer, since a complete oauth2 system needn’t be implemented.

Summary

There are two styles of CI systems: one like TravisCI, which can automatically import projects from repositories and parse the ci files in projects; the other like Jenkins, requiring manual project creation and specifying the CI flow.

Agola is a system similar to a local TravisCI, compatible with multiple online repositories — basically anything providing oauth support can be integrated into the system. For existing systems wanting to integrate, it’s best to provide basic oauth2 system functionality; future extensibility is also strong.

Concourse-CI is more like a slimmed-down Jenkins — of course it’s lighter. Its yaml descriptive statements may not be as powerful as groovy statements, but as CI it’s already sufficient. Referring to TravisCI’s features, simple extensions should also suffice. The development and integration cycle is short; because integration with the existing platform is relatively tight, future extensibility may not be as strong.