免费注册 查看新帖 |

Chinaunix

  平台 论坛 博客 文库
最近访问板块 发新帖
查看: 30783 | 回复: 0

Faster Builds with Container-Based Infrastructure and Docker [复制链接]

论坛徽章:
32
CU大牛徽章
日期:2013-05-20 10:45:13每日论坛发贴之星
日期:2015-09-07 06:20:00每日论坛发贴之星
日期:2015-09-07 06:20:00数据库技术版块每日发帖之星
日期:2015-12-13 06:20:0015-16赛季CBA联赛之江苏
日期:2016-03-03 11:56:13IT运维版块每日发帖之星
日期:2016-03-06 06:20:00fulanqi
日期:2016-06-17 17:54:25IT运维版块每日发帖之星
日期:2016-07-23 06:20:0015-16赛季CBA联赛之佛山
日期:2016-08-11 18:06:41JAVA
日期:2016-10-25 16:09:072017金鸡报晓
日期:2017-01-10 15:13:292017金鸡报晓
日期:2017-02-08 10:33:21
发表于 2015-11-08 12:25 |显示全部楼层
Stability and reliability in your builds is the one thing we aimed to give since Travis CI came about. But we haven't always been able to live up to this expectation. Network issues, insufficient build capacity (for open source projects) which in turn lead to long wait times for your build to start, constrained CPU and memory resources in the builds environment, lack of caching for open source projects. As most our current Linux stack runs on a private cloud, we've also had issues adding capacity, as we had to go through the process of ordering and waiting for more capacity.

Today we're happy to announce our new build infrastructure, which addresses some of these issues. It's available for private and open source repositories as of today!

What benefits does this new infrastructure have?
Builds start in seconds

Rather than wait for builds to start for a long time, your builds now start in less than 10 seconds. Assuming that capacity is available (which is now a lot easier for us to scale), you'll see your builds going from being scheduled to starting in only a few seconds.

Faster builds

We've been helping projects move over to this new stack, and we've seen much better build times for most of them. The mileage may still vary based on how heavy your builds are on I/O, but most projects should see an improvement. We'd love to hear from your if you don't or if you see slower build times.

More available resources in a build container

The build containers in our legacy build infrastructure have had 1.5 cores (with burst capacity) and 3GB of memory. The CPU resources are now guaranteed, which means less impact by noisy neighbors on the same host machine. Build times throughout the day should be much more consistent on the new container stack.

The new containers have 2 dedicated cores available and 4 GB of memory.

Better network capacity, availability and throughput

Our new stack is running on EC2, which means much faster network access to most services, especially those hosted on EC2 as well. Access to S3 is now also a ton faster than on our legacy stack.

Caching available for open source projects

The best news for open source projects is that our build caching is now available for them too. That means faster build speeds by caching dependencies. Make sure to read the docs before trying it out.

For Ruby projects, it's as simple as adding cache: bundler to your .travis.yml.

Easier to scale

This might not be a direct benefit to our users and customers, but it is one for us. We can now respond to demand much quicker and increase our build capacity in mere minutes. That allows us to ensure that open source projects are less likely to hit capacity peaks and wait in line for too long until they run.

How can I use it?
Using our new container-based stack only requires one additional line in your .travis.yml:

sudo: false

What are the restrictions?
Using sudo isn't possible (right now)

Our new container stack uses Docker under the hood. This has a lot of benefits like fast boot times and better utilization of available machine resources. But it also comes with restrictions imposed by security.

At this point, it's not possible to use any command requiring sudo in your builds.

If you require sudo, for instance to install Ubuntu packages, a workaround is to use precompiled binaries, uploading them to S3 and downloading them as part of your build, installing them into a non-root directory.

Databases don't run off a memory disk

On our legacy and legacy legacy stack, both MySQL and PostgreSQL ran off a memory disk to greatly increase transaction and query speed. This can impact projects depending on their use of transaction, fixtures and general database usage, but the impact generally shouldn't be negative.

Tell me about this new stack?
Over the past year, we've been working on Travis CI Enterprise, our on-premises solution for GitHub Enterprise customers. Part of working on this stack required us to think about how to best run on custom infrastructure like internal OpenStack, VMware and other virtualization setups, and even on bare metal hardware.

Our legacy stack runs on a proprietary, managed cloud with APIs not available in our customers' datacenters. Both for packaging our entire stack for installation, but also for running builds in isolated environments, we started looking at Docker as an alternative about a year ago.

Our new stack can run on pretty much anything we or our customers would like it too. That allows for custom Travis CI Enterprise installations on EC2, Digital Ocean, Linode, Rackspace, or fully in-house infrastructure.

We've verified a few options upfront, but EC2 turned out to have the better performance and more reliable network.

Our new stack for hosted private and open source projects is running on EC2 inside of Docker containers. We've seen great performance both in the network and with compute resources, and the direct access to S3 makes build caching a lot faster.

I have more questions...
Can I provide my own Docker containers?

The build environments we're currently using on our Docker-based setup provide the same services, programming languages and tools offered by our legacy stack. We've been keeping them on par as much as possible.

The lack of sudo does make it hard to install custom libraries and services, running them on privileged ports, or simply using apt. The question for providing your own images is a natural one.

We're not there yet in allowing you to bring your own, but it's something we have in mind for the future.

Can I build Docker containers as part of my build?

Same as the above. It's not currently possible, mostly for security reasons, but is something we want to address in the future.

Can I run Docker inside Docker?

Running Docker containers as part of your build currently isn't possible because of questions related to security due to the underlying container technology. We have things planned to address this in the future.

Containers everywhere
This is only the beginning of offering better, faster and more reliable builds on Travis CI. We have a lot more things planned to improve stability, but we're excited to ship this today.
您需要登录后才可以回帖 登录 | 注册

本版积分规则 发表回复

  

北京盛拓优讯信息技术有限公司. 版权所有 京ICP备16024965号-6 北京市公安局海淀分局网监中心备案编号:11010802020122 niuxiaotong@pcpop.com 17352615567
未成年举报专区
中国互联网协会会员  联系我们:huangweiwei@itpub.net
感谢所有关心和支持过ChinaUnix的朋友们 转载本站内容请注明原作者名及出处

清除 Cookies - ChinaUnix - Archiver - WAP - TOP