Cake.ParallelExecution

Форк
0

README.md

Cake.ParallelExecution

A fairly simple solution to speed up your 🍰 Cake tasks using parallel execution

Table of Contents

Overview

Are you fed up with long running build processes? That's enough! Let's get rid of it. The proposed solution allows you to save lots of time during daily routines. The steps are pretty easy:

  1. Review your Cake build process and find some independent and long running tasks. For instance, it might be separate build and deployment tasks for backend and frontend parts of the solution. Usually such tasks do not depend on the order of execution, nor on each other. So they can be started separately. Found some? Good. Now let's create a wrappers for them.
  2. Add new import for your
    build.cake
    script if it doesn't exist already:
#addin nuget:?package=Cake.Powershell
  1. Wrap your task(s) with a call to a new Powershell instance:
start powershell {.\\build.ps1 -Target 'Your-Task-Name'; echo 'Parallel task execution finished.'; Read-Host}
  1. Create a new parallel wrapper:
// Initializes a task to run in parallel
Task("Init-Parallel-Your-Task-Name")
.Does(() => {
// Runs in parallel
StartPowershellScript("start powershell {.\\build.ps1 -Target 'Your-Task-Name'; echo 'Parallel task execution finished.'; Read-Host}");
});
  1. Replace your sequental task with the parallel one. Change:
// Initializes a task to run in parallel
Task("Build")
.IsDependentOn("Some-Task-Name")
.IsDependentOn("Your-Task-Name")
.IsDependentOn("Last-Task-Name");

To:

// Initializes a task to run in parallel
Task("Build")
.IsDependentOn("Some-Task-Name")
.IsDependentOn("Init-Parallel-Your-Task-Name")
.IsDependentOn("Last-Task-Name");
  1. Replace any other parallelizable task if needed.

Benchmarks

Let's compare sequential and parallelized runs of typical build and deployment process for web application. It consists of the 2 major tasks - backend and frontend.

Test bench

I'm using a laptop with Intel Core i5-10310U processor which constantly suffers from 15W power limit. The difference may be even more significant, if you have let's say AMD Ryzen 9 5950X with 16 cores/32 threads and no power limit. You are welcome to share your results in discussions.

CPUIntel Core i5-10310U
Cores4
Threads8

Time measurements

Sequential run

RunTime, mm:ss.ms
108:03.1562279
208:07.0316733
308:59.2787849
408:34.0494476
508:37.1383982
AVERAGE08:28.1300000

Parallel run

RunTime, mm:ss.ms
105:50.4245998
206:33.5425879
306:24.1244980
406:31.0146683
506:55.8249122
AVERAGE06:26.9800000

Comparison

TypeAverage time, mm:ss.msTime saved, mm:ss.msPerformance gain
Sequential08:28.13 508,1300
Parallel06:26.98 386,9802:01.15~ 23%

As you can see, we managed to save almost a quarter of the time using parallel tasks. Moreover, the worst parallelized result still better than the best sequential with the minute break. It's 18% of time saving!

CPU Utilization

Measurements have been taken from Intel Extreme Tuning Utility monitoring.

Sequential

Sequential

Parallel

Parallel

Comparison

TypeAverage active coreAverage CPU utilization
Sequential128 %
Parallel345 %

It is clear that processor resources are used most efficiently with parallel tasks execution.

How To Use

Explore

/examples/
folder to know how it works.

Pitfalls

As it was mentioned earlier this solution is applicable to parallelizable and long-running tasks. If the task run time is less than Cake build time it may be better to leave it as is and run in a sequential order.

Known issues

See the Issues section for the actual information.

  1. If you have a multidiplay workstation, the

    start
    command might open a new powershell window outside of the screen in some cases.

  2. A failed task does not cancel other tasks running in parallel. In fact, this behavior may be preferable because you don't have to restart all the tasks.

Useful Tips

  1. Try to determine which parallelizable task is the most long-running. If possible, leave it in a main powershell instance, so other parallel task(s) will finish faster even with Cake script build step.

  2. If task dependencies tree is way too complex you can invoke parallel tasks from another parallel tasks. But don't forget about Cake script build process duration. Sometimes it's better to run small tasks sequentially.

Описание

A pretty simple solution to speed up your Cake tasks using parallel execution

Языки

Text

Сообщить о нарушении

Использование cookies

Мы используем файлы cookie в соответствии с Политикой конфиденциальности и Политикой использования cookies.

Нажимая кнопку «Принимаю», Вы даете АО «СберТех» согласие на обработку Ваших персональных данных в целях совершенствования нашего веб-сайта и Сервиса GitVerse, а также повышения удобства их использования.

Запретить использование cookies Вы можете самостоятельно в настройках Вашего браузера.