Instagram
Azure Pipelines

Speeding up the Azure DevOps Extension tasks

I've been maintaining and using the Azure DevOps Extension tasks for years now and I've always been frustrated by the slowness of some steps.

Jesse Houwing

• 3 min read
Speeding up the Azure DevOps Extension tasks
A faster snail.

In particular the Publish Azure DevOps Extension task that spent quite a bit of time extracting the vsix prior to putting it back together.

Azure DevOps Extension Tasks - Visual Studio Marketplace
Extension for Azure DevOps - Azure Pipelines tasks for packaging and publishing Azure Devops and Visual Studio extensions to the Visual Studio Marketplace.
Visual Studio Marketplace
A little background

Every Azure DevOps Extension is published using a vsix file. Basically a zip file with some additional manifest metadata. Before the Azure DevOps Extension Tasks were created I had created a very similar extension that had built-in support to update the version number of tasks prior to publishing. I later contributed that code to the Azure DevOps extension tasks.

tfx-cli, the utility used to publish extensions from the command-line has never gained said functionality, it can do all kinds of magic for the extension manifest, but not for the embedded task manifests. Plus, it can only perform its magic when publishing from source, not from vsix.

To support these features in the Azure DevOps Extension tasks, we basically extract the whole vsix, patch the manifest files and zip the vsix back up.

When building the Azure DevOps Task Zips scripts - in which I do a very similar thing to generate a copy of each built-in task that can be installed side-by-side the original version - I discovered a way to speed up the process. Instead of extracting the whole vsix file, I only extracted the files that need patching:

  • extension.vsixmanifest
  • extension.vsomanifest
  • **/task.json
  • **/task.loc.json

Then patch the manifest files and then update the original vsix file with the patched manifest files.

The result is that instead of extracting an archive of 31MB and 30k files, I now only have to extract 22 files. Instead of searching 30k files for manifest files, I only have to search those 22 files. And instead of compressing 81MB and 30k files again, I now only have to add 1 updated file.

Extracting archive: D:\a\1\vsix\vsts-developer-tools-build-tasks.vsix

Files: 21
Size:       81485
Compressed: 31328189

============================================================================

Scanning the drive:
29 folders, 21 files, 81287 bytes (80 KiB)

Updating archive: jessehouwing.vsts-developer-tools-build-tasks-dev-4.0.143.gen.vsix

Keep old data in archive: 4710 folders, 30155 files, 85539629 bytes (82 MiB)
Add new data to archive: 29 folders, 1 file, 2888 bytes (3 KiB)
Log file showing the new process

The result that counts is that this process reduces the time to publish from a vsix file from 0:02:58 to 0:00:14. A 92% reduction in time!

When will you benefit from this time saving?

To gain these benefits you must:

  • Use the PublishAzureDevOpsExtension@4 task (only @4 will have this new feature).
  • Publish from vsix.
  • Use any of the override properties: extensionId, extensionName, extensionVersion,  publisherId, extensionTag, updateTasksVersion, updateTasksId, extensionVisibility, extensionPricing.

This feature now works on all operating systems: Windows, Linux and MacOS.

Unfortunately, this isn't the only "slow" part of the extension publishing process, but it's a part I can speed up.

In case you run into trouble, you can turn off this feature:

variables:
- name: PREVIEW_FAST_UPDATE
  value: false

Feedback wanted

Please let me know whether this works for you! I'm interested in both success and failure reports. Please reply to the issue below:

And if you've been using the Azure DevOps Extension tasks, please consider: