Tasks/New-WhiskeyProGetUniversalPackage.ps1
|
function New-WhiskeyProGetUniversalPackage { <# .SYNOPSIS Creates a ProGet universal package. .DESCRIPTION The `ProGetUniversalPackage` task creates a universal ProGet package of your application. You must specify a package name and description with the Name and Description properties, respecitvely. Specify the directories and files you want in the package with the `Path` property. The paths should be relative to the whiskey.yml file. Each item is added to your package at the same relative path. The contents directories are filtered by the `Include` property, which is a list of filenames and/or wildcard patterns. Only files that match at least one item in this list will be included in your package. We use whitelists so we know what files are getting packaged and deployed. Without a whitelist, any file put into a directory that gets packaged would be included. This is a security risk. We can audit whitelists. Using whitelists also helps keep the size of our packages to a minimum. This PowerShell command will create a YAML whitelist for all files under a path: Get-ChildItem -Path $PATH -Recurse | Select-Object -ExpandProperty 'Extension' | Select-Object -unique | Sort-Object | ForEach-Object { '- "*{0}"' -f $_ } The package is saved to the output directory as `$Name.upack` where `$Name is replaced with the name of your package. A version.json file is put into the root of your package. It contains the version information for the current build. It looks like this: { "SemVer2": "2017.412.286-rc.1+master.1acb317", "SemVer2NoBuildMetadata": "2017.412.286-rc.1", "PrereleaseMetadata": "rc.1", "BuildMetadata": "master.1acb317", "SemVer1": "2017.412.286-rc1", "Version": "2017.412.286" } It has these properties: * `BuildMetadata`: this is either the build number, branch, and commit ID or the branch and commit ID, each separated by a period .. * `PrereleasMetadata`: this is any pre-release metadata from the Version property from your whiskey.yml file. If there is not Version in your whiskey.yml file, this field will be empty. * `SemVer2`: the full semantic version of the application. * `SemVer2NoBuildMetadata`: this is the version number used when creating packages. We omit the build metadata so that we don't upload duplicate packages (i.e. every build generates unique build metadata even if that version of code was already built and published. * `SemVer1`: a semantic version compatible for use with systems that don't yet support the v2 semantic version spec, e.g. NuGet. * `Version`: the MAJOR.MINOR.PATCH version number of the application. ## Properties * `Name` (mandatory): the package's name. * `Description` (mandatory): the package's description. This shows in ProGet and helps people know about your application. * `Path` (mandatory): the directories and filenames to include in the package. Each path must relative to the whiskey.yml file. You can change the root path the task uses to resolve these paths with the `SourceRoot` property. Each item is added to the package at the same relative path as its source item. If you have two paths with the same name, the second item will replace the first. You can customize the path of the item in the package by converting the value into a key/value pair, e.g. `source_dir\source_file.ps1`: `destination_dir\destination_file.ps1`. * `Include` (mandatory): a whitelist of wildcards and file names. All directories in the `Path` property are filtered with this list, i.e. only items under each directory in `Path` that matches an item in `Include` will be added to your package. * `Exclude`: a list of wildcards, file names, and directory names to exclude from the package. Sometimes a whitelist can be a little greedy and include some files or directories you might not want. Any file or directory that matches an item in this list will be excluded from the package. * `ThirdPartyPath`: a list of directores and files that should be included in the package unfiltered. These are paths that are copied without using the Include or Exclude elements. This is useful to include items you depend on but have no control over, like Node.js applications' node_modules directory. * `SourceRoot`: this changes the root path used to resolve the relative paths in the Path property. Use this element when your application's root directory isn't the same directory your whiskey.yml file is in. This path should be relative to the whiskey.yml file. * `CompressionLevel`: the compression level to use when creating the package. Can be a value from 1 (fastest but largest file) to 9 (slowest but smallest file). The default is `1`. * `Version`: the package version (MAJOR.MINOR.PATCH), without any prerelease or build metadata. Usually, the version for the current build is used. Prerelease and build metadata for the current build is added. ## Examples ### Example 1 BuildTasks: - ProGetUniversalPackage: Name: Example1 Description: This package demonstrates the YAML for using the ProGetUniversalPackage task. Path: - bin - REAMDE.md Include: - "*.dll" The above example shows the YAML for creating a ProGet Universal Package. Given the file system looks like this: bin\ Assembly.dll Assembly.pdb Assembly.xml src\ Assembly.cs README.md whiskey.yml The package will look like this: package\ bin\ Assembly.dll README.md version.json upack.json Because the `Include` list only includes `*.dll`, the `Assembly.pdb` and `Assembly.xml` files are not included in the package. The `version.json` file is created by the task and contains the version metadata for this build. The `upack.json` file contains the universal package metadata required by ProGet. ### Example 2 BuildTasks: - ProGetUniversalPackage Name: Example2 Description: This package demonstrates the YAML for using the ProGetUniversalPackage task. Path: - bin Include: - "*.dll" - "*.pdb" Exclude: - SomeOtherAssembly.pdb The above demonstrates how to use the `Exclude` property to exclude files from the package. If this is what's on the file system: bin\ Assembly.dll Assembly.pdb SomeOtherAssembly.dll SomeOtherAssembly.pdb whiskey.yml The package will look like this: package\ bin\ Assembly.dll Assembly.pdb SomeOtherAssembly.dll version.json upack.json Note that the `bin\SomeOtherAssembly.pdb` file is not in the package even though it matches an item in the `Include` whitelist. It is excluded because it matches an item in the `Exclude` blacklist. ## Example 3 BuildTasks: - ProGetUniversalPackage Name: Example3 Description: This package demonstrates how the `ThirdPartyPath` property works. Path: - dist Include: - "*.js" - "*.json" - "*.css" ThirdPartyPath: - node_modules Thie example demonstrates how to use the `ThirdPartyPath` property. If the file system looks like this: dist\ index.js default.css data.json node_modules\ rimraf\ LICENSE otherfiles whiskey.yml the package will look like this: package\ dist\ index.js default.css data.json node_modules\ rimraf\ LICENSE otherfiles version.json upack.json Notice that all files/directories under `node_modules` are included because `node_modules` is in the `ThirdPartyPath` list. Directores in `ThirdPartyPath` are included in the package as-is, with no filtering. ## Example 4 BuildTasks: - ProGetUniversalPackage Name: Example4 Description: This package demonstrates how the customize paths in the package. Path: - source: destination Include: - "*.dll" Thie example demonstrates how to use customize the path an item should have in the package. If this is the file system: source\ Assembly.dll Assembly.pdb whiskey.yml the package will look like this: package\ destination\ Assembly.dll version.json upack.json Notice that the `source` directory is added to the package as `destination`. This is done by making the value of an item in the `Path` list from a string into a key/value pair (e.g. `key: value`). ## Example 5 BuildTasks: - ProGetUniversalPackage Name: Example5 Description: Demonstration of the SourceRoot property. SourceRoot: Whiskey Path: - Functions - "*.ps*1" Include: - "*.ps*1" ThirdPartyPath: - ProGetAutomation Thie example demonstrates how to change the root directory the task uses to resolve the relative paths in the `Path`. If the file system is: Whiskey\ Functions\ New-WhiskeyContext.ps1 Whiskey.psd1 Whiskey.psm1 BuildMasterAutomation\ BuildMasterAutomation.psd1 BuildMasterAutomation.psm1 whiskey.yml the package will be: package\ Functions\ New-WhiskeyContext.ps1 Whiskey.psd1 Whiskey.psm1 BuildMasterAutomation\ BuildMasterAutomation.psd1 BuildMasterAutomation.psm1 version.json upack.json Notice that the top-level Whiskey directory found on the file system isn't part of the package. Because it is defined as the source root, it is considered the root of the files to put in the package, so is omitted from the package. #> [CmdletBinding()] [Whiskey.Task("ProGetUniversalPackage")] param( [Parameter(Mandatory=$true)] [object] $TaskContext, [Parameter(Mandatory=$true)] [hashtable] $TaskParameter ) Set-StrictMode -Version 'Latest' Use-CallerPreference -Cmdlet $PSCmdlet -SessionState $ExecutionContext.SessionState foreach( $mandatoryName in @( 'Name', 'Description' ) ) { if( -not $TaskParameter.ContainsKey($mandatoryName) ) { Stop-WhiskeyTask -TaskContext $TaskContext -Message ('Property ''{0}'' is mandatory.' -f $mandatoryName) } } # ProGet uses build metadata to distinguish different versions, so we can't use a full semantic version. $version = $TaskContext.Version if( $TaskParameter.ContainsKey('Version') ) { if( ($TaskParameter['Version'] -notmatch '^\d+\.\d+\.\d+$') ) { Stop-WhiskeyTask -TaskContext $TaskContext -Message ('Property ''Version'' is invalid. It must be a three part version number, i.e. MAJOR.MINOR.PATCH.') } [SemVersion.SemanticVersion]$semVer = $null if( -not ([SemVersion.SemanticVersion]::TryParse($TaskParameter['Version'], [ref]$semVer)) ) { Stop-WhiskeyTask -TaskContext $TaskContext -Message ('Property ''Version'' is not a valid semantic version.') } $semVer = New-Object 'SemVersion.SemanticVersion' $semVer.Major,$semVer.Minor,$semVer.Patch,$version.SemVer2.Prerelease,$version.SemVer2.Build $version = New-WhiskeyVersionObject -SemVer $semVer } $name = $TaskParameter['Name'] $description = $TaskParameter['Description'] $exclude = $TaskParameter['Exclude'] $thirdPartyPath = $TaskParameter['ThirdPartyPath'] $compressionLevel = 1 if( $TaskParameter['CompressionLevel'] ) { $compressionLevel = $TaskParameter['CompressionLevel'] | ConvertFrom-WhiskeyYamlScalar -ErrorAction Ignore if( $compressionLevel -eq $null ) { Stop-WhiskeyTask -TaskContext $TaskContext -Message ('Property ''CompressionLevel'': ''{0}'' is not a valid compression level. It must be an integer between 0-9.' -f $TaskParameter['CompressionLevel']); } } $parentPathParam = @{ } $sourceRoot = $TaskContext.BuildRoot if( $TaskParameter.ContainsKey('SourceRoot') ) { $sourceRoot = $TaskParameter['SourceRoot'] | Resolve-WhiskeyTaskPath -TaskContext $TaskContext -PropertyName 'SourceRoot' $parentPathParam['ParentPath'] = $sourceRoot } $badChars = [IO.Path]::GetInvalidFileNameChars() | ForEach-Object { [regex]::Escape($_) } $fixRegex = '[{0}]' -f ($badChars -join '') $fileName = '{0}.{1}.upack' -f $name,($version.SemVer2NoBuildMetadata -replace $fixRegex,'-') $outDirectory = $TaskContext.OutputDirectory $outFile = Join-Path -Path $outDirectory -ChildPath $fileName $tempRoot = $TaskContext.Temp $tempPackageRoot = Join-Path -Path $tempRoot -ChildPath 'package' New-Item -Path $tempPackageRoot -ItemType 'Directory' | Out-Null $upackJsonPath = Join-Path -Path $tempRoot -ChildPath 'upack.json' @{ name = $name; version = $version.SemVer2NoBuildMetadata.ToString(); title = $name; description = $description } | ConvertTo-Json | Set-Content -Path $upackJsonPath # Add the version.json file @{ Version = $version.Version.ToString(); SemVer2 = $version.SemVer2.ToString(); SemVer2NoBuildMetadata = $version.SemVer2NoBuildMetadata.ToString(); PrereleaseMetadata = $version.SemVer2.Prerelease; BuildMetadata = $version.SemVer2.Build; SemVer1 = $version.SemVer1.ToString(); } | ConvertTo-Json -Depth 1 | Set-Content -Path (Join-Path -Path $tempPackageRoot -ChildPath 'version.json') function Copy-ToPackage { param( [Parameter(Mandatory=$true)] [object[]] $Path, [Switch] $AsThirdPartyItem ) foreach( $item in $Path ) { $override = $False if( (Get-Member -InputObject $item -Name 'Keys') ) { $sourcePath = $null $override = $True foreach( $key in $item.Keys ) { $destinationItemName = $item[$key] $sourcePath = $key } } else { $sourcePath = $item } $pathparam = 'path' if( $AsThirdPartyItem ) { $pathparam = 'ThirdPartyPath' } $sourcePaths = $sourcePath | Resolve-WhiskeyTaskPath -TaskContext $TaskContext -PropertyName $pathparam @parentPathParam if( -not $sourcePaths ) { return } foreach( $sourcePath in $sourcePaths ) { $relativePath = $sourcePath -replace ('^{0}' -f ([regex]::Escape($sourceRoot))),'' $relativePath = $relativePath.Trim("\") if( -not $override ) { $destinationItemName = $relativePath } $destination = Join-Path -Path $tempPackageRoot -ChildPath $destinationItemName $parentDestinationPath = ( Split-Path -Path $destination -Parent) #if parent doesn't exist in the destination dir, create it if( -not ( Test-Path -Path $parentDestinationPath ) ) { New-Item -Path $parentDestinationPath -ItemType 'Directory' -Force | Out-String | Write-Verbose } if( (Test-Path -Path $sourcePath -PathType Leaf) ) { Copy-Item -Path $sourcePath -Destination $destination } else { $destinationDisplay = $destination -replace [regex]::Escape($tempRoot),'' $destinationDisplay = $destinationDisplay.Trim('\') if( $AsThirdPartyItem ) { $robocopyExclude = @() $whitelist = @( ) $operationDescription = 'packaging third-party {0} -> {1}' -f $sourcePath,$destinationDisplay } else { if( -not $TaskParameter['Include'] ) { Stop-WhiskeyTask -TaskContext $TaskContext -Message ('Property ''Include'' is mandatory because ''{0}'' is in your ''Path'' property and it is a directory. The ''Include'' property is a whitelist of files (wildcards supported) to include in your package. Only files in directories that match an item in the ''Include'' list will be added to your package.' -f $sourcePath) return } $robocopyExclude = & { $TaskParameter['Exclude'] ; (Join-Path -Path $destination -ChildPath 'version.json') } $operationDescription = 'packaging {0} -> {1}' -f $sourcePath,$destinationDisplay $whitelist = & { 'upack.json' ; $TaskParameter['Include'] } } Write-Verbose -Message $operationDescription Invoke-WhiskeyRobocopy -Source $sourcePath.trim("\") -Destination $destination.trim("\") -WhiteList $whitelist -Exclude $robocopyExclude | Write-Verbose } } } } if( $TaskParameter['Path'] ) { Copy-ToPackage -Path $TaskParameter['Path'] } if( $TaskParameter.ContainsKey('ThirdPartyPath') -and $TaskParameter['ThirdPartyPath'] ) { Copy-ToPackage -Path $TaskParameter['ThirdPartyPath'] -AsThirdPartyItem } Write-Verbose -Message ('Creating universal package {0}' -f $outFile) & $7z 'a' '-tzip' ('-mx{0}' -f $compressionLevel) $outFile (Join-Path -Path $tempRoot -ChildPath '*') Write-Verbose -Message ('returning package path ''{0}''' -f $outFile) $outFile } |