Sync eng/common directory with azure-sdk-tools for PR 5027 (#4221)
* move things out from eng/common/scripts/job-matrix * fix broken link for CI/CD verification Co-authored-by: Konrad Jamrozik <kojamroz@microsoft.com>
This commit is contained in:
Azure SDK Bot
GitHub
parent
8f28aabe67
commit
bf5e9bb757
@ -1,561 +1,5 @@
|
||||
# Azure Pipelines Matrix Generator
|
||||
|
||||
* [Usage in a pipeline](#usage-in-a-pipeline)
|
||||
* [Matrix config file syntax](#matrix-config-file-syntax)
|
||||
* [Fields](#fields)
|
||||
* [matrix](#matrix)
|
||||
* [include](#include)
|
||||
* [exclude](#exclude)
|
||||
* [displayNames](#displaynames)
|
||||
* [$IMPORT](#import)
|
||||
* [Matrix Generation behavior](#matrix-generation-behavior)
|
||||
* [all](#all)
|
||||
* [sparse](#sparse)
|
||||
* [include/exclude](#includeexclude)
|
||||
* [displayNames](#displaynames-1)
|
||||
* [Filters](#filters)
|
||||
* [Replace/Modify/Append](#replacemodifyappend-values)
|
||||
* [NonSparseParameters](#nonsparseparameters)
|
||||
* [Under the hood](#under-the-hood)
|
||||
* [Testing](#testing)
|
||||
The documentation for Azure Pipelines Matrix Generator can be found at
|
||||
|
||||
|
||||
This directory contains scripts supporting dynamic, cross-product matrix generation for azure pipeline jobs.
|
||||
It aims to replicate the [cross-product matrix functionality in github actions](https://docs.github.com/free-pro-team@latest/actions/reference/workflow-syntax-for-github-actions#example-running-with-more-than-one-version-of-nodejs),
|
||||
but also adds some additional features like sparse matrix generation, cross-product includes and excludes, and programmable matrix filters.
|
||||
|
||||
This functionality is made possible by the ability for the azure pipelines yaml to take a [dynamic variable as an input
|
||||
for a job matrix definition](https://docs.microsoft.com/azure/devops/pipelines/process/phases?view=azure-devops&tabs=yaml#multi-job-configuration) (see the code sample at the bottom of the linked section).
|
||||
|
||||
## Usage in a pipeline
|
||||
|
||||
In order to use these scripts in a pipeline, you must provide a config file and call the matrix creation script within a powershell job.
|
||||
|
||||
For a single matrix, you can include the `/eng/common/pipelines/templates/jobs/archetype-sdk-tests-generate.yml` template in a pipeline (see /eng/common/scripts/job-matrix/samples/matrix-test.yml for a full working example):
|
||||
|
||||
```
|
||||
jobs:
|
||||
- template: /eng/common/pipelines/templates/jobs/archetype-sdk-tests-generate.yml
|
||||
parameters:
|
||||
MatrixConfigs:
|
||||
- Name: base_product_matrix
|
||||
Path: eng/common/scripts/job-matrix/samples/matrix.json
|
||||
Selection: all
|
||||
NonSparseParameters:
|
||||
- framework
|
||||
GenerateVMJobs: true
|
||||
- Name: sparse_product_matrix
|
||||
Path: eng/common/scripts/job-matrix/samples/matrix.json
|
||||
Selection: sparse
|
||||
GenerateVMJobs: true
|
||||
JobTemplatePath: /eng/common/scripts/job-matrix/samples/matrix-job-sample.yml
|
||||
AdditionalParameters: []
|
||||
CloudConfig:
|
||||
SubscriptionConfiguration: $(sub-config-azure-cloud-test-resources)
|
||||
Location: eastus2
|
||||
Cloud: Public
|
||||
MatrixFilters: []
|
||||
MatrixReplace: []
|
||||
PreGenerationSteps: []
|
||||
```
|
||||
|
||||
### A note regarding PreGenerationSteps
|
||||
|
||||
The generation template laid out above runs as its own job. A limitation of this method is that it disallows any runtime matrix customization due to the fact that an individual job clones the targeted build SHA. The stepList `PreGenerationSteps` allows users to update matrix json however they like prior to actually invoking the matrix generation. Injected steps are run after the repository checkout, but before any matrix generation is invoked.
|
||||
|
||||
## Matrix config file syntax
|
||||
|
||||
Matrix parameters can either be a list of strings, or a set of grouped strings (represented as a hash). The latter parameter
|
||||
type is useful for when 2 or more parameters need to be grouped together, but without generating more than one matrix permutation.
|
||||
|
||||
```
|
||||
"matrix": {
|
||||
"<parameter1 name>": [ <values...> ],
|
||||
"<parameter2 name>": [ <values...> ],
|
||||
"<parameter set>": {
|
||||
"<parameter set 1 name>": {
|
||||
"<parameter set 1 value 1": "value",
|
||||
"<parameter set 1 value 2": "<value>",
|
||||
},
|
||||
"<parameter set 2 name>": {
|
||||
"<parameter set 2 value 1": "value",
|
||||
"<parameter set 2 value 2": "<value>",
|
||||
}
|
||||
}
|
||||
}
|
||||
"include": [ <matrix>, <matrix>, ... ],
|
||||
"exclude": [ <matrix>, <matrix>, ... ],
|
||||
"displayNames": { <parameter value>: <human readable override> }
|
||||
```
|
||||
|
||||
See `samples/matrix.json` for a full sample.
|
||||
|
||||
### Fields
|
||||
|
||||
#### matrix
|
||||
|
||||
The `matrix` field defines the base cross-product matrix. The generated matrix can be full or sparse.
|
||||
|
||||
Example:
|
||||
```
|
||||
"matrix": {
|
||||
"operatingSystem": [
|
||||
"windows-2022",
|
||||
"ubuntu-18.04",
|
||||
"macos-11"
|
||||
],
|
||||
"framework": [
|
||||
"net461",
|
||||
"netcoreapp2.1",
|
||||
"net50"
|
||||
],
|
||||
"additionalTestArguments": [
|
||||
"",
|
||||
"/p:UseProjectReferenceToAzureClients=true",
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
#### include
|
||||
|
||||
The `include` field defines any number of matrices to be appended to the base matrix after processing exclusions.
|
||||
|
||||
```
|
||||
# matrix entry format:
|
||||
{
|
||||
"a": 1,
|
||||
"b": 2,
|
||||
"c": 3,
|
||||
}
|
||||
|
||||
# Include field in a matrix config
|
||||
{
|
||||
"include": [
|
||||
{
|
||||
"a": 1,
|
||||
"b": 2
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
#### exclude
|
||||
|
||||
The `exclude` field defines any number of matrices to be removed from the base matrix. Exclude parameters can be a partial
|
||||
set, meaning as long as all exclude parameters match against a matrix entry (even if the matrix entry has additional parameters),
|
||||
then it will be excluded from the matrix. For example, the below entry will match the exclusion and be removed:
|
||||
|
||||
```
|
||||
# matrix entry format:
|
||||
{
|
||||
"a": 1,
|
||||
"b": 2,
|
||||
"c": 3,
|
||||
}
|
||||
|
||||
# Exclude field in a matrix config
|
||||
{
|
||||
"exclude": [
|
||||
{
|
||||
"a": 1,
|
||||
"b": 2
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
#### displayNames
|
||||
|
||||
Specify any overrides for the azure pipelines definition and UI that determines the matrix job name. If some parameter
|
||||
values are too long or unreadable for this purpose (e.g. a command line argument), then you can replace them with a more
|
||||
readable value here. For example:
|
||||
|
||||
```
|
||||
"displayNames": {
|
||||
"/p:UseProjectReferenceToAzureClients=true": "UseProjectRef"
|
||||
},
|
||||
"matrix": {
|
||||
"additionalTestArguments": [
|
||||
"/p:UseProjectReferenceToAzureClients=true"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
#### $IMPORT
|
||||
|
||||
Matrix configs can also import another matrix config. The effect of this is the imported matrix will be generated,
|
||||
and then the importing config will be combined with that matrix (as if each entry of the imported matrix was a parameter).
|
||||
To import a matrix, add a parameter with the key `$IMPORT`:
|
||||
|
||||
```
|
||||
"matrix": {
|
||||
"$IMPORT": "path/to/matrix.json",
|
||||
"JavaVersion": [ "1.8", "1.11" ]
|
||||
}
|
||||
```
|
||||
|
||||
Importing can be useful, for example, in cases where there is a shared base matrix, but there is a need to run it
|
||||
once for each instance of a language version. Importing does not support overriding duplicate parameters. To achieve
|
||||
this, use the [Replace](#replacemodifyappend-values) argument instead.
|
||||
|
||||
The `Selection` and `NonSparseParameters` parameters are respected when generating an imported matrix.
|
||||
|
||||
The processing order is as follows:
|
||||
|
||||
Given a matrix and import matrix like below:
|
||||
```
|
||||
{
|
||||
"matrix": {
|
||||
"$IMPORT": "example-matrix.json",
|
||||
"endpointType": [ "storage", "cosmos" ],
|
||||
"JavaVersion": [ "1.8", "1.11" ]
|
||||
},
|
||||
"include": [
|
||||
{
|
||||
"operatingSystem": "windows",
|
||||
"mode": "TestFromSource",
|
||||
"JavaVersion": "1.8"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
### example-matrix.json to import
|
||||
{
|
||||
"matrix": {
|
||||
"operatingSystem": [ "windows", "linux" ],
|
||||
"client": [ "netty", "okhttp" ]
|
||||
},
|
||||
"include": [
|
||||
{
|
||||
"operatingSystem": "mac",
|
||||
"client": "netty"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
1. The base matrix is generated (sparse in this example):
|
||||
```
|
||||
{
|
||||
"storage_18": {
|
||||
"endpointType": "storage",
|
||||
"JavaVersion": "1.8"
|
||||
},
|
||||
"cosmos_111": {
|
||||
"endpointType": "cosmos",
|
||||
"JavaVersion": "1.11"
|
||||
}
|
||||
}
|
||||
```
|
||||
1. The imported base matrix is generated (sparse in this example):
|
||||
```
|
||||
{
|
||||
"windows_netty": {
|
||||
"operatingSystem": "windows",
|
||||
"client": "netty"
|
||||
},
|
||||
"linux_okhttp": {
|
||||
"operatingSystem": "linux",
|
||||
"client": "okhttp"
|
||||
}
|
||||
}
|
||||
```
|
||||
1. Includes/excludes from the imported matrix get applied to the imported matrix
|
||||
```
|
||||
{
|
||||
"windows_netty": {
|
||||
"operatingSystem": "windows",
|
||||
"client": "netty"
|
||||
},
|
||||
"linux_okhttp": {
|
||||
"operatingSystem": "linux",
|
||||
"client": "okhttp"
|
||||
},
|
||||
"mac_netty": {
|
||||
"operatingSystem": "mac",
|
||||
"client": "netty"
|
||||
}
|
||||
}
|
||||
```
|
||||
1. The base matrix is multipled by the imported matrix (in this case, the base matrix has 2 elements, and the imported
|
||||
matrix has 3 elements, so the product is a matrix with 6 elements:
|
||||
```
|
||||
"storage_18_windows_netty": {
|
||||
"endpointType": "storage",
|
||||
"JavaVersion": "1.8",
|
||||
"operatingSystem": "windows",
|
||||
"client": "netty"
|
||||
},
|
||||
"storage_18_linux_okhttp": {
|
||||
"endpointType": "storage",
|
||||
"JavaVersion": "1.8",
|
||||
"operatingSystem": "linux",
|
||||
"client": "okhttp"
|
||||
},
|
||||
"storage_18_mac_netty": {
|
||||
"endpointType": "storage",
|
||||
"JavaVersion": "1.8",
|
||||
"operatingSystem": "mac",
|
||||
"client": "netty"
|
||||
},
|
||||
"cosmos_111_windows_netty": {
|
||||
"endpointType": "cosmos",
|
||||
"JavaVersion": "1.11",
|
||||
"operatingSystem": "windows",
|
||||
"client": "netty"
|
||||
},
|
||||
"cosmos_111_linux_okhttp": {
|
||||
"endpointType": "cosmos",
|
||||
"JavaVersion": "1.11",
|
||||
"operatingSystem": "linux",
|
||||
"client": "okhttp"
|
||||
},
|
||||
"cosmos_111_mac_netty": {
|
||||
"endpointType": "cosmos",
|
||||
"JavaVersion": "1.11",
|
||||
"operatingSystem": "mac",
|
||||
"client": "netty"
|
||||
}
|
||||
}
|
||||
```
|
||||
1. Includes/excludes from the top-level matrix get applied to the multiplied matrix, so the below element will be added
|
||||
to the above matrix, for an output matrix with 7 elements:
|
||||
```
|
||||
"windows_TestFromSource_18": {
|
||||
"operatingSystem": "windows",
|
||||
"mode": "TestFromSource",
|
||||
"JavaVersion": "1.8"
|
||||
}
|
||||
```
|
||||
|
||||
## Matrix Generation behavior
|
||||
|
||||
#### all
|
||||
|
||||
`all` will output the full matrix, i.e. every possible permutation of all parameters given (p1.Length * p2.Length * ...).
|
||||
|
||||
#### sparse
|
||||
|
||||
`sparse` outputs the minimum number of parameter combinations while ensuring that all parameter values are present in at least one matrix job.
|
||||
Effectively this means the total length of a sparse matrix will be equal to the largest matrix dimension, i.e. `max(p1.Length, p2.Length, ...)`.
|
||||
|
||||
To build a sparse matrix, a full matrix is generated, and then walked diagonally N times where N is the largest matrix dimension.
|
||||
This pattern works for any N-dimensional matrix, via an incrementing index (n, n, n, ...), (n+1, n+1, n+1, ...), etc.
|
||||
Index lookups against matrix dimensions are calculated modulus the dimension size, so a two-dimensional matrix of 4x2 might be walked like this:
|
||||
|
||||
```
|
||||
index: 0, 0:
|
||||
o . . .
|
||||
. . . .
|
||||
|
||||
index: 1, 1:
|
||||
. . . .
|
||||
. o . .
|
||||
|
||||
index: 2, 2 (modded to 2, 0):
|
||||
. . o .
|
||||
. . . .
|
||||
|
||||
index: 3, 3 (modded to 3, 1):
|
||||
. . . .
|
||||
. . . o
|
||||
```
|
||||
|
||||
#### include/exclude
|
||||
|
||||
Include and exclude support additions and subtractions off the base matrix. Both include and exclude take an array of matrix values.
|
||||
Typically these values will be a single entry, but they also support the cross-product matrix definition syntax of the base matrix.
|
||||
|
||||
Include and exclude are parsed fully. So if a sparse matrix is called for, a sparse version of the base matrix will be generated, but
|
||||
the full matrix of both include and exclude will be processed.
|
||||
|
||||
Excludes are processed first, so includes can be used to add back any specific jobs to the matrix.
|
||||
|
||||
#### displayNames
|
||||
|
||||
In the matrix job output that azure pipelines consumes, the format is a dictionary of dictionaries. For example:
|
||||
|
||||
```
|
||||
{
|
||||
"net461_macOS1015": {
|
||||
"framework": "net461",
|
||||
"operatingSystem": "macos-11"
|
||||
},
|
||||
"net50_ubuntu1804": {
|
||||
"framework": "net50",
|
||||
"operatingSystem": "ubuntu-18.04"
|
||||
},
|
||||
"netcoreapp21_windows2022": {
|
||||
"framework": "netcoreapp2.1",
|
||||
"operatingSystem": "windows-2022"
|
||||
},
|
||||
"UseProjectRef_net461_windows2022": {
|
||||
"additionalTestArguments": "/p:UseProjectReferenceToAzureClients=true",
|
||||
"framework": "net461",
|
||||
"operatingSystem": "windows-2022"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The top level keys are used as job names, meaning they get displayed in the azure pipelines UI when running the pipeline.
|
||||
|
||||
The logic for generating display names works like this:
|
||||
|
||||
- Join parameter values by "_"
|
||||
a. If the parameter value exists as a key in `displayNames` in the matrix config, replace it with that value.
|
||||
b. For each name value, strip all non-alphanumeric characters (excluding "_").
|
||||
c. If the name is greater than 100 characters, truncate it.
|
||||
|
||||
#### Filters
|
||||
|
||||
Filters can be passed to the matrix as an array of strings, each matching the format of `<key>=<regex>`. When a matrix entry
|
||||
does not contain the specified key, it will default to a value of empty string for regex parsing. This can be used to specify
|
||||
filters for keys that don't exist or keys that optionally exist and match a regex, as seen in the below example.
|
||||
|
||||
Display name filters can also be passed as a single regex string that runs against the [generated display name](#displaynames) of the matrix job.
|
||||
The intent of display name filters is to be defined primarily as a top level variable at template queue time in the azure pipelines UI.
|
||||
|
||||
For example, the below command will filter for matrix entries with "windows" in the job display name, no matrix variable
|
||||
named "ExcludedKey", a framework variable containing either "461" or "5.0", and an optional key "SupportedClouds" that, if exists, must contain "Public":
|
||||
|
||||
```
|
||||
./Create-JobMatrix.ps1 `
|
||||
-ConfigPath samples/matrix.json `
|
||||
-Selection all `
|
||||
-DisplayNameFilter ".*windows.*" `
|
||||
-Filters @("ExcludedKey=^$", "framework=(461|5\.0)", "SupportedClouds=^$|.*Public.*")
|
||||
```
|
||||
|
||||
#### Replace/Modify/Append Values
|
||||
|
||||
Replacements for values can be passed to the matrix as an array of strings, each matching the format of `<keyRegex>=<valueRegex>/<replacementValue>`.
|
||||
The replace argument will find any permutations where the key fully matches the key regex and the value fully matches the value regex, and replace the value with
|
||||
the replacement specified.
|
||||
|
||||
NOTE:
|
||||
- The replacement value supports regex capture groups, enabling substring transformations, e.g. `Foo=(.*)-replaceMe/$1-replaced`. See the below examples for usage.
|
||||
- For each key/value, the first replacement provided that matches will be the only one applied.
|
||||
- If `=` or `/` characters need to be part of the regex or replacement, escape them with `\`.
|
||||
|
||||
For example, given a matrix config like below:
|
||||
|
||||
```
|
||||
{
|
||||
"matrix": {
|
||||
"Agent": {
|
||||
"ubuntu-1804": { "OSVmImage": "MMSUbuntu18.04", "Pool": "azsdk-pool-mms-ubuntu-1804-general" }
|
||||
},
|
||||
"JavaTestVersion": [ "1.8", "1.11" ]
|
||||
}
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
The normal matrix output (without replacements), looks like:
|
||||
|
||||
```
|
||||
$ ./Create-JobMatrix.ps1 -ConfigPath <test> -Selection all
|
||||
{
|
||||
"ubuntu1804_18": {
|
||||
"OSVmImage": "MMSUbuntu18.04",
|
||||
"Pool": "azsdk-pool-mms-ubuntu-1804-general",
|
||||
"JavaTestVersion": "1.8"
|
||||
},
|
||||
"ubuntu1804_111": {
|
||||
"OSVmImage": "MMSUbuntu18.04",
|
||||
"Pool": "azsdk-pool-mms-ubuntu-1804-general",
|
||||
"JavaTestVersion": "1.11"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Passing in multiple replacements, the output will look like below. Note that replacing key/values that appear nested within a grouping
|
||||
will not affect that segment of the job name, since the job takes the grouping name (in this case "ubuntu1804").
|
||||
|
||||
The below example includes samples of regex grouping references, and wildcard key/value regexes:
|
||||
|
||||
```
|
||||
$ $replacements = @('.*Version=1.11/2.0', 'Pool=(.*ubuntu.*)-general/$1-custom')
|
||||
$ ../Create-JobMatrix.ps1 -ConfigPath ./test.Json -Selection all -Replace $replacements
|
||||
{
|
||||
"ubuntu1804_18": {
|
||||
"OSVmImage": "MMSUbuntu18.04",
|
||||
"Pool": "azsdk-pool-mms-ubuntu-1804-custom",
|
||||
"JavaTestVersion": "1.8"
|
||||
},
|
||||
"ubuntu1804_20": {
|
||||
"OSVmImage": "MMSUbuntu18.04",
|
||||
"Pool": "azsdk-pool-mms-ubuntu-1804-custom",
|
||||
"JavaTestVersion": "2.0"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### NonSparseParameters
|
||||
|
||||
Sometimes it may be necessary to generate a sparse matrix, but keep the full combination of a few parameters. The
|
||||
NonSparseParameters argument allows for more fine-grained control of matrix generation. For example:
|
||||
|
||||
```
|
||||
./Create-JobMatrix.ps1 `
|
||||
-ConfigPath /path/to/matrix.json `
|
||||
-Selection sparse `
|
||||
-NonSparseParameters @("JavaTestVersion")
|
||||
```
|
||||
|
||||
Given a matrix like below with `JavaTestVersion` marked as a non-sparse parameter:
|
||||
|
||||
```
|
||||
{
|
||||
"matrix": {
|
||||
"Agent": {
|
||||
"windows-2022": { "OSVmImage": "MMS2022", "Pool": "azsdk-pool-mms-win-2022-general" },
|
||||
"ubuntu-1804": { "OSVmImage": "MMSUbuntu18.04", "Pool": "azsdk-pool-mms-ubuntu-1804-general" },
|
||||
"macos-11": { "OSVmImage": "macos-11", "Pool": "Azure Pipelines" }
|
||||
},
|
||||
"JavaTestVersion": [ "1.8", "1.11" ],
|
||||
"AZURE_TEST_HTTP_CLIENTS": "netty",
|
||||
"ArmTemplateParameters": [ "@{endpointType='storage'}", "@{endpointType='cosmos'}" ]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
A matrix with 6 entries will be generated: A sparse matrix of Agent, AZURE_TEST_HTTP_CLIENTS and ArmTemplateParameters
|
||||
(3 total entries) will be multipled by the two `JavaTestVersion` parameters `1.8` and `1.11`.
|
||||
|
||||
NOTE: NonSparseParameters are also applied when generating an imported matrix.
|
||||
|
||||
#### Under the hood
|
||||
|
||||
The script generates an N-dimensional matrix with dimensions equal to the parameter array lengths. For example,
|
||||
the below config would generate a 2x2x1x1x1 matrix (five-dimensional):
|
||||
|
||||
```
|
||||
"matrix": {
|
||||
"framework": [ "net461", "netcoreapp2.1" ],
|
||||
"additionalTestArguments": [ "", "/p:SuperTest=true" ]
|
||||
"pool": [ "ubuntu-18.04" ],
|
||||
"container": [ "ubuntu-18.04" ],
|
||||
"testMode": [ "Record" ]
|
||||
}
|
||||
```
|
||||
|
||||
The matrix is stored as a one-dimensional array, with a row-major indexing scheme (e.g. `(2, 1, 0, 1, 0)`).
|
||||
|
||||
## Testing
|
||||
|
||||
The matrix functions can be tested using [pester](https://pester.dev/). The test command must be run from within the tests directory.
|
||||
|
||||
```
|
||||
$ cd tests
|
||||
$ Invoke-Pester
|
||||
|
||||
Starting discovery in 3 files.
|
||||
Discovery finished in 75ms.
|
||||
[+] /home/ben/sdk/azure-sdk-tools/eng/common/scripts/job-matrix/tests/job-matrix-functions.filter.tests.ps1 750ms (309ms|428ms)
|
||||
[+] /home/ben/sdk/azure-sdk-tools/eng/common/scripts/job-matrix/tests/job-matrix-functions.modification.tests.ps1 867ms (250ms|608ms)
|
||||
[+] /home/ben/sdk/azure-sdk-tools/eng/common/scripts/job-matrix/tests/job-matrix-functions.tests.ps1 2.71s (725ms|1.93s)
|
||||
Tests completed in 4.33s
|
||||
Tests Passed: 141, Failed: 0, Skipped: 4 NotRun: 0
|
||||
```
|
||||
`azure-sdk-tools/doc/common/matrix_generator.md`
|
||||
|
||||
@ -1,42 +0,0 @@
|
||||
parameters:
|
||||
- name: CloudConfig
|
||||
type: object
|
||||
- name: Matrix
|
||||
type: string
|
||||
- name: DependsOn
|
||||
type: string
|
||||
default: ''
|
||||
- name: UsePlatformContainer
|
||||
type: boolean
|
||||
default: false
|
||||
|
||||
jobs:
|
||||
- job:
|
||||
dependsOn: ${{ parameters.DependsOn }}
|
||||
condition: ne(${{ parameters.Matrix }}, '{}')
|
||||
strategy:
|
||||
matrix: $[ ${{ parameters.Matrix }} ]
|
||||
|
||||
pool:
|
||||
name: $(Pool)
|
||||
vmImage: $(OSVmImage)
|
||||
|
||||
${{ if eq(parameters.UsePlatformContainer, 'true') }}:
|
||||
container: $[ variables['Container'] ]
|
||||
|
||||
steps:
|
||||
- pwsh: |
|
||||
Write-Output "MATRIX JOB PARAMETERS"
|
||||
Write-Output $(Agent.JobName)
|
||||
Write-Output "-----------------"
|
||||
Write-Output $(OSVmImage)
|
||||
Write-Output $(TestTargetFramework)
|
||||
try {
|
||||
Write-Output $(additionalTestArguments)
|
||||
} catch {}
|
||||
displayName: Print matrix job variables
|
||||
|
||||
- pwsh: |
|
||||
Write-Output "Success"
|
||||
displayName: linux OS condition example
|
||||
condition: and(succeededOrFailed(), contains(variables['OSVmImage'], 'Ubuntu'))
|
||||
@ -1,33 +0,0 @@
|
||||
trigger: none
|
||||
|
||||
variables:
|
||||
- template: /eng/pipelines/templates/variables/globals.yml
|
||||
|
||||
jobs:
|
||||
- template: /eng/common/pipelines/templates/jobs/archetype-sdk-tests-generate.yml
|
||||
parameters:
|
||||
JobTemplatePath: /eng/common/scripts/job-matrix/samples/matrix-job-sample.yml
|
||||
AdditionalParameters: {}
|
||||
Pool: Azure Pipelines
|
||||
OsVmImage: ubuntu-18.04
|
||||
CloudConfig:
|
||||
SubscriptionConfiguration: $(sub-config-azure-cloud-test-resources)
|
||||
Location: eastus2
|
||||
Cloud: Public
|
||||
MatrixFilters:
|
||||
# Exclusion example
|
||||
- OSVmImage=^(?!macOS).*
|
||||
MatrixReplace:
|
||||
- OsVmImage=(ubuntu).*/$1-20.04
|
||||
- .*Framework.*=net5.0/net5.1
|
||||
MatrixConfigs:
|
||||
- Name: base_product_matrix
|
||||
Path: eng/common/scripts/job-matrix/samples/matrix.json
|
||||
Selection: all
|
||||
GenerateVMJobs: true
|
||||
- Name: sparse_product_matrix
|
||||
Path: eng/common/scripts/job-matrix/samples/matrix.json
|
||||
Selection: sparse
|
||||
NonSparseParameters:
|
||||
- framework
|
||||
GenerateVMJobs: true
|
||||
@ -1,28 +0,0 @@
|
||||
{
|
||||
"displayNames": {
|
||||
"/p:UseProjectReferenceToAzureClients=true": "UseProjectRef"
|
||||
},
|
||||
"matrix": {
|
||||
"Agent": {
|
||||
"ubuntu": { "OSVmImage": "ubuntu-18.04", "Pool": "Azure Pipelines" },
|
||||
"windows": { "OSVmImage": "windows-2022", "Pool": "Azure Pipelines" },
|
||||
"macOS": { "OSVmImage": "macos-11", "Pool": "Azure Pipelines" }
|
||||
},
|
||||
"TestTargetFramework": [ "netcoreapp2.1", "net461", "net5.0" ]
|
||||
},
|
||||
"include": [
|
||||
{
|
||||
"Agent": {
|
||||
"windows": { "OSVmImage": "windows-2022", "Pool": "Azure Pipelines" }
|
||||
},
|
||||
"TestTargetFramework": [ "net461", "net5.0" ],
|
||||
"AdditionalTestArguments": "/p:UseProjectReferenceToAzureClients=true"
|
||||
}
|
||||
],
|
||||
"exclude": [
|
||||
{
|
||||
"OSVmImage": "MMS2022",
|
||||
"framework": "netcoreapp2.1"
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -1,84 +0,0 @@
|
||||
Import-Module Pester
|
||||
|
||||
BeforeAll {
|
||||
. $PSScriptRoot/../job-matrix-functions.ps1
|
||||
|
||||
$matrixConfig = @"
|
||||
{
|
||||
"matrix": {
|
||||
"operatingSystem": [ "windows-2022", "ubuntu-18.04", "macos-11" ],
|
||||
"framework": [ "net461", "netcoreapp2.1" ],
|
||||
"additionalArguments": [ "", "mode=test" ]
|
||||
}
|
||||
}
|
||||
"@
|
||||
$config = GetMatrixConfigFromJson $matrixConfig
|
||||
}
|
||||
|
||||
Describe "Matrix Filter" -Tag "filter" {
|
||||
It "Should filter by matrix display name" -TestCases @(
|
||||
@{ regex = "windows.*"; expectedFirst = "windows2022_net461"; length = 4 }
|
||||
@{ regex = "windows2022_netcoreapp21_modetest"; expectedFirst = "windows2022_netcoreapp21_modetest"; length = 1 }
|
||||
@{ regex = ".*ubuntu.*"; expectedFirst = "ubuntu1804_net461"; length = 4 }
|
||||
) {
|
||||
[array]$matrix = GenerateMatrix $config "all" $regex
|
||||
$matrix.Length | Should -Be $length
|
||||
$matrix[0].Name | Should -Be $expectedFirst
|
||||
}
|
||||
|
||||
It "Should handle no display name filter matches" {
|
||||
$matrix = GenerateMatrix $config "all"
|
||||
[array]$filtered = FilterMatrixDisplayName $matrix "doesnotexist"
|
||||
$filtered | Should -BeNullOrEmpty
|
||||
}
|
||||
|
||||
It "Should filter by matrix key/value" -TestCases @(
|
||||
@{ filterString = "operatingSystem=windows.*"; expectedFirst = "windows2022_net461"; length = 4 }
|
||||
@{ filterString = "operatingSystem=windows-2022"; expectedFirst = "windows2022_net461"; length = 4 }
|
||||
@{ filterString = "framework=.*"; expectedFirst = "windows2022_net461"; length = 12 }
|
||||
@{ filterString = "additionalArguments=mode=test"; expectedFirst = "windows2022_net461_modetest"; length = 6 }
|
||||
@{ filterString = "additionalArguments=^$"; expectedFirst = "windows2022_net461"; length = 6 }
|
||||
) {
|
||||
[array]$matrix = GenerateMatrix $config "all" -filters @($filterString)
|
||||
$matrix.Length | Should -Be $length
|
||||
$matrix[0].Name | Should -Be $expectedFirst
|
||||
}
|
||||
|
||||
It "Should filter by optional matrix key/value" -TestCases @(
|
||||
@{ filterString = "operatingSystem=^$|windows.*"; expectedFirst = "windows2022_net461"; length = 4 }
|
||||
@{ filterString = "doesnotexist=^$|.*"; expectedFirst = "windows2022_net461"; length = 12 }
|
||||
) {
|
||||
[array]$matrix = GenerateMatrix $config "all" -filters @($filterString)
|
||||
$matrix.Length | Should -Be $length
|
||||
$matrix[0].Name | Should -Be $expectedFirst
|
||||
}
|
||||
|
||||
It "Should handle multiple matrix key/value filters " {
|
||||
[array]$matrix = GenerateMatrix $config "all" -filters "operatingSystem=windows.*","framework=.*","additionalArguments=mode=test"
|
||||
$matrix.Length | Should -Be 2
|
||||
$matrix[0].Name | Should -Be "windows2022_net461_modetest"
|
||||
}
|
||||
|
||||
It "Should handle no matrix key/value filter matches" {
|
||||
[array]$matrix = GenerateMatrix $config "all" -filters @("doesnot=exist")
|
||||
$matrix | Should -BeNullOrEmpty
|
||||
}
|
||||
|
||||
It "Should handle invalid matrix key/value filter syntax" {
|
||||
{ GenerateMatrix $config "all" -filters @("invalid") } | Should -Throw
|
||||
{ GenerateMatrix $config "all" -filters @("emptyvalue=") } | Should -Throw
|
||||
{ GenerateMatrix $config "all" -filters @("=emptykey") } | Should -Throw
|
||||
{ GenerateMatrix $config "all" -filters @("=") } | Should -Throw
|
||||
}
|
||||
|
||||
It "Should filter by key exclude" {
|
||||
[array]$matrix = GenerateMatrix $config "all" -filters @("operatingSystem=^$")
|
||||
$matrix | Should -BeNullOrEmpty
|
||||
|
||||
[array]$matrix = GenerateMatrix $config "all"
|
||||
$matrix.Length | Should -Be 12
|
||||
$matrix += @{ Name = "excludeme"; Parameters = [Ordered]@{ "foo" = 1 } }
|
||||
[array]$matrix = FilterMatrix $matrix @("foo=^$")
|
||||
$matrix.Length | Should -Be 12
|
||||
}
|
||||
}
|
||||
@ -1,572 +0,0 @@
|
||||
Import-Module Pester
|
||||
|
||||
|
||||
BeforeAll {
|
||||
. $PSScriptRoot/../job-matrix-functions.ps1
|
||||
|
||||
function CompareMatrices([Array]$matrix, [Array]$expected) {
|
||||
$matrix.Length | Should -Be $expected.Length
|
||||
|
||||
for ($i = 0; $i -lt $matrix.Length; $i++) {
|
||||
foreach ($entry in $matrix[$i]) {
|
||||
$entry.name | Should -Be $expected[$i].name
|
||||
foreach ($param in $entry.parameters.GetEnumerator()) {
|
||||
$param.Value | Should -Be $expected[$i].parameters[$param.Name]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Describe "Platform Matrix nonSparse" -Tag "nonsparse" {
|
||||
BeforeEach {
|
||||
$matrixJson = @'
|
||||
{
|
||||
"matrix": {
|
||||
"testField1": [ 1, 2 ],
|
||||
"testField2": [ 1, 2, 3 ],
|
||||
"testField3": [ 1, 2, 3, 4 ],
|
||||
}
|
||||
}
|
||||
'@
|
||||
$config = GetMatrixConfigFromJson $matrixJson
|
||||
}
|
||||
|
||||
It "Should process nonSparse parameters" {
|
||||
$parameters, $nonSparse = ProcessNonSparseParameters $config.matrixParameters "testField1","testField3"
|
||||
|
||||
$parameters.Count | Should -Be 1
|
||||
$parameters[0].Name | Should -Be "testField2"
|
||||
$parameters[0].Value | Should -Be 1,2,3
|
||||
|
||||
$nonSparse.Count | Should -Be 2
|
||||
$nonSparse[0].Name | Should -Be "testField1"
|
||||
$nonSparse[0].Value | Should -Be 1,2
|
||||
$nonSparse[1].Name | Should -Be "testField3"
|
||||
$nonSparse[1].Value | Should -Be 1,2,3,4
|
||||
|
||||
$parameters, $nonSparse = ProcessNonSparseParameters $config.matrixParameters "testField3"
|
||||
$parameters.Count | Should -Be 2
|
||||
($parameters).Name -match "testField3" | Should -Be $null
|
||||
|
||||
$nonSparse.Count | Should -Be 1
|
||||
$nonSparse[0].Name | Should -Be "testField3"
|
||||
$nonSparse[0].Value | Should -Be 1,2,3,4
|
||||
}
|
||||
|
||||
It "Should ignore nonSparse with all selection" {
|
||||
$matrix = GenerateMatrix $config "all" -nonSparseParameters "testField3"
|
||||
$matrix.Length | Should -Be 24
|
||||
}
|
||||
|
||||
It "Should combine sparse matrix with nonSparse parameters" {
|
||||
$matrix = GenerateMatrix $config "sparse" -nonSparseParameters "testField3"
|
||||
$matrix.Length | Should -Be 12
|
||||
}
|
||||
|
||||
It "Should combine with multiple nonSparse fields" {
|
||||
$matrixJson = @'
|
||||
{
|
||||
"matrix": {
|
||||
"testField1": [ 1, 2 ],
|
||||
"testField2": [ 1, 2 ],
|
||||
"testField3": [ 31, 32 ],
|
||||
"testField4": [ 41, 42 ]
|
||||
}
|
||||
}
|
||||
'@
|
||||
$config = GetMatrixConfigFromJson $matrixJson
|
||||
|
||||
$matrix = GenerateMatrix $config "all" -nonSparseParameters "testField3","testField4"
|
||||
$matrix.Length | Should -Be 16
|
||||
|
||||
$matrix = GenerateMatrix $config "sparse" -nonSparseParameters "testField3","testField4"
|
||||
$matrix.Length | Should -Be 8
|
||||
}
|
||||
|
||||
It "Should apply nonSparseParameters to an imported matrix" {
|
||||
$matrixJson = @'
|
||||
{
|
||||
"matrix": {
|
||||
"$IMPORT": "./eng/common/scripts/job-matrix/tests/test-import-matrix.json",
|
||||
"TestField1": "test1"
|
||||
},
|
||||
"exclude": [ { "Baz": "importedBaz" } ]
|
||||
}
|
||||
'@
|
||||
|
||||
$expectedMatrix = @'
|
||||
[
|
||||
{
|
||||
"parameters": { "TestField1": "test1", "Foo": "foo1", "Bar": "bar1" },
|
||||
"name": "test1_foo1_bar1"
|
||||
},
|
||||
{
|
||||
"parameters": { "TestField1": "test1", "Foo": "foo1", "Bar": "bar2" },
|
||||
"name": "test1_foo1_bar2"
|
||||
},
|
||||
{
|
||||
"parameters": { "TestField1": "test1", "Foo": "foo2", "Bar": "bar1" },
|
||||
"name": "test1_foo2_bar1"
|
||||
},
|
||||
{
|
||||
"parameters": { "TestField1": "test1", "Foo": "foo2", "Bar": "bar2" },
|
||||
"name": "test1_foo2_bar2"
|
||||
}
|
||||
]
|
||||
'@
|
||||
|
||||
$importConfig = GetMatrixConfigFromJson $matrixJson
|
||||
$matrix = GenerateMatrix $importConfig "sparse" -nonSparseParameters "Foo"
|
||||
$expected = $expectedMatrix | ConvertFrom-Json -AsHashtable
|
||||
|
||||
$matrix.Length | Should -Be 4
|
||||
CompareMatrices $matrix $expected
|
||||
}
|
||||
}
|
||||
|
||||
Describe "Platform Matrix Import" -Tag "import" {
|
||||
It "Should generate a sparse matrix where the entire base matrix is imported" {
|
||||
$matrixJson = @'
|
||||
{
|
||||
"matrix": {
|
||||
"$IMPORT": "./eng/common/scripts/job-matrix/tests/test-import-matrix.json"
|
||||
},
|
||||
"include": [
|
||||
{
|
||||
"fooinclude": "fooinclude"
|
||||
}
|
||||
]
|
||||
}
|
||||
'@
|
||||
|
||||
$expectedMatrix = @'
|
||||
[
|
||||
{
|
||||
"parameters": { "Foo": "foo1", "Bar": "bar1" },
|
||||
"name": "foo1_bar1"
|
||||
},
|
||||
{
|
||||
"parameters": { "Foo": "foo2", "Bar": "bar2" },
|
||||
"name": "foo2_bar2"
|
||||
},
|
||||
{
|
||||
"parameters": { "Baz": "importedBaz" },
|
||||
"name": "importedBazName"
|
||||
},
|
||||
{
|
||||
"parameters": { "fooinclude": "fooinclude" },
|
||||
"name": "fooinclude"
|
||||
},
|
||||
]
|
||||
'@
|
||||
|
||||
$importConfig = GetMatrixConfigFromJson $matrixJson
|
||||
$matrix = GenerateMatrix $importConfig "sparse"
|
||||
$expected = $expectedMatrix | ConvertFrom-Json -AsHashtable
|
||||
|
||||
$matrix.Length | Should -Be 4
|
||||
CompareMatrices $matrix $expected
|
||||
}
|
||||
|
||||
It "Should import a matrix and combine with length=1 vectors" {
|
||||
$matrixJson = @'
|
||||
{
|
||||
"matrix": {
|
||||
"$IMPORT": "./eng/common/scripts/job-matrix/tests/test-import-matrix.json",
|
||||
"TestField1": "test1",
|
||||
"TestField2": "test2"
|
||||
},
|
||||
"exclude": [ { "Baz": "importedBaz" } ]
|
||||
}
|
||||
'@
|
||||
|
||||
$expectedMatrix = @'
|
||||
[
|
||||
{
|
||||
"parameters": { "TestField1": "test1", "TestField2": "test2", "Foo": "foo1", "Bar": "bar1" },
|
||||
"name": "test1_test2_foo1_bar1"
|
||||
},
|
||||
{
|
||||
"parameters": { "TestField1": "test1", "TestField2": "test2", "Foo": "foo2", "Bar": "bar2" },
|
||||
"name": "test1_test2_foo2_bar2"
|
||||
}
|
||||
]
|
||||
'@
|
||||
|
||||
$importConfig = GetMatrixConfigFromJson $matrixJson
|
||||
$matrix = GenerateMatrix $importConfig "sparse"
|
||||
$expected = $expectedMatrix | ConvertFrom-Json -AsHashtable
|
||||
|
||||
$matrix.Length | Should -Be 2
|
||||
CompareMatrices $matrix $expected
|
||||
}
|
||||
|
||||
It "Should generate a matrix with nonSparseParameters and an imported sparse matrix" {
|
||||
$matrixJson = @'
|
||||
{
|
||||
"matrix": {
|
||||
"$IMPORT": "./eng/common/scripts/job-matrix/tests/test-import-matrix.json",
|
||||
"testField": [ "test1", "test2" ]
|
||||
}
|
||||
}
|
||||
'@
|
||||
$importConfig = GetMatrixConfigFromJson $matrixJson
|
||||
$matrix = GenerateMatrix $importConfig "sparse" -nonSparseParameters "testField"
|
||||
|
||||
$matrix.Length | Should -Be 6
|
||||
|
||||
$matrix[0].name | Should -Be test1_foo1_bar1
|
||||
$matrix[0].parameters.testField | Should -Be "test1"
|
||||
$matrix[0].parameters.Foo | Should -Be "foo1"
|
||||
$matrix[2].name | Should -Be test1_importedBazName
|
||||
$matrix[2].parameters.testField | Should -Be "test1"
|
||||
$matrix[2].parameters.Baz | Should -Be "importedBaz"
|
||||
$matrix[4].name | Should -Be test2_foo2_bar2
|
||||
$matrix[4].parameters.testField | Should -Be "test2"
|
||||
$matrix[4].parameters.Foo | Should -Be "foo2"
|
||||
}
|
||||
|
||||
It "Should source imported display name lookups" {
|
||||
$matrixJson = @'
|
||||
{
|
||||
"displayNames": {
|
||||
"test1": "test1DisplayName",
|
||||
"importedBaz": "importedBazNameOverride"
|
||||
},
|
||||
"matrix": {
|
||||
"$IMPORT": "./eng/common/scripts/job-matrix/tests/test-import-matrix.json",
|
||||
"testField": [ "test1", "test2" ]
|
||||
}
|
||||
}
|
||||
'@
|
||||
$importConfig = GetMatrixConfigFromJson $matrixJson
|
||||
$matrix = GenerateMatrix $importConfig "sparse" -nonSparseParameters "testField"
|
||||
|
||||
$matrix[0].name | Should -Be test1DisplayName_foo1_bar1
|
||||
$matrix[2].name | Should -Be test1DisplayName_importedBazNameOverride
|
||||
}
|
||||
|
||||
It "Should generate a sparse matrix with an imported sparse matrix" {
|
||||
$matrixJson = @'
|
||||
{
|
||||
"matrix": {
|
||||
"$IMPORT": "./eng/common/scripts/job-matrix/tests/test-import-matrix.json",
|
||||
"testField1": [ "test11", "test12" ],
|
||||
"testField2": [ "test21", "test22" ]
|
||||
}
|
||||
}
|
||||
'@
|
||||
|
||||
$expectedMatrix = @'
|
||||
[
|
||||
{
|
||||
"parameters": { "testField1": "test11", "testField2": "test21", "Foo": "foo1", "Bar": "bar1" },
|
||||
"name": "test11_test21_foo1_bar1"
|
||||
},
|
||||
{
|
||||
"parameters": { "testField1": "test11", "testField2": "test21", "Foo": "foo2", "Bar": "bar2" },
|
||||
"name": "test11_test21_foo2_bar2"
|
||||
},
|
||||
{
|
||||
"parameters": { "testField1": "test11", "testField2": "test21", "Baz": "importedBaz" },
|
||||
"name": "test11_test21_importedBazName"
|
||||
},
|
||||
{
|
||||
"parameters": { "testField1": "test12", "testField2": "test22", "Foo": "foo1", "Bar": "bar1" },
|
||||
"name": "test12_test22_foo1_bar1"
|
||||
},
|
||||
{
|
||||
"parameters": { "testField1": "test12", "testField2": "test22", "Foo": "foo2", "Bar": "bar2" },
|
||||
"name": "test12_test22_foo2_bar2"
|
||||
},
|
||||
{
|
||||
"parameters": { "testField1": "test12", "testField2": "test22", "Baz": "importedBaz" },
|
||||
"name": "test12_test22_importedBazName"
|
||||
}
|
||||
]
|
||||
'@
|
||||
|
||||
$importConfig = GetMatrixConfigFromJson $matrixJson
|
||||
$matrix = GenerateMatrix $importConfig "sparse"
|
||||
$expected = $expectedMatrix | ConvertFrom-Json -AsHashtable
|
||||
|
||||
$matrix.Length | Should -Be 6
|
||||
CompareMatrices $matrix $expected
|
||||
}
|
||||
|
||||
It "Should import a sparse matrix with import, include, and exclude" {
|
||||
$matrixJson = @'
|
||||
{
|
||||
"matrix": {
|
||||
"$IMPORT": "./eng/common/scripts/job-matrix/tests/test-import-matrix.json",
|
||||
"testField": [ "test1", "test2", "test3" ],
|
||||
},
|
||||
"include": [
|
||||
{
|
||||
"testImportIncludeName": [ "testInclude1", "testInclude2" ]
|
||||
}
|
||||
],
|
||||
"exclude": [
|
||||
{
|
||||
"testField": "test1"
|
||||
},
|
||||
{
|
||||
"testField": "test3",
|
||||
"Baz": "importedBaz"
|
||||
}
|
||||
]
|
||||
}
|
||||
'@
|
||||
|
||||
$expectedMatrix = @'
|
||||
[
|
||||
{
|
||||
"parameters": { "testField": "test2", "Foo": "foo1", "Bar": "bar1" },
|
||||
"name": "test2_foo1_bar1"
|
||||
},
|
||||
{
|
||||
"parameters": { "testField": "test2", "Foo": "foo2", "Bar": "bar2" },
|
||||
"name": "test2_foo2_bar2"
|
||||
},
|
||||
{
|
||||
"parameters": { "testField": "test2", "Baz": "importedBaz" },
|
||||
"name": "test2_importedBazName"
|
||||
},
|
||||
{
|
||||
"parameters": { "testField": "test3", "Foo": "foo1", "Bar": "bar1" },
|
||||
"name": "test3_foo1_bar1"
|
||||
},
|
||||
{
|
||||
"parameters": { "testField": "test3", "Foo": "foo2", "Bar": "bar2" },
|
||||
"name": "test3_foo2_bar2"
|
||||
},
|
||||
{
|
||||
"parameters": { "testImportIncludeName": "testInclude1" },
|
||||
"name": "testInclude1"
|
||||
},
|
||||
{
|
||||
"parameters": { "testImportIncludeName": "testInclude2" },
|
||||
"name": "testInclude2"
|
||||
}
|
||||
]
|
||||
'@
|
||||
|
||||
$importConfig = GetMatrixConfigFromJson $matrixJson
|
||||
$matrix = GenerateMatrix $importConfig "sparse"
|
||||
$expected = $expectedMatrix | ConvertFrom-Json -AsHashtable
|
||||
|
||||
$matrix.Length | Should -Be 7
|
||||
CompareMatrices $matrix $expected
|
||||
}
|
||||
|
||||
It "Should not combine matrices with duplicate keys" {
|
||||
$matrixJson = @'
|
||||
{
|
||||
"matrix": {
|
||||
"$IMPORT": "./eng/common/scripts/job-matrix/tests/test-import-matrix.json",
|
||||
"Foo": [ "fooOverride1", "fooOverride2" ],
|
||||
}
|
||||
}
|
||||
'@
|
||||
|
||||
$importConfig = GetMatrixConfigFromJson $matrixJson
|
||||
{ GenerateMatrix $importConfig "sparse" } | Should -Throw
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
Describe "Platform Matrix Replace" -Tag "replace" {
|
||||
It "Should parse replacement syntax" -TestCases @(
|
||||
@{ query = 'foo=bar/baz'; key = '^foo$'; value = '^bar$'; replace = 'baz' },
|
||||
@{ query = 'foo=\/p:bar/\/p:baz'; key = '^foo$'; value = '^\/p:bar$'; replace = '/p:baz' },
|
||||
@{ query = 'f\=o\/o=\/p:b\=ar/\/p:b\=az'; key = '^f\=o\/o$'; value = '^\/p:b\=ar$'; replace = '/p:b=az' },
|
||||
@{ query = 'foo=bar/'; key = '^foo$'; value = '^bar$'; replace = '' },
|
||||
@{ query = 'foo=/baz'; key = '^foo$'; value = '^$'; replace = 'baz' }
|
||||
) {
|
||||
$parsed = ParseReplacement $query
|
||||
$parsed.key | Should -Be $key
|
||||
$parsed.value | Should -Be $value
|
||||
$parsed.replace | Should -Be $replace
|
||||
}
|
||||
|
||||
It "Should fail for invalid replacement syntax" -TestCases @(
|
||||
@{ query = '' },
|
||||
@{ query = 'asdf' },
|
||||
@{ query = 'asdf=foo/bar/baz' },
|
||||
@{ query = 'asdf=foo=bar/baz' },
|
||||
@{ query = 'asdf=foo' }
|
||||
) {
|
||||
{ $parsed = ParseReplacement $query } | Should -Throw
|
||||
{ $parsed = ParseReplacement $query } | Should -Throw
|
||||
{ $parsed = ParseReplacement $query } | Should -Throw
|
||||
{ $parsed = ParseReplacement $query } | Should -Throw
|
||||
{ $parsed = ParseReplacement $query } | Should -Throw
|
||||
}
|
||||
|
||||
It "Should replace values in a matrix" {
|
||||
$matrixJson = @'
|
||||
{
|
||||
"matrix": {
|
||||
"Foo": [ "foo1", "foo2" ],
|
||||
"Bar": [ "bar1", "bar2" ]
|
||||
},
|
||||
"include": [ { "Baz": "baz1" } ]
|
||||
}
|
||||
'@
|
||||
|
||||
$expectedMatrix = @'
|
||||
[
|
||||
{
|
||||
"parameters": { "Foo": "foo1Replaced", "Bar": "bar1" },
|
||||
"name": "foo1Replaced_bar1"
|
||||
},
|
||||
{
|
||||
"parameters": { "Foo": "fooDefaultReplaced", "Bar": "bar2" },
|
||||
"name": "fooDefaultReplaced_bar2"
|
||||
},
|
||||
{
|
||||
"parameters": { "Baz": "bazReplaced" },
|
||||
"name": "bazReplaced"
|
||||
}
|
||||
]
|
||||
'@
|
||||
|
||||
$replace = @(
|
||||
"Foo=foo1/foo1Replaced",
|
||||
"Foo=foo.*/fooDefaultReplaced",
|
||||
".*=B.z\d/bazReplaced"
|
||||
)
|
||||
|
||||
$importConfig = GetMatrixConfigFromJson $matrixJson
|
||||
$matrix = GenerateMatrix -config $importConfig -selectFromMatrixType "sparse" -replace $replace
|
||||
$expected = $expectedMatrix | ConvertFrom-Json -AsHashtable
|
||||
|
||||
$matrix.Length | Should -Be 3
|
||||
CompareMatrices $matrix $expected
|
||||
}
|
||||
|
||||
It "Should replace values in a matrix with import and nonSparseParameters" {
|
||||
$matrixJson = @'
|
||||
{
|
||||
"matrix": {
|
||||
"$IMPORT": "./eng/common/scripts/job-matrix/tests/test-import-matrix.json",
|
||||
"testField": [ "test1", "test2" ]
|
||||
}
|
||||
}
|
||||
'@
|
||||
$importConfig = GetMatrixConfigFromJson $matrixJson
|
||||
$matrix = GenerateMatrix $importConfig "sparse" -nonSparseParameters "testField" -replace @("testField=test1/testReplaced", "Baz=.*/bazReplaced")
|
||||
|
||||
$matrix.Length | Should -Be 6
|
||||
|
||||
$matrix[0].name | Should -Be testReplaced_foo1_bar1
|
||||
$matrix[0].parameters.testField | Should -Be "testReplaced"
|
||||
$matrix[0].parameters.Foo | Should -Be "foo1"
|
||||
$matrix[2].name | Should -Be testReplaced_bazReplaced
|
||||
$matrix[2].parameters.testField | Should -Be "testReplaced"
|
||||
$matrix[2].parameters.Baz | Should -Be "bazReplaced"
|
||||
$matrix[4].name | Should -Be test2_foo2_bar2
|
||||
$matrix[4].parameters.testField | Should -Be "test2"
|
||||
$matrix[4].parameters.Foo | Should -Be "foo2"
|
||||
}
|
||||
|
||||
It "Should replace values in groupings" {
|
||||
$matrixJson = @'
|
||||
{
|
||||
"matrix": {
|
||||
"Agent": {
|
||||
"ubuntu-1804": { "OSVmImage": "MMSUbuntu18.04", "Pool": "azsdk-pool-mms-ubuntu-1804-general" }
|
||||
},
|
||||
"JavaTestVersion": [ "1.8", "1.11" ]
|
||||
}
|
||||
}
|
||||
'@
|
||||
$importConfig = GetMatrixConfigFromJson $matrixJson
|
||||
$matrix = GenerateMatrix $importConfig "all" -replace @("JavaTestVersion=1.8/2.0", "Pool=.*ubuntu.*/custom-ubuntu-pool")
|
||||
|
||||
$matrix.Length | Should -Be 2
|
||||
# Replacements of inner values will preserve the grouping name
|
||||
$matrix[0].name | Should -Be "ubuntu1804_20"
|
||||
$matrix[0].parameters.JavaTestVersion | Should -Be "2.0"
|
||||
$matrix[0].parameters.Pool | Should -Be "custom-ubuntu-pool"
|
||||
$matrix[0].parameters.OSVmImage | Should -Be "MMSUbuntu18.04"
|
||||
|
||||
# Make sure non-literal keys still replace under the hood
|
||||
$matrix = GenerateMatrix $importConfig "all" -replace ".*=.*ubuntu.*/custom-ubuntu-pool"
|
||||
|
||||
$matrix.Length | Should -Be 2
|
||||
$matrix[0].name | Should -Be "ubuntu1804_18"
|
||||
$matrix[0].parameters.Pool | Should -Be "custom-ubuntu-pool"
|
||||
}
|
||||
|
||||
It "Should replace values and apply regex capture groups" {
|
||||
$matrixJson = @'
|
||||
{
|
||||
"matrix": {
|
||||
"Foo": [ "foo1", "foo2" ],
|
||||
"Bar": [ "bar1", "bar2" ]
|
||||
}
|
||||
}
|
||||
'@
|
||||
$importConfig = GetMatrixConfigFromJson $matrixJson
|
||||
$replace = 'Foo=(foo)1/$1ReplacedFoo1', 'B.*=(.*)2/$1ReplacedBar2'
|
||||
$matrix = GenerateMatrix $importConfig "sparse" -replace $replace
|
||||
|
||||
$matrix.Length | Should -Be 2
|
||||
$matrix[0].name | Should -Be "fooReplacedFoo1_bar1"
|
||||
$matrix[0].parameters.Foo | Should -Be "fooReplacedFoo1"
|
||||
|
||||
$matrix[1].name | Should -Be "foo2_barReplacedBar2"
|
||||
$matrix[1].parameters.Bar | Should -Be "barReplacedBar2"
|
||||
}
|
||||
|
||||
It "Should only fully match a string for replace" {
|
||||
$matrixJson = @'
|
||||
{
|
||||
"matrix": {
|
||||
"Foo": [ "foo1", "foo2" ],
|
||||
"Bar": "bar1"
|
||||
}
|
||||
}
|
||||
'@
|
||||
|
||||
$importConfig = GetMatrixConfigFromJson $matrixJson
|
||||
|
||||
$replace = @("Foo=foo/shouldNotReplaceFoo", "B=bar1/shouldNotReplaceBar")
|
||||
$matrix = GenerateMatrix -config $importConfig -selectFromMatrixType "sparse" -replace $replace
|
||||
|
||||
$matrix.Length | Should -Be 2
|
||||
$matrix[0].parameters.Foo | Should -Be "foo1"
|
||||
$matrix[0].parameters.Bar | Should -Be "bar1"
|
||||
$matrix[1].parameters.Foo | Should -Be "foo2"
|
||||
$matrix[1].parameters.Bar | Should -Be "bar1"
|
||||
}
|
||||
|
||||
It "Should parse replacement syntax and source imported display name lookups" {
|
||||
$matrixJson = @'
|
||||
{
|
||||
"displayNames": {
|
||||
"replaceme": ""
|
||||
},
|
||||
"matrix": {
|
||||
"$IMPORT": "./eng/common/scripts/job-matrix/tests/test-import-matrix.json",
|
||||
"replaceme": "replaceme"
|
||||
}
|
||||
}
|
||||
'@
|
||||
$importConfig = GetMatrixConfigFromJson $matrixJson
|
||||
$replace = 'Foo=(foo)1/$1ReplacedFoo1', 'B.*=(.*)2/$1ReplacedBar2'
|
||||
$matrix = GenerateMatrix $importConfig "sparse" -replace $replace
|
||||
|
||||
$matrix.Length | Should -Be 3
|
||||
$matrix[0].name | Should -Be "fooReplacedFoo1_bar1"
|
||||
$matrix[0].parameters.Foo | Should -Be "fooReplacedFoo1"
|
||||
$matrix[1].name | Should -Be "foo2_barReplacedBar2"
|
||||
$matrix[1].parameters.Bar | Should -Be "barReplacedBar2"
|
||||
$matrix[2].name | Should -Be "importedBazName"
|
||||
$matrix[2].parameters.Baz | Should -Be "importedBaz"
|
||||
$matrix[2].parameters.replaceme | Should -Be "replaceme"
|
||||
}
|
||||
|
||||
}
|
||||
@ -1,654 +0,0 @@
|
||||
Import-Module Pester
|
||||
|
||||
BeforeAll {
|
||||
. $PSScriptRoot/../job-matrix-functions.ps1
|
||||
|
||||
$matrixConfig = @"
|
||||
{
|
||||
"displayNames": {
|
||||
"--enableFoo": "withfoo"
|
||||
},
|
||||
"matrix": {
|
||||
"operatingSystem": [
|
||||
"windows-2022",
|
||||
"ubuntu-18.04",
|
||||
"macos-11"
|
||||
],
|
||||
"framework": [
|
||||
"net461",
|
||||
"netcoreapp2.1"
|
||||
],
|
||||
"additionalArguments": [
|
||||
"",
|
||||
"--enableFoo"
|
||||
]
|
||||
},
|
||||
"include": [
|
||||
{
|
||||
"operatingSystem": "windows-2022",
|
||||
"framework": ["net461", "netcoreapp2.1", "net50"],
|
||||
"additionalArguments": "--enableWindowsFoo"
|
||||
}
|
||||
],
|
||||
"exclude": [
|
||||
{
|
||||
"operatingSystem": "windows-2022",
|
||||
"framework": "net461"
|
||||
},
|
||||
{
|
||||
"operatingSystem": "macos-11",
|
||||
"framework": "netcoreapp2.1"
|
||||
},
|
||||
{
|
||||
"operatingSystem": ["macos-11", "ubuntu-18.04"],
|
||||
"additionalArguments": "--enableFoo"
|
||||
}
|
||||
]
|
||||
}
|
||||
"@
|
||||
}
|
||||
|
||||
Describe "Matrix-Lookup" -Tag "lookup" {
|
||||
It "Should navigate a 2d matrix: <row> <col>" -TestCases @(
|
||||
@{ row = 0; col = 0; expected = 1 },
|
||||
@{ row = 0; col = 1; expected = 2 },
|
||||
@{ row = 1; col = 0; expected = 3 },
|
||||
@{ row = 1; col = 1; expected = 4 }
|
||||
) {
|
||||
$dimensions = @(2, 2)
|
||||
$matrix = @(
|
||||
1, 2, 3, 4
|
||||
)
|
||||
GetNdMatrixElement @($row, $col) $matrix $dimensions | Should -Be $expected
|
||||
}
|
||||
|
||||
It "Should navigate a 3d matrix: <z> <row> <col>" -TestCases @(
|
||||
@{ z = 0; row = 0; col = 0; expected = 1 }
|
||||
@{ z = 0; row = 0; col = 1; expected = 2 }
|
||||
@{ z = 0; row = 1; col = 0; expected = 3 }
|
||||
@{ z = 0; row = 1; col = 1; expected = 4 }
|
||||
@{ z = 1; row = 0; col = 0; expected = 5 }
|
||||
@{ z = 1; row = 0; col = 1; expected = 6 }
|
||||
@{ z = 1; row = 1; col = 0; expected = 7 }
|
||||
@{ z = 1; row = 1; col = 1; expected = 8 }
|
||||
) {
|
||||
$dimensions = @(2, 2, 2)
|
||||
$matrix = @(
|
||||
1, 2, 3, 4, 5, 6, 7, 8
|
||||
)
|
||||
GetNdMatrixElement @($z, $row, $col) $matrix $dimensions | Should -Be $expected
|
||||
}
|
||||
|
||||
It "Should navigate a 4d matrix: <t> <z> <row> <col>" -TestCases @(
|
||||
@{ t = 0; z = 0; row = 0; col = 0; expected = 1 }
|
||||
@{ t = 0; z = 0; row = 0; col = 1; expected = 2 }
|
||||
@{ t = 0; z = 0; row = 1; col = 0; expected = 3 }
|
||||
@{ t = 0; z = 0; row = 1; col = 1; expected = 4 }
|
||||
@{ t = 0; z = 1; row = 0; col = 0; expected = 5 }
|
||||
@{ t = 0; z = 1; row = 0; col = 1; expected = 6 }
|
||||
@{ t = 0; z = 1; row = 1; col = 0; expected = 7 }
|
||||
@{ t = 0; z = 1; row = 1; col = 1; expected = 8 }
|
||||
@{ t = 1; z = 0; row = 0; col = 0; expected = 9 }
|
||||
@{ t = 1; z = 0; row = 0; col = 1; expected = 10 }
|
||||
@{ t = 1; z = 0; row = 1; col = 0; expected = 11 }
|
||||
@{ t = 1; z = 0; row = 1; col = 1; expected = 12 }
|
||||
@{ t = 1; z = 1; row = 0; col = 0; expected = 13 }
|
||||
@{ t = 1; z = 1; row = 0; col = 1; expected = 14 }
|
||||
@{ t = 1; z = 1; row = 1; col = 0; expected = 15 }
|
||||
@{ t = 1; z = 1; row = 1; col = 1; expected = 16 }
|
||||
) {
|
||||
$dimensions = @(2, 2, 2, 2)
|
||||
$matrix = @(
|
||||
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16
|
||||
)
|
||||
GetNdMatrixElement @($t, $z, $row, $col) $matrix $dimensions | Should -Be $expected
|
||||
}
|
||||
|
||||
It "Should navigate a 4d matrix: <t> <z> <row> <col>" -TestCases @(
|
||||
@{ t = 0; z = 0; row = 0; col = 0; expected = 1 }
|
||||
@{ t = 0; z = 0; row = 0; col = 1; expected = 2 }
|
||||
@{ t = 0; z = 0; row = 0; col = 2; expected = 3 }
|
||||
@{ t = 0; z = 0; row = 0; col = 3; expected = 4 }
|
||||
|
||||
@{ t = 0; z = 0; row = 1; col = 0; expected = 5 }
|
||||
@{ t = 0; z = 0; row = 1; col = 1; expected = 6 }
|
||||
@{ t = 0; z = 0; row = 1; col = 2; expected = 7 }
|
||||
@{ t = 0; z = 0; row = 1; col = 3; expected = 8 }
|
||||
|
||||
@{ t = 0; z = 1; row = 0; col = 0; expected = 9 }
|
||||
@{ t = 0; z = 1; row = 0; col = 1; expected = 10 }
|
||||
@{ t = 0; z = 1; row = 0; col = 2; expected = 11 }
|
||||
@{ t = 0; z = 1; row = 0; col = 3; expected = 12 }
|
||||
|
||||
@{ t = 0; z = 1; row = 1; col = 0; expected = 13 }
|
||||
@{ t = 0; z = 1; row = 1; col = 1; expected = 14 }
|
||||
@{ t = 0; z = 1; row = 1; col = 2; expected = 15 }
|
||||
@{ t = 0; z = 1; row = 1; col = 3; expected = 16 }
|
||||
) {
|
||||
$dimensions = @(1, 2, 2, 4)
|
||||
$matrix = @(
|
||||
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16
|
||||
)
|
||||
GetNdMatrixElement @($t, $z, $row, $col) $matrix $dimensions | Should -Be $expected
|
||||
}
|
||||
|
||||
# Skipping since by default wrapping behavior on indexing is disabled.
|
||||
# Keeping here in case we want to enable it.
|
||||
It -Skip "Should handle index wrapping: <row> <col>" -TestCases @(
|
||||
@{ row = 2; col = 2; expected = 1 }
|
||||
@{ row = 2; col = 3; expected = 2 }
|
||||
@{ row = 4; col = 4; expected = 1 }
|
||||
@{ row = 4; col = 5; expected = 2 }
|
||||
) {
|
||||
$dimensions = @(2, 2)
|
||||
$matrix = @(
|
||||
1, 2, 3, 4
|
||||
)
|
||||
GetNdMatrixElement @($row, $col) $matrix $dimensions | Should -Be $expected
|
||||
}
|
||||
}
|
||||
|
||||
Describe "Matrix-Reverse-Lookup" -Tag "lookup" {
|
||||
It "Should lookup a 2d matrix index: <index>" -TestCases @(
|
||||
@{ index = 0; expected = @(0,0) }
|
||||
@{ index = 1; expected = @(0,1) }
|
||||
@{ index = 2; expected = @(1,0) }
|
||||
@{ index = 3; expected = @(1,1) }
|
||||
) {
|
||||
$dimensions = @(2, 2)
|
||||
$matrix = @(1, 2, 3, 4)
|
||||
GetNdMatrixElement $expected $matrix $dimensions | Should -Be $matrix[$index]
|
||||
GetNdMatrixIndex $index $dimensions | Should -Be $expected
|
||||
}
|
||||
|
||||
It "Should lookup a 3d matrix index: <index>" -TestCases @(
|
||||
@{ index = 0; expected = @(0,0,0) }
|
||||
@{ index = 1; expected = @(0,0,1) }
|
||||
@{ index = 2; expected = @(0,1,0) }
|
||||
@{ index = 3; expected = @(0,1,1) }
|
||||
|
||||
@{ index = 4; expected = @(1,0,0) }
|
||||
@{ index = 5; expected = @(1,0,1) }
|
||||
@{ index = 6; expected = @(1,1,0) }
|
||||
@{ index = 7; expected = @(1,1,1) }
|
||||
) {
|
||||
$dimensions = @(2, 2, 2)
|
||||
$matrix = @(0, 1, 2, 3, 4, 5, 6, 7)
|
||||
GetNdMatrixElement $expected $matrix $dimensions | Should -Be $matrix[$index]
|
||||
GetNdMatrixIndex $index $dimensions | Should -Be $expected
|
||||
}
|
||||
|
||||
It "Should lookup a 3d matrix index: <index>" -TestCases @(
|
||||
@{ index = 0; expected = @(0,0,0) }
|
||||
@{ index = 1; expected = @(0,0,1) }
|
||||
@{ index = 2; expected = @(0,0,2) }
|
||||
|
||||
@{ index = 3; expected = @(0,1,0) }
|
||||
@{ index = 4; expected = @(0,1,1) }
|
||||
@{ index = 5; expected = @(0,1,2) }
|
||||
|
||||
@{ index = 6; expected = @(1,0,0) }
|
||||
@{ index = 7; expected = @(1,0,1) }
|
||||
@{ index = 8; expected = @(1,0,2) }
|
||||
|
||||
@{ index = 9; expected = @(1,1,0) }
|
||||
@{ index = 10; expected = @(1,1,1) }
|
||||
@{ index = 11; expected = @(1,1,2) }
|
||||
) {
|
||||
$dimensions = @(2, 2, 3)
|
||||
$matrix = @(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
|
||||
GetNdMatrixElement $expected $matrix $dimensions | Should -Be $matrix[$index]
|
||||
GetNdMatrixIndex $index $dimensions | Should -Be $expected
|
||||
}
|
||||
|
||||
It "Should lookup a 3d matrix index: <index>" -TestCases @(
|
||||
@{ index = 0; expected = @(0,0,0) }
|
||||
@{ index = 1; expected = @(0,0,1) }
|
||||
@{ index = 2; expected = @(0,1,0) }
|
||||
@{ index = 3; expected = @(0,1,1) }
|
||||
|
||||
@{ index = 4; expected = @(1,0,0) }
|
||||
@{ index = 5; expected = @(1,0,1) }
|
||||
@{ index = 6; expected = @(1,1,0) }
|
||||
@{ index = 7; expected = @(1,1,1) }
|
||||
|
||||
@{ index = 8; expected = @(2,0,0) }
|
||||
@{ index = 9; expected = @(2,0,1) }
|
||||
@{ index = 10; expected = @(2,1,0) }
|
||||
@{ index = 11; expected = @(2,1,1) }
|
||||
) {
|
||||
$dimensions = @(3, 2, 2)
|
||||
$matrix = @(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
|
||||
GetNdMatrixElement $expected $matrix $dimensions | Should -Be $matrix[$index]
|
||||
GetNdMatrixIndex $index $dimensions | Should -Be $expected
|
||||
}
|
||||
|
||||
It "Should lookup a 4d matrix index: <index>" -TestCases @(
|
||||
@{ index = 0; expected = @(0,0,0,0) }
|
||||
@{ index = 1; expected = @(0,0,0,1) }
|
||||
@{ index = 2; expected = @(0,0,0,2) }
|
||||
@{ index = 3; expected = @(0,0,0,3) }
|
||||
|
||||
@{ index = 4; expected = @(0,0,1,0) }
|
||||
@{ index = 5; expected = @(0,0,1,1) }
|
||||
@{ index = 6; expected = @(0,0,1,2) }
|
||||
@{ index = 7; expected = @(0,0,1,3) }
|
||||
|
||||
@{ index = 8; expected = @(0,1,0,0) }
|
||||
@{ index = 9; expected = @(0,1,0,1) }
|
||||
@{ index = 10; expected = @(0,1,0,2) }
|
||||
@{ index = 11; expected = @(0,1,0,3) }
|
||||
|
||||
@{ index = 12; expected = @(0,1,1,0) }
|
||||
@{ index = 13; expected = @(0,1,1,1) }
|
||||
@{ index = 14; expected = @(0,1,1,2) }
|
||||
@{ index = 15; expected = @(0,1,1,3) }
|
||||
) {
|
||||
$dimensions = @(1, 2, 2, 4)
|
||||
$matrix = @(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
|
||||
GetNdMatrixElement $expected $matrix $dimensions | Should -Be $matrix[$index]
|
||||
GetNdMatrixIndex $index $dimensions | Should -Be $expected
|
||||
}
|
||||
}
|
||||
|
||||
Describe 'Matrix-Set' -Tag "set" {
|
||||
It "Should set a matrix element" -TestCases @(
|
||||
@{ value = "set"; index = @(0,0,0,0); arrayIndex = 0 }
|
||||
@{ value = "ones"; index = @(0,1,1,1); arrayIndex = 13 }
|
||||
) {
|
||||
$dimensions = @(1, 2, 2, 4)
|
||||
$matrix = @(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
|
||||
|
||||
SetNdMatrixElement $value $index $matrix $dimensions
|
||||
$matrix[$arrayIndex] | Should -Be $value
|
||||
}
|
||||
}
|
||||
|
||||
Describe "Platform Matrix Generation" -Tag "generate" {
|
||||
BeforeEach {
|
||||
$matrixConfigForGenerate = @"
|
||||
{
|
||||
"displayNames": {
|
||||
"--enableFoo": "withfoo"
|
||||
},
|
||||
"matrix": {
|
||||
"operatingSystem": [
|
||||
"windows-2022",
|
||||
"ubuntu-18.04",
|
||||
"macos-11"
|
||||
],
|
||||
"framework": [
|
||||
"net461",
|
||||
"netcoreapp2.1"
|
||||
],
|
||||
"additionalArguments": [
|
||||
"",
|
||||
"--enableFoo"
|
||||
]
|
||||
},
|
||||
"include": [
|
||||
{
|
||||
"operatingSystem": "windows-2022",
|
||||
"framework": "net461",
|
||||
"additionalTestArguments": "/p:UseProjectReferenceToAzureClients=true"
|
||||
}
|
||||
],
|
||||
"exclude": [
|
||||
{
|
||||
"foo": "bar"
|
||||
},
|
||||
{
|
||||
"foo2": "bar2"
|
||||
}
|
||||
]
|
||||
}
|
||||
"@
|
||||
$generateConfig = GetMatrixConfigFromJson $matrixConfigForGenerate
|
||||
}
|
||||
|
||||
It "Should get matrix dimensions from Nd parameters" {
|
||||
GetMatrixDimensions $generateConfig.matrixParameters | Should -Be 3, 2, 2
|
||||
}
|
||||
|
||||
It "Should use name overrides from displayNames" {
|
||||
$dimensions = GetMatrixDimensions $generateConfig.matrixParameters
|
||||
$matrix = GenerateFullMatrix $generateConfig.matrixParameters $generateconfig.displayNamesLookup
|
||||
|
||||
$element = GetNdMatrixElement @(0, 0, 0) $matrix $dimensions
|
||||
$element.name | Should -Be "windows2022_net461"
|
||||
|
||||
$element = GetNdMatrixElement @(1, 1, 1) $matrix $dimensions
|
||||
$element.name | Should -Be "ubuntu1804_netcoreapp21_withFoo"
|
||||
|
||||
$element = GetNdMatrixElement @(2, 1, 1) $matrix $dimensions
|
||||
$element.name | Should -Be "macOS11_netcoreapp21_withFoo"
|
||||
}
|
||||
|
||||
It "Should initialize an N-dimensional matrix from all parameter permutations" {
|
||||
$dimensions = GetMatrixDimensions $generateConfig.matrixParameters
|
||||
$matrix = GenerateFullMatrix $generateConfig.matrixParameters $generateConfig.displayNamesLookup
|
||||
$matrix.Count | Should -Be 12
|
||||
|
||||
$element = $matrix[0].parameters
|
||||
$element.operatingSystem | Should -Be "windows-2022"
|
||||
$element.framework | Should -Be "net461"
|
||||
$element.additionalArguments | Should -Be ""
|
||||
|
||||
$element = GetNdMatrixElement @(1, 1, 1) $matrix $dimensions
|
||||
$element.parameters.operatingSystem | Should -Be "ubuntu-18.04"
|
||||
$element.parameters.framework | Should -Be "netcoreapp2.1"
|
||||
$element.parameters.additionalArguments | Should -Be "--enableFoo"
|
||||
|
||||
$element = GetNdMatrixElement @(2, 1, 1) $matrix $dimensions
|
||||
$element.parameters.operatingSystem | Should -Be "macos-11"
|
||||
$element.parameters.framework | Should -Be "netcoreapp2.1"
|
||||
$element.parameters.additionalArguments | Should -Be "--enableFoo"
|
||||
}
|
||||
|
||||
It "Should initialize a sparse matrix from an N-dimensional matrix" -TestCases @(
|
||||
@{ i = 0; name = "windows2022_net461"; operatingSystem = "windows-2022"; framework = "net461"; additionalArguments = ""; }
|
||||
@{ i = 1; name = "ubuntu1804_netcoreapp21_withfoo"; operatingSystem = "ubuntu-18.04"; framework = "netcoreapp2.1"; additionalArguments = "--enableFoo"; }
|
||||
@{ i = 2; name = "macOS11_net461"; operatingSystem = "macos-11"; framework = "net461"; additionalArguments = ""; }
|
||||
) {
|
||||
$sparseMatrix = GenerateSparseMatrix $generateConfig.matrixParameters $generateConfig.displayNamesLookup
|
||||
$dimensions = GetMatrixDimensions $generateConfig.matrixParameters
|
||||
$size = ($dimensions | Measure-Object -Maximum).Maximum
|
||||
$sparseMatrix.Count | Should -Be $size
|
||||
|
||||
$sparseMatrix[$i].name | Should -Be $name
|
||||
$element = $sparseMatrix[$i].parameters
|
||||
$element.operatingSystem | Should -Be $operatingSystem
|
||||
$element.framework | Should -Be $framework
|
||||
$element.additionalArguments | Should -Be $additionalArguments
|
||||
}
|
||||
|
||||
It "Should generate a sparse matrix from an N-dimensional matrix config" {
|
||||
$sparseMatrix = GenerateMatrix $generateConfig "sparse"
|
||||
$sparseMatrix.Length | Should -Be 4
|
||||
}
|
||||
|
||||
It "Should initialize a full matrix from an N-dimensional matrix config" {
|
||||
$matrix = GenerateMatrix $generateConfig "all"
|
||||
$matrix.Length | Should -Be 13
|
||||
}
|
||||
}
|
||||
|
||||
Describe "Config File Object Conversion" -Tag "convert" {
|
||||
BeforeEach {
|
||||
$config = GetMatrixConfigFromJson $matrixConfig
|
||||
}
|
||||
|
||||
It "Should convert a matrix config" {
|
||||
$config.matrixParameters[0].Name | Should -Be "operatingSystem"
|
||||
$config.matrixParameters[0].Flatten()[0].Value | Should -Be "windows-2022"
|
||||
|
||||
$config.displayNamesLookup | Should -BeOfType [Hashtable]
|
||||
$config.displayNamesLookup["--enableFoo"] | Should -Be "withFoo"
|
||||
|
||||
$config.include.Length | Should -Be 1
|
||||
$config.exclude.Length | Should -Be 3
|
||||
}
|
||||
}
|
||||
|
||||
Describe "Platform Matrix Post Transformation" -Tag "transform" {
|
||||
BeforeEach {
|
||||
$config = GetMatrixConfigFromJson $matrixConfig
|
||||
}
|
||||
|
||||
It "Should match partial matrix elements" -TestCases @(
|
||||
@{ source = [Ordered]@{ a = 1; b = 2; }; target = [Ordered]@{ a = 1 }; expected = $true }
|
||||
@{ source = [Ordered]@{ a = 1; b = 2; }; target = [Ordered]@{ a = 1; b = 2 }; expected = $true }
|
||||
@{ source = [Ordered]@{ a = 1; b = 2; }; target = [Ordered]@{ a = 1; b = 2; c = 3 }; expected = $false }
|
||||
@{ source = [Ordered]@{ a = 1; b = 2; }; target = [Ordered]@{ }; expected = $false }
|
||||
@{ source = [Ordered]@{ }; target = [Ordered]@{ a = 1; b = 2; }; expected = $false }
|
||||
) {
|
||||
MatrixElementMatch $source $target | Should -Be $expected
|
||||
}
|
||||
|
||||
It "Should remove matrix elements based on exclude filters" {
|
||||
$matrix = GenerateFullMatrix $config.matrixParameters $config.displayNamesLookup
|
||||
$withExclusion = ProcessExcludes $matrix $config.exclude
|
||||
$withExclusion.Length | Should -Be 5
|
||||
|
||||
$matrix = GenerateSparseMatrix $config.matrixParameters $config.displayNamesLookup
|
||||
[array]$withExclusion = ProcessExcludes $matrix $config.exclude
|
||||
$withExclusion.Length | Should -Be 1
|
||||
}
|
||||
|
||||
It "Should add matrix elements based on include elements" {
|
||||
$matrix = GenerateFullMatrix $config.matrixParameters $config.displayNamesLookup
|
||||
$withInclusion = ProcessIncludes $config $matrix "all"
|
||||
$withInclusion.Length | Should -Be 15
|
||||
}
|
||||
|
||||
It "Should include and exclude values with a matrix" {
|
||||
[Array]$matrix = GenerateMatrix $config "all"
|
||||
$matrix.Length | Should -Be 8
|
||||
|
||||
$matrix[0].name | Should -Be "windows2022_netcoreapp21"
|
||||
$matrix[0].parameters.operatingSystem | Should -Be "windows-2022"
|
||||
$matrix[0].parameters.framework | Should -Be "netcoreapp2.1"
|
||||
$matrix[0].parameters.additionalArguments | Should -Be ""
|
||||
|
||||
$matrix[1].name | Should -Be "windows2022_netcoreapp21_withfoo"
|
||||
$matrix[1].parameters.operatingSystem | Should -Be "windows-2022"
|
||||
$matrix[1].parameters.framework | Should -Be "netcoreapp2.1"
|
||||
$matrix[1].parameters.additionalArguments | Should -Be "--enableFoo"
|
||||
|
||||
$matrix[2].name | Should -Be "ubuntu1804_net461"
|
||||
$matrix[2].parameters.framework | Should -Be "net461"
|
||||
$matrix[2].parameters.operatingSystem | Should -Be "ubuntu-18.04"
|
||||
$matrix[2].parameters.additionalArguments | Should -Be ""
|
||||
|
||||
$matrix[4].name | Should -Be "macOS11_net461"
|
||||
$matrix[4].parameters.framework | Should -Be "net461"
|
||||
$matrix[4].parameters.operatingSystem | Should -Be "macos-11"
|
||||
$matrix[4].parameters.additionalArguments | Should -Be ""
|
||||
|
||||
$matrix[7].name | Should -Be "windows2022_net50_enableWindowsFoo"
|
||||
$matrix[7].parameters.framework | Should -Be "net50"
|
||||
$matrix[7].parameters.operatingSystem | Should -Be "windows-2022"
|
||||
$matrix[7].parameters.additionalArguments | Should -Be "--enableWindowsFoo"
|
||||
}
|
||||
|
||||
It "Should parse a config with an empty base matrix" {
|
||||
$matrixConfigForIncludeOnly = @"
|
||||
{
|
||||
"include": [
|
||||
{
|
||||
"operatingSystem": "windows-2022",
|
||||
"framework": "net461"
|
||||
}
|
||||
]
|
||||
}
|
||||
"@
|
||||
|
||||
$config = GetMatrixConfigFromJson $matrixConfigForIncludeOnly
|
||||
[Array]$matrix = GenerateMatrix $config "all"
|
||||
$matrix.Length | Should -Be 1
|
||||
$matrix[0].name | Should -Be "windows2022_net461"
|
||||
}
|
||||
|
||||
It "Should parse a config with an empty include" {
|
||||
$matrixConfigForIncludeOnly = @"
|
||||
{
|
||||
"matrix": {
|
||||
"operatingSystem": "windows-2022",
|
||||
"framework": "net461"
|
||||
}
|
||||
}
|
||||
"@
|
||||
|
||||
$config = GetMatrixConfigFromJson $matrixConfigForIncludeOnly
|
||||
[Array]$matrix = GenerateMatrix $config "all"
|
||||
$matrix.Length | Should -Be 1
|
||||
$matrix[0].name | Should -Be "windows2022_net461"
|
||||
}
|
||||
}
|
||||
|
||||
Describe "Platform Matrix Generation With Object Fields" -Tag "objectfields" {
|
||||
BeforeEach {
|
||||
$matrixConfigForObject = @"
|
||||
{
|
||||
"matrix": {
|
||||
"testObject": {
|
||||
"testObjectName1": { "testObject1Value1": "1", "testObject1Value2": "2" },
|
||||
"testObjectName2": { "testObject2Value1": "1", "testObject2Value2": "2" }
|
||||
},
|
||||
"secondTestObject": {
|
||||
"secondTestObjectName1": { "secondTestObject1Value1": "1", "secondTestObject1Value2": "2" }
|
||||
},
|
||||
"testField": [ "footest", "bartest" ]
|
||||
},
|
||||
"include": [
|
||||
{
|
||||
"testObjectInclude": {
|
||||
"testObjectIncludeName": { "testObjectValue1": "1", "testObjectValue2": "2" }
|
||||
},
|
||||
"testField": "footest"
|
||||
}
|
||||
]
|
||||
}
|
||||
"@
|
||||
$objectFieldConfig = GetMatrixConfigFromJson $matrixConfigForObject
|
||||
}
|
||||
|
||||
It "Should parse dimensions properly" {
|
||||
[Array]$dimensions = GetMatrixDimensions $objectFieldConfig.matrixParameters
|
||||
$dimensions.Length | Should -Be 3
|
||||
$dimensions[0] | Should -Be 2
|
||||
$dimensions[1] | Should -Be 1
|
||||
$dimensions[2] | Should -Be 2
|
||||
}
|
||||
|
||||
It "Should populate a sparse matrix dimensions properly" {
|
||||
[Array]$matrix = GenerateMatrix $objectFieldConfig "sparse"
|
||||
$matrix.Length | Should -Be 3
|
||||
|
||||
$matrix[0].name | Should -Be "testObjectName1_secondTestObjectName1_footest"
|
||||
$matrix[0].parameters.testField | Should -Be "footest"
|
||||
$matrix[0].parameters.testObject1Value1 | Should -Be "1"
|
||||
$matrix[0].parameters.testObject1Value2 | Should -Be "2"
|
||||
$matrix[0].parameters.secondTestObject1Value1 | Should -Be "1"
|
||||
$matrix[0].parameters.Count | Should -Be 5
|
||||
|
||||
$matrix[1].name | Should -Be "testObjectName2_secondTestObjectName1_bartest"
|
||||
$matrix[1].parameters.testField | Should -Be "bartest"
|
||||
$matrix[1].parameters.testObject2Value1 | Should -Be "1"
|
||||
$matrix[1].parameters.testObject2Value2 | Should -Be "2"
|
||||
$matrix[1].parameters.secondTestObject1Value1 | Should -Be "1"
|
||||
$matrix[1].parameters.Count | Should -Be 5
|
||||
|
||||
$matrix[2].name | Should -Be "testObjectIncludeName_footest"
|
||||
$matrix[2].parameters.testField | Should -Be "footest"
|
||||
$matrix[2].parameters.testObjectValue1 | Should -Be "1"
|
||||
$matrix[2].parameters.testObjectValue2 | Should -Be "2"
|
||||
$matrix[2].parameters.Count | Should -Be 3
|
||||
}
|
||||
|
||||
It "Should splat matrix entries that are objects into key/values" {
|
||||
[Array]$matrix = GenerateMatrix $objectFieldConfig "all"
|
||||
$matrix.Length | Should -Be 5
|
||||
|
||||
$matrix[0].name | Should -Be "testObjectName1_secondTestObjectName1_footest"
|
||||
$matrix[0].parameters.testField | Should -Be "footest"
|
||||
$matrix[0].parameters.testObject1Value1 | Should -Be "1"
|
||||
$matrix[0].parameters.testObject1Value2 | Should -Be "2"
|
||||
$matrix[0].parameters.secondTestObject1Value1 | Should -Be "1"
|
||||
$matrix[0].parameters.Count | Should -Be 5
|
||||
|
||||
$matrix[3].name | Should -Be "testObjectName2_secondTestObjectName1_bartest"
|
||||
$matrix[3].parameters.testField | Should -Be "bartest"
|
||||
$matrix[3].parameters.testObject2Value1 | Should -Be "1"
|
||||
$matrix[3].parameters.testObject2Value2 | Should -Be "2"
|
||||
$matrix[3].parameters.secondTestObject1Value1 | Should -Be "1"
|
||||
$matrix[3].parameters.Count | Should -Be 5
|
||||
|
||||
$matrix[4].name | Should -Be "testObjectIncludeName_footest"
|
||||
$matrix[4].parameters.testField | Should -Be "footest"
|
||||
$matrix[4].parameters.testObjectValue1 | Should -Be "1"
|
||||
$matrix[4].parameters.testObjectValue2 | Should -Be "2"
|
||||
$matrix[4].parameters.Count | Should -Be 3
|
||||
}
|
||||
}
|
||||
|
||||
Describe "Platform Matrix Job and Display Names" -Tag "displaynames" {
|
||||
BeforeEach {
|
||||
$matrixConfigForGenerate = @"
|
||||
{
|
||||
"displayNames": {
|
||||
"--enableFoo": "withfoo"
|
||||
},
|
||||
"matrix": {
|
||||
"operatingSystem": "ubuntu-18.04",
|
||||
"framework": [
|
||||
"net461",
|
||||
"netcoreapp2.1"
|
||||
],
|
||||
"TestNullField": null,
|
||||
"TestObjectField": {
|
||||
"TestObjectValueName": {
|
||||
"foo": "bar",
|
||||
"baz": "qux"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
"@
|
||||
$generateConfig = GetMatrixConfigFromJson $matrixConfigForGenerate
|
||||
}
|
||||
|
||||
It "Should enforce valid display name format" {
|
||||
$generateconfig.displayNamesLookup["net461"] = '123.Some.456.Invalid_format-name$(foo)'
|
||||
$generateconfig.displayNamesLookup["netcoreapp2.1"] = (New-Object string[] 150) -join "a"
|
||||
$matrix = GenerateFullMatrix $generateconfig.matrixParameters $generateconfig.displayNamesLookup
|
||||
|
||||
$matrix[0].name | Should -Be "ubuntu1804_123some456invalid_formatnamefoo_TestObjectValueName"
|
||||
|
||||
$matrix[1].name.Length | Should -Be 100
|
||||
# The withfoo part of the argument gets cut off at the character limit
|
||||
$matrix[1].name | Should -BeLike "ubuntu1804_aaaaaaaaaaaaaaaaa*"
|
||||
}
|
||||
|
||||
It "Should create a valid display name when there are leading numbers" {
|
||||
$generateconfig.displayNamesLookup["ubuntu-18.04"] = '123_ubuntu1804'
|
||||
$matrix = GenerateFullMatrix $generateconfig.matrixParameters $generateconfig.displayNamesLookup
|
||||
|
||||
$matrix[0].name | Should -Be "job_123_ubuntu1804_net461_TestObjectValueName"
|
||||
}
|
||||
|
||||
It "Should create a valid job name when there are leading numbers" {
|
||||
$matrixConfigForGenerate = @"
|
||||
{
|
||||
"matrix": {
|
||||
"numField1": [1, 2, 3],
|
||||
"letterField": ["a", "b", "c"]
|
||||
}
|
||||
}
|
||||
"@
|
||||
$generateConfig = GetMatrixConfigFromJson $matrixConfigForGenerate
|
||||
$matrix = GenerateFullMatrix $generateconfig.matrixParameters $generateconfig.displayNamesLookup
|
||||
$matrix[0].name | Should -Be "job_1_a"
|
||||
}
|
||||
|
||||
It "Should create a valid job name when parameter values are all numbers" {
|
||||
$matrixConfigForGenerate = @"
|
||||
{
|
||||
"matrix": {
|
||||
"numField1": ["1", "2", "3"],
|
||||
"numField2": [4, 5, 6]
|
||||
}
|
||||
}
|
||||
"@
|
||||
$generateConfig = GetMatrixConfigFromJson $matrixConfigForGenerate
|
||||
$matrix = GenerateSparseMatrix $generateconfig.matrixParameters $generateconfig.displayNamesLookup
|
||||
$matrix[0].name | Should -Be "job_1_4"
|
||||
$matrix[1].name | Should -Be "job_2_5"
|
||||
$matrix[2].name | Should -Be "job_3_6"
|
||||
}
|
||||
|
||||
It "Should generate a display name with null and object values" {
|
||||
$matrix = GenerateMatrix $generateConfig "sparse"
|
||||
$matrix[0].name | Should -Be "ubuntu1804_net461_TestObjectValueName"
|
||||
}
|
||||
}
|
||||
@ -1,14 +0,0 @@
|
||||
{
|
||||
"displayNames": {
|
||||
"importedBaz": "importedBazName"
|
||||
},
|
||||
"matrix": {
|
||||
"Foo": [ "foo1", "foo2" ],
|
||||
"Bar": [ "bar1", "bar2" ]
|
||||
},
|
||||
"include": [
|
||||
{
|
||||
"Baz": "importedBaz"
|
||||
}
|
||||
]
|
||||
}
|
||||
Loading…
Reference in New Issue
Block a user