* Retry container deletion
* Do not try to purge keyvaults with purge protection
* Delete all blobs when container has immutability
* Skip missing blob container properties
* Fix null container
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* identify and resolve missing function GenerateMatrixForConfig
* identify and resolve miss-used null-coalesce operator
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* package-properties are now populated with matrix configurations from their ci.yml if present
* create new code path for generate-job-matrix.yml which combines Create-JobMatrix and the "distribute-packages-to-matrix" action to generate dynamic matrices for PRs
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* Run perf tests under federated auth
Run the tests under the same federated auth used to deploy the tests and setup the variables needed to configure AzurePipelineCredential.
* Fix indention for working directory
---------
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Refactor PackageProps to make the Yaml loading and processing methods common
* Updates for feedback
---------
Co-authored-by: James Suplizio <jasupliz@microsoft.com>
* Remove default sub config sub-config-azure-cloud-test-resources
We no longer want to default the configuration and instead
default from the service connection which had the info we need.
* Fix quoting
---------
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
* save-package-properties.yml only activates PR diff mechanism if the service directory is "auto"
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* Escape parentheses in includes to fix issues in publishing
* Only escape close parens
* Comments
---------
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
* Changes required for sdk-repo based emitter pipelines
* Rename template
* Convert emitter archetype to pipeline template
* Move the archetype template out of the stages folder
---------
Co-authored-by: Patrick Hallisey <pahallis@microsoft.com>
* Add CompatibleConvertFrom-Yaml to common Package-Helpers
* Add CI artifact parsing to Save-Package-Properties, now each individual packageinfo.json file contains the relevant ci artifact lines for the package. This short circuits needing to re-parse this information in common checks.
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* Export the subscription data from the service connection
* Update deploy-test-resources.yml
---------
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* don't verify changelog for packages that aren't actively published anymore
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
Co-authored-by: Scott Beddall <45376673+scbedd@users.noreply.github.com>
Co-authored-by: Ben Broderick Phillips <ben@benbp.net>
* updating package properties with direct/indirect (if named differently) as well as pulling BuildDocs from ci.yml artifact list if it exists
* eliminate the addition of buildDocs property. it requires powershell-yaml to be present on our base function. not good
* remove call to InitializeBuildDocs
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* Add descriptive error when variable groups are not added to pipeline
* Add emoji to deploy step title to make it easier to find
* Move yaml json checking into powershell script
* Fix base sub config empty/string checking
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* Refactor prepare-pipelines
* Convert pipeling generation single to use template
* Convert to pwsh for setting the variables
* Fix switch expression
* Add path separator in front of devopspath
* Continue on error for individual step failures
* Remove unnecessary parameters
---------
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Apply a few changes to Get-PRPackageProperties to make it more customizable for our users. Remove wonky gap + extra spacing that should pass the patch application process now that we've manually aligned common.ps1
* account for feedback -- rename CanaryPackages to AdditionalValidationPackages
* dependentPackagesforInclusion -> additionalValidationPackages
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* bump proxy version
* bump proxy version
* restore most of the old performance, only fall back to json matching when something goes wrong
* use newest version with protection from invalid json
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
Given we cannot create subfolders under .github\workflow folder
and we have repos that want additional workflows then we have
in the tools repo we need to setup a file pattern matching for
the sync processing to only sync the ones we want to be common.
For now just use the existing *event* pattern but we can move them
to another pattern similar to *common* or *sync* in the future as
we add more common workflows.
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
* Add RestAPISpecsDocsRepos to branch cleanup.
* Remove commented out yml, add pre-message to limit output and output core limit at the start of processing
* updates for feedback
---------
Co-authored-by: James Suplizio <jasupliz@microsoft.com>
* we want the path that we write to to be defined by literalpath, otherwise files with [ or ] will break powershell when writing the file to disk
* slight wording adjustment so the command can be invoked directly
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* more compatibility with expanding/contracting packages. add ability for packages to have DependentPackages that must be included in the set of packages that should be built given a changeset
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
Co-authored-by: Scott Beddall <45376673+scbedd@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Do not mark values as secret that are from git-hosted sub config files
* Simplify sub config build yaml
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* missed a condition passthrough
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
Co-authored-by: Scott Beddall <45376673+scbedd@users.noreply.github.com>
* add additional argument to Save-Package-Properties to allow for usage in pull request context
---------
Co-authored-by: Scott Beddall (from Dev Box) <scbedd@microsoft.com>
Co-authored-by: Scott Beddall <45376673+scbedd@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Set storage account test resources to disable blob public access
* Skip adding network rules to storage accounts that don't need them during cleanup
* Add succeeded check to set pipeline subnet info step
* Disable network firewall by default in resource creation/removal
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* Support storage network access and worm removal in remove test resources script
* Move storage network access script to common resource helpers file
* Improve storage container deletion resilience
* Plumb through pool variable to live test cleanup template
* Add sleep for network rule application
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* commit the file changes so that we can see them running
* use standalone tool
---------
Co-authored-by: Scott Beddall (from Dev Box) <scbedd@microsoft.com>
* Ensure subConfigFiles is not an empty string
* Skip instances where $file is an empty string
---------
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
* Fix default value for env vars in build-test-resource-config template
* Add empty pool condition
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* Restrict live test storage account access to client IP
* Add storage pools to test resources vnet allowlist
* Use pool subnet map to reduce number of subnets added to live test resources
* snap
* Add test resource parameter to add ip ranges to storage firewall
* Validate and update ip firewall rules
* Get pool subnet using separate workload identity
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
Add logging and fix display name given Agent.JobStatus doesn't evaluate currently in the display name property.
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Merge subscription configuration file values with the supplied subscription config
* Bring forward working parts
* Collapse into a single task
* SubConfig-Helpers.ps1 changes
* Update remove-test-resources.yml
---------
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
* Removal of the devops release PAT
* Don't pass in the access token, just use the AzureCLI task and do everything in the scripts
* Updates for feedback
---------
Co-authored-by: James Suplizio <jasupliz@microsoft.com>
* Plumb env vars through deploy test resources
* Update eng/common/TestResources/deploy-test-resources.yml
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
* Update eng/common/TestResources/deploy-test-resources.yml
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
* PAT Removal from pipeline generation yml
* update version of PipelineGenerator to install
---------
Co-authored-by: James Suplizio <jasupliz@microsoft.com>
* Reuse same branch name for previews of a PR
* Default AddBuildSummary parameter using pipeline variable existence
---------
Co-authored-by: Patrick Hallisey <pahallis@microsoft.com>
* Changes to scripts to deal with PATs and AccessTokens
* swap access and auth for add-retention-lease
* AuthToken to BearerToken and remove unused Base64EncodedAuthToken from the script parameters
* remove unneccsary if not null check for the mandatory parameter
---------
Co-authored-by: James Suplizio <jasupliz@microsoft.com>
* Migrate stress cluster to use workload identity
* Only sleep to sync fed creds when namespace is new
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
