Commit Graph

87 Commits

Author SHA1 Message Date
Azure SDK Bot
eeaace3e51
Handle multi-line string concatenation with Write-Warning (#4321)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2023-02-01 20:44:41 -05:00
Azure SDK Bot
c88d0287e9
Sync eng/common directory with azure-sdk-tools for PR 5242 (#4294)
* Consolidate naming logic and generate short hash names for local use

* Shorten long lines

* Handle issues with EnvironmentVariable parameter ref being updated

* Warn on env variable overwrite. Base name generation off resource group

* Use SHA256 algorithm for short name hash

---------

Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2023-01-27 15:17:58 -05:00
Azure SDK Bot
dd58487030
Handle service principal creation quota errors (#4261)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2023-01-24 16:19:05 -05:00
Azure SDK Bot
4dace9b114
Add optional TestResourcesDirectory override parameter (#4105)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2022-11-10 16:37:39 -05:00
Azure SDK Bot
c9ea71f1d6
Add public cloud cosmos endpoint suffix (#4091)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2022-11-07 18:38:28 -05:00
Azure SDK Bot
4c70c2fdc6
Dump out correlation id without verbose logging for resource deployment (#4069)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2022-10-31 17:15:04 -04:00
Azure SDK Bot
9eae0dff88
Fix misleading step name in az module install (#3859)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2022-07-29 14:30:41 -07:00
Azure SDK Bot
8c5dec33be
Use westus as default region in test resource deployments (#3759)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2022-06-21 09:41:39 -04:00
Azure SDK Bot
61636bba41
Add ResourceType parameter - Selects live test or perf test resources (#3740)
Co-authored-by: Mike Harder <mharder@microsoft.com>
2022-06-16 22:19:21 -07:00
Azure SDK Bot
a43622b616
Add skip variable for remove test resources to support debugging (#3648)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2022-05-13 14:12:08 -07:00
Azure SDK Bot
f4a5dbe217
String parsing improvements to service directory resource names (#3644)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2022-05-12 16:02:21 -04:00
Azure SDK Bot
0e68b12f44
Update readme for -pre and -post TestResource scripts (#3645)
Co-authored-by: Christopher Scott <chriss@microsoft.com>
2022-05-11 15:59:18 -07:00
Azure SDK Bot
201d588b6d
Do not pass Generated to -BaseName (#3562)
Co-authored-by: Heath Stewart <heaths@microsoft.com>
2022-04-15 13:24:09 -07:00
Azure SDK Bot
164346796f
Support BaseName overrides in CI mode for New-TestResources.ps1 (#3559)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2022-04-14 17:29:16 -07:00
Azure SDK Bot
7889304391
Sync eng/common directory with azure-sdk-tools for PR 3124 (#3546)
* Do not fail remove test resources step when env var is not set.

* Handle empty service directories in remove test resources script

Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2022-04-11 17:28:51 -07:00
Azure SDK Bot
f0e0070223
Sync eng/common directory with azure-sdk-tools for PR 3067 (#3529)
* Infer live resource group name based on service directory name

* Simplify service directory path splitting

* Use common logic for username and basename generation

* Rename GetServiceName to GetServiceLeafDirectoryName

Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2022-04-07 11:33:38 -07:00
Azure SDK Bot
5f8268dd73
Support nested service directories for local test resources resource group names (#3500)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2022-04-01 18:32:42 -04:00
Azure SDK Bot
6dc2599141
Set owners tag in test resource deployments (#3436)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2022-03-15 16:18:57 -07:00
Azure SDK Bot
31edf57add
Support local test resources behavior with provisioner config (#3408)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2022-03-09 14:04:26 -08:00
Azure SDK Bot
e077374ea5
Add AZURE_SERVICE_DIRECTORY environment variable in test deployment script (#3373)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2022-03-09 13:57:53 -08:00
Azure SDK Bot
5049b8f615
Add warning/exit for Az.Resources 5.3.0 service principal provisioning error (#3343)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2022-02-16 13:01:40 -08:00
Azure SDK Bot
503a870c09
Add catch-all parameter to test resources script (#3334)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2022-02-09 14:37:17 -08:00
Azure SDK Bot
1b12863281
Add max value to DeleteAfter of 168 hours (#3327)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2022-02-09 13:09:16 -08:00
Azure SDK Bot
df691195da
Only fetch provision application oid via API if not supplied (#3266)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2022-01-21 12:03:23 -08:00
Azure SDK Bot
203028a180
Sync eng/common directory with azure-sdk-tools for PR 2513 (#3226)
* Update test resources SP password creation to support Az >= 7.1.0

* Check Az.Resources module version

Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2022-01-11 12:20:24 -08:00
Azure SDK Bot
5ff89bb9e5
Sync eng/common directory with azure-sdk-tools for PR 2464 (#3206)
* Support AAD graph and Microsoft Graph service principal APIs

* Consolidate service principal wrapper creation

Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2022-01-03 16:00:40 -08:00
Azure SDK Bot
e5e637afa3
Ensure ownership grant (#3173)
The focus of these changes is to ensure that the service principal is
explicitly granted the "Owner" role on the active resource group, whether
the principal was newly created or a cached instance was used.

Co-authored-by: Jesse Squire <jesse.squire@gmail.com>
2021-12-09 15:43:15 -05:00
Azure SDK Bot
1ae6ecd0a5
Sync eng/common directory with azure-sdk-tools for PR 2363 (#3159)
* Use stress test environment defaults for group and subscription

* Fix parameter passing from deploy-stress-tests.ps1 script

* Redact stress deployment logs and simplify image handling

* Use DevopsLogging parameter to prevent secret logging in non-devops CI environments

* Use switch type for DevopsLogging parameter

* Remove boolean parameter usage in favor of [switch]

* Add default parameter set usage comment

* Throw when clusterGroup and/or subscription is not specified for custom environments

* Add helper function for logging azure pipelines vso commands

* Invert SuppressVsoCommands binary default value

* Vso command fixes

Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2021-12-07 13:45:51 -08:00
Azure SDK Bot
575af58bfb
Restore deploymentOutputs variable in scope for post deployment script (#3153)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2021-12-02 15:01:54 -05:00
Azure SDK Bot
411331d0ec
Sync eng/common directory with azure-sdk-tools for PR 2248 (#3122)
* Exclude certain live test deployment outputs from being marked as log secrets

* debug

* Update subscription configuration merge jobs to use secret handler

* Rename subscription config helper function script

* Fix variable name reference in scope

Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2021-12-01 16:11:10 -05:00
Azure SDK Bot
ed842fd92b
Add default cloud configuration values to source (#3099)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2021-11-12 10:36:10 -08:00
Azure SDK Bot
b3bd976866
Sync eng/common directory with azure-sdk-tools for PR 2053 (#3027)
* Changing inline bash for stress test resource deployment

* PR-mod

* pr-mod

* pr-mod

* pr-mod

Co-authored-by: Albert Cheng <albertcheng@microsoft.com>
2021-11-02 23:05:22 -07:00
Azure SDK Bot
e7747b4af9
Skip unnecessary and misleading role assignment warnings in local test resource deployment (#2954)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2021-10-07 13:06:09 -04:00
Azure SDK Bot
e5f190c7a5
Handle multiple segments in service directory path (#2943)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2021-10-05 11:56:45 -07:00
Azure SDK Bot
3e2e5a6dfc
Sync eng/common directory with azure-sdk-tools for PR 1983 (#2860)
* Doc Updates and Revisions for External Use

The focus of these changes is to revise the script to better support use
by external contributors and others outside of the Azure SDK ecosystem and
without access to the Microsoft AAD Tenant.

Changes include:

- Creation of a new Test Application service principal is now possible
  from a non-Microsoft AAD tenant.

- When a new Test Application principal is created, the principle of least
  privilege is now applied; the new Test Application is granted ownership
  of the resource group associated with the test resources and no longer
  has access to any other resources in the subscription.

- If an existing Test Application principal is specified, it will be
  assigned ownership of the resource group created.  This supports using
  a Test Application principal without privileges at the subscription-level.

- When no provisioner is specified, the script is now executed in the
  context of the caller rather than the Test Application principal.
  This supports using a Test Application principal that has restricted
  privileges and better aligns to the purpose of the Test Application
  principal.

- The `$TestApplicationOid` is now explicitly bound at the time a new Test
  Application principal is created rather than having to query for it later.

- Common error scenarios resulting from lack of permissions now provide
  messaging with more context of why the failure occurred and suggest
  remediation.

- Added new examples to illustrate the common call patterns needed by
  external contributors running the script, outside of the Microsoft tenant
  and Azure SDK ecosystem.

- Documentation has been enhanced with additional context to detail the
  permissions and roles assigned by the script.

- Added documentation details for Bicep template use.

* Add the provisioner OID to the deployment params

Key Vault needs this to deploy Managed HSMs. There's a corresponding change necessary in test-resources.json I'll roll out across languages.

* Fixing typos and spelling mistakes

Co-authored-by: Jesse Squire <jesse.squire@gmail.com>
Co-authored-by: Heath Stewart <heaths@microsoft.com>
2021-09-13 09:21:47 -07:00
Azure SDK Bot
bc38afd6ef
Sync eng/common directory with azure-sdk-tools for PR 1953 (#2814)
* Add random elements to default base name

* Fix principal

* tab

Co-authored-by: JoshLove-msft <54595583+JoshLove-msft@users.noreply.github.com>
2021-08-30 17:45:51 -07:00
Azure SDK Bot
0758a43000
Sync eng/common directory with azure-sdk-tools for PR 1943 (#2810)
* Bug fix: Storing .env file for bicep file at service dir

* Styling

Co-authored-by: Albert Cheng <albertcheng@microsoft.com>
2021-08-27 16:05:33 -07:00
Azure SDK Bot
2ef96b0b72
ensure test principal creation succeeds properly (#2807)
Co-authored-by: scbedd <45376673+scbedd@users.noreply.github.com>
2021-08-27 20:59:54 +00:00
Azure SDK Bot
8a2c551230
Sync eng/common directory with azure-sdk-tools for PR 1912 (#2767)
* Attempt to purge all vaults, managed HSMs

Reverts #1910. Vaults and managed HSMs are automatically purged on their purge date. The point was to purge them daily to preserve capacity. The default purge date is +90 days.

* Add timeout and more logging

* Pass required -Resource

* Fix log message

* Ensure the $Resource is correctly captured

Added comment to new code explaining why, since ScriptBlock.GetNewClosure() is not working as expected.

* Add -ErrorAction to Receive-Job

Worked without terminating when run locally, but failed on the first error in the AzDO agent.

* Use $using:r instead of creating ScriptBlock

More idiomatic for passing ScriptBlocks to jobs.

* Resolve PR feedback

* Change default DeleteAfterHours to 120

Resolves #1917

* Use the Az cmdlets built-in -AsJob

Co-authored-by: Heath Stewart <heaths@microsoft.com>
2021-08-18 13:23:31 -07:00
Azure SDK Bot
45e48d3be5
Fix resource clean-up script (#2747)
- Fix rg.Name to rg.ResourceGroupName
- Add more verbose logging for better debugging
- Handle deleted resource groups when gathering puragable resource
- Remove coerce now that we are collecting in functions

Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
2021-08-13 16:03:04 -07:00
Azure SDK Bot
531c7df24d
Sync eng/common directory with azure-sdk-tools for PR 1876 (#2742)
* rebase

* Resolve conflict

Co-authored-by: Albert Cheng <albertcheng@microsoft.com>
2021-08-13 11:08:31 -07:00
Azure SDK Bot
c92b095088
Sync eng/common directory with azure-sdk-tools for PR 1894 (#2743)
* Purge Key Vaults after deleting resource group

* Resolve PR feedback

* Purge Key Vaults and Managed HSMs

* Clean up all purgeable KVs and MHSMs

* Ignore null/empty purgeable resources

* Resolve PR feedback

* Resolve PR feedback

* Fix logging, process each item when collection piped

Co-authored-by: Heath Stewart <heaths@microsoft.com>
2021-08-12 19:50:53 -07:00
Azure SDK Bot
fe54edf017
Sync eng/common directory with azure-sdk-tools for PR 1767 (#2554)
* Support building and deploying bicep templates

* Add bicep powershell install aka link to deployment error message

* Write bicep compiled arm templates to temp directory

* Simplify bicep building code/function usage

* Use bicep location for compiled arm templates, and remove them on success

Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2021-07-07 15:13:54 -04:00
Azure SDK Bot
5e071a07d7
Enable Caching of PS Modules (#2363)
- Remove copied AzPowershell utilities
- Add latest AZ module path already on hosted agents to PSModulePath
- Rename setup-az-modules template setup-environments to reflect what is is doing
- Add support for Caching the current user PS Module folder
- Add support for install-module if not already present in module folder
- Organize the live test clean-up script to be in the standard location

Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
2021-05-27 19:04:12 -04:00
Azure SDK Bot
d3d4fdfe2a
Use underscore-prefixed variable name when setting sub config values as secrets (#2134)
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
2021-04-19 14:43:09 -07:00
Azure SDK Bot
760026715a
Sync eng/common directory with azure-sdk-tools for PR 1560 (#2133)
* Add template to merge multiple subscription configurations

* Update eng/common/TestResources/build-test-resource-config.yml

Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>

Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
Co-authored-by: Ben Broderick Phillips <ben@benbp.net>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
2021-04-19 11:00:40 -07:00
Azure SDK Bot
14cc77a963
Sync eng/common directory with azure-sdk-tools for PR 1548 (#2098)
* Remove passing -Mode Complete to deployment

When passing Complete it will remove any resources
already in the resource group that weren't part
of the current deployment. That removal breaks a
lot of assumptions, like multiple deployments when
testing things like smoke-tests or if you are reusing
an existing resource group. We don't want that to happen.

* Remove ServiceDirectory as required parameter for remove/update

When this was made mandatory it broke some usages which didn't
pass the value. Those usages don't need to pass it because
the pass required information in other ways so removing the
requirement for the parameter to be passed.

Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
2021-04-09 18:10:44 -07:00
Azure SDK Bot
1ab49d17d0
Fix BaseName initialization (#2096)
Initializing the BaseName with a ServiceDirectory that contains
a "/", because it is a multiple level path, causes the BaseName
initialization code to fail because it doesn't support the
validation pattern.

In all the known cases we already pass the ResourceGroupName
explicitly so don't need to set the BaseName so we can
skip the initialization in those cases.

Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
2021-04-09 13:32:50 -07:00
Azure SDK Bot
0c1ec5c69e
Sync eng/common directory with azure-sdk-tools for PR 1525 (#2015)
* Use SubscriptionId throughout TestResources

Fixes #1454

* Resolve PR feedback

* Default DeleteAfterHours to 48 for SDK team

Also makes a few other adjustments for subscriptions, like restoring the previous one if available and another was specified.

* Resolve PR feedback

* Change deployment mode to Complete

Also fixes an issue where if the user opted not to deploy to the same resource group, the script would continue execution anyway.

* Use consistent aka links to satisfy link checker

Only need it for the new Update-TestResources.ps1 script, but I wanted them to look consistent.

Co-authored-by: Heath Stewart <heaths@microsoft.com>
2021-04-01 09:39:24 -07:00
Azure SDK Bot
fcbaf7301c
Sync eng/common directory with azure-sdk-tools repository (#2014) 2021-03-31 11:19:45 -07:00