* Support AAD graph and Microsoft Graph service principal APIs
* Consolidate service principal wrapper creation
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* Generate map files in build
* Add .map file artifact publishing
* Add Xlinker to spelling exceptions
* Remove PublishMapFiles from clang builds
* Generate the map file artifact name
* CXX_COMPILER_ID and some logging for Clang which might be linking using the GNU linker by default
* More logging
* Move logging out of conditional
* Logging up high, use OS to determine which link flags to set
* Use linker options specific to AppleClang's context, publish map files for all platforms
The focus of these changes is to ensure that the service principal is
explicitly granted the "Owner" role on the active resource group, whether
the principal was newly created or a cached instance was used.
Co-authored-by: Jesse Squire <jesse.squire@gmail.com>
* Update vcpkg to the newest published tag
* Update to latest commit sha from 12-02-2021
* Update to a commit sha from 11-10-2021, after last SDK release
* Treat third-party dependencies as system to avoid triggering warnings in code we don't control
* Use stress test environment defaults for group and subscription
* Fix parameter passing from deploy-stress-tests.ps1 script
* Redact stress deployment logs and simplify image handling
* Use DevopsLogging parameter to prevent secret logging in non-devops CI environments
* Use switch type for DevopsLogging parameter
* Remove boolean parameter usage in favor of [switch]
* Add default parameter set usage comment
* Throw when clusterGroup and/or subscription is not specified for custom environments
* Add helper function for logging azure pipelines vso commands
* Invert SuppressVsoCommands binary default value
* Vso command fixes
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* Add disk metrics script and template
* Support linux disk matrics
* Add continueOnError to write-filesystemmetrics.yml
Co-authored-by: Patrick Hallisey <pahallis@microsoft.com>
* Add PowerShell Script for Validating READMEs
* Remember my VSCode doesn't use autosave
Co-authored-by: Alan Zimmer <48699787+alzimmermsft@users.noreply.github.com>
* Add spell check pin changes
* Apply suggestions from code review
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Add package-lock.json
* Set node version and upgrade npm in check-spelling.yml
* Do not mutate local NPM environment
* Remove unnecessary comment, we no longer use npm exec
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Exclude certain live test deployment outputs from being marked as log secrets
* debug
* Update subscription configuration merge jobs to use secret handler
* Rename subscription config helper function script
* Fix variable name reference in scope
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* update docker-start-proxy to have a slightly different default parameter
* we were surrounding the value in single quotes, breaking recording load
Co-authored-by: scbedd <45376673+scbedd@users.noreply.github.com>
* Narrower build trigger matrix, introduce batching so changes to main don't trigger builds for every checkin if there are other builds running, expand core to test everything in CI
* Wire up CtestExcludeRegex
* Improvements including scoping to exclude storage tests
* Use correct trigger settings for storage
* Add test result uploading
* Use -T
* Add displayName and buildConfiguration
* Add live testing result upload
* buildConfiguration -> testRunTitle
* Live tests 'Validate' -> 'ValidateLive' to differentiate job names in Test results output
* Refactor template to use multiple builds on the same machine resetting the machine and runtime varaibles along the way
Move template expression out of default matrix parameters
Remove extra reference to ci/jobs.yml
Remove unnecessary ci.yml, change param declaration in devops-variables-set.yml
Heredoc
Fix set/reset scripts after testing with devops outputs
-ErrorAction Ignore
Smaller matrix
OsVMImage comes from machine, not parameters
Add Ubuntu 20 to matrix
Remove checkout: self, it is implied
Move machine-setup steps outside of loop
CODE_COVERAGE handling
tab
Remove creation of CODE_COVERAGE
Use AptDependencies and add apt update to dependency install step
Remove extra preparation steps
Enable code coverage check scenario
Add MacOS 11
Disable vcpkg cache
Coverage configuration
Pass parameter properly
bool -> boolean
Install coverage tools if it's possible that coverage may run
More logging on code coverage
Show contents of Makefile
One vcpkg to build them all
Verbose
SourcesDirectory
Nesting VcpkgInstall
workingDirectory for vcpkg install
AptDependencies
complete the conditional
Enable all matrix entries
sudo
Remove extra chatter
More scrubbing post-build to help with Windows scenarios
sudo echo
Get-ChildItem
ErrorAction
sudo sh -c
More ErrorAction
output sourceListFile
Disable additional sources.list
Install coverage tools when used for coverage
COndition
Do not build testing for Windows arm64
Correct path for coverage tools
ne -> eq
Add more scenarios to the matrix
Remove libxml2
Add MacOS 10.15 to matrix
Revert "Remove libxml2"
This reverts commit 178af8b89abb21ee84dbcd25b4d54bd90ec1d44c.
Add libxml2 to MacOS 10.15 matrix entry
Display name decorator, refine the matrix
Remove OsVmImage variable
Add commentary to tests about matrix
Rebalance matrix, nest EnvVars, DisplayNameDecorator
Correct parameter access
AptDependencies
Ensure building for WindowsStore in UWP builds, Only build tests for debug builds in altered matrix entires, Linux build type specified as environment variable
Rebalance matrix for better timing
Remove all instances of libxml2 in vcpkg dependencies
Move common devops tasks to eng/common
Set eng/common paths in pipelines
Revert "Remove all instances of libxml2 in vcpkg dependencies"
This reverts commit 35ffbc7aac87f400fdf3351076346750c1c750ed.
https://github.com/Azure/azure-sdk-for-cpp/issues/2946
Permit words like iname in more pipeline .yml files
Remove libxml2 from vcpkg dependencies for Windows
Perf test matrix maxParallel: 10
Validate
apt install first
apt install first
maxParallel: 12
-j 8
-j 10
Use matrix generator
Add expected parameters
Move parameters to appropriate file, fix cloud spec
Specify correct platform matrix location
DependsOn
Platform matrix
Remove DisplayNameDecorator, try more clang information
Rapid iteration matrix
clang --version
install clang-11
Use correct pool
Use full matrix, revert additional logic in jobs/ci.yml
* Review feedback: newline
* Review feedback: Test in release, too
* Remove DisplayNameDecorator (it was useful when we were putting multiple builds into a single job, now it is redundant)
* Remove displayNames
* Matrix review feedback
* Conditions
* Rename ci.yml -> ci.tests.yml
* Consume Codeowners parser library, ceperate users from teams in codeownerse
* Update get-pr-owners and related logic
Co-authored-by: Chidozie Ononiwu <chononiw@microsoft.com>
* Fix up identity resolver to make it more reliable, publish notification configuration as a tool
* Use both username and email to resolve queuing user
Co-authored-by: Chidozie Ononiwu <chononiw@microsoft.com>
* Mitigate relative path calculation error on multiple iterations
* Revert "Mitigate relative path calculation error on multiple iterations"
This reverts commit 45baedd990c6a3085742a38a4891d8706a93be77.
* GetRelativePath should check if path is already relative before calling [IO.Path]::GetRelativePath
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
* Handle assembly already loaded errors in deploy stress test script
* Add dependency checks and installation links to deploy-stress-tests.ps1
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* Remove empty sections in changelog entry as part of prepare release
* Update SECTIONS_HEADER_REGEX to accomodate various header levels
Co-authored-by: Chidozie Ononiwu <chononiw@microsoft.com>
* Common pipeline template and script to detect API changes
* Cahnges as per review comments
* Review comment changes
* CCahgnes as per review comments
* Show warning for list of failed packages to detect API changes
* Apply suggestions from code review
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: praveenkuttappan <prmarott@microsoft.com>
Co-authored-by: praveenkuttappan <55455725+praveenkuttappan@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Make ServiceName optional for verify-changelog
In cases like go where we don't pass a service name having this default to "not-specified" breaks things so we should allow for ServiceName and/or ServiceDirectory to be empty.
* Put quotes around the strings to allow for empty
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Since we are treating v0 versions as prerelease we need to make sure
we don't accidently start to add the bogus prerelease label in cases where
we call ToString() on the version.
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Add ability to exit gracefully when all files in the diff are excluded
* Address case where cspell exits with an error when all files from the 'files' config list are excluded by the expressions in 'ignorePaths'
* Add tests
* Review feedback: impl goes at the bottom and should be treated as a script, logic for testing should happen above that and exit appropriately if running tests
* Import common instead of logging
* Enable strict mode
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
* Doc Updates and Revisions for External Use
The focus of these changes is to revise the script to better support use
by external contributors and others outside of the Azure SDK ecosystem and
without access to the Microsoft AAD Tenant.
Changes include:
- Creation of a new Test Application service principal is now possible
from a non-Microsoft AAD tenant.
- When a new Test Application principal is created, the principle of least
privilege is now applied; the new Test Application is granted ownership
of the resource group associated with the test resources and no longer
has access to any other resources in the subscription.
- If an existing Test Application principal is specified, it will be
assigned ownership of the resource group created. This supports using
a Test Application principal without privileges at the subscription-level.
- When no provisioner is specified, the script is now executed in the
context of the caller rather than the Test Application principal.
This supports using a Test Application principal that has restricted
privileges and better aligns to the purpose of the Test Application
principal.
- The `$TestApplicationOid` is now explicitly bound at the time a new Test
Application principal is created rather than having to query for it later.
- Common error scenarios resulting from lack of permissions now provide
messaging with more context of why the failure occurred and suggest
remediation.
- Added new examples to illustrate the common call patterns needed by
external contributors running the script, outside of the Microsoft tenant
and Azure SDK ecosystem.
- Documentation has been enhanced with additional context to detail the
permissions and roles assigned by the script.
- Added documentation details for Bicep template use.
* Add the provisioner OID to the deployment params
Key Vault needs this to deploy Managed HSMs. There's a corresponding change necessary in test-resources.json I'll roll out across languages.
* Fixing typos and spelling mistakes
Co-authored-by: Jesse Squire <jesse.squire@gmail.com>
Co-authored-by: Heath Stewart <heaths@microsoft.com>
* Attempt tagging of @danieljurek in PRBody
* Attempt to assign reviewers
* Use codeowners, filter out groups, use a default owner, add comments to issue.
* Move variables to right level of config
* Set PrOwners variable
* Remove PR owners, requires admin permissions
* Escape DevOps variable to expand into string
* Add AuthToken
* Tweak portfile and codeowners to validate multiple artifact scenario
* Revert "Tweak portfile and codeowners to validate multiple artifact scenario"
This reverts commit bcc7a36119c7f399f1b079f51889d6ff0d499142.
* Review feedback
* Validate fallback logic
* Deal with empty list of owners
* Revert "Validate fallback logic"
This reverts commit ade2dff5a7d9cac9bb61fdc0f92cd50412b7c31c.
* ignore result of function
* changelog
* Add g++5 to CI legs
* use inline config for g++ 5
* update flag
* set default values for variables
* fix format
* Stress test deployment script resiliency fixes
* Use PSModule-Helpers to install powershell-yaml
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* More logging, faster matrix
* Verbose makefile
* Add code that is expected to cause failure
* Use clang with Ubuntu 20
* Install clang-format-11 in another step
* Revert "Add code that is expected to cause failure"
This reverts commit 7ff7201e1b81e17c168b7598f0d23c489b5e6f43.
* Remove cmake args, add build args -j
* Attempt to purge all vaults, managed HSMs
Reverts #1910. Vaults and managed HSMs are automatically purged on their purge date. The point was to purge them daily to preserve capacity. The default purge date is +90 days.
* Add timeout and more logging
* Pass required -Resource
* Fix log message
* Ensure the $Resource is correctly captured
Added comment to new code explaining why, since ScriptBlock.GetNewClosure() is not working as expected.
* Add -ErrorAction to Receive-Job
Worked without terminating when run locally, but failed on the first error in the AzDO agent.
* Use $using:r instead of creating ScriptBlock
More idiomatic for passing ScriptBlocks to jobs.
* Resolve PR feedback
* Change default DeleteAfterHours to 120
Resolves#1917
* Use the Az cmdlets built-in -AsJob
Co-authored-by: Heath Stewart <heaths@microsoft.com>
- Fix rg.Name to rg.ResourceGroupName
- Add more verbose logging for better debugging
- Handle deleted resource groups when gathering puragable resource
- Remove coerce now that we are collecting in functions
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
* Deploy for stress tests at resource group scope
* Automatically compile bicep files in stress test deploy script
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
Explicitly add '/*' '!/*/' to the list of paths so we include
the files in the root of the repo.
Added some extra command logging so we can easily repro locally without
needing to find the code that ran.
Update to '/eng' so that we only get paths that start with eng instead
of every path that has eng as a folder.
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
* Support building and deploying bicep templates
* Add bicep powershell install aka link to deployment error message
* Write bicep compiled arm templates to temp directory
* Simplify bicep building code/function usage
* Use bicep location for compiled arm templates, and remove them on success
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* Update change log headers based on guideline update
Updates based on https://github.com/Azure/azure-sdk/pull/3103
- Renamed "Key Bugs Fixed" to "Bugs Fixed"
- Renamed "Fixed" to "Other Changes"
Added a warning in validation if at lease one of the recommended
headers aren't used.
* Update eng/common/scripts/ChangeLog-Operations.ps1
Co-authored-by: JoshLove-msft <54595583+JoshLove-msft@users.noreply.github.com>
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: JoshLove-msft <54595583+JoshLove-msft@users.noreply.github.com>
* Put dependency versions
* Update version
* Core does not have any 1.1 functionality
* Core 1.1
* 1.1.0
Co-authored-by: Anton Kolesnyk <antkmsft@users.noreply.github.com>
* Add dependency on LiveTest to release
* Add DependsOn to archetype-cpp-release.yml stages, conditions around LiveTest (if we don't execute live tests don't require them for release)
* Remove template conditions for LiveTest and always require it
* Revert "Remove template conditions for LiveTest and always require it"
This reverts commit dc0615ca084b7bb476cbc3806df70f92f2bf0304.
* Bring changes from JS docs metadata
* Move business logic inside Update-DocsMsMetadata.ps1
* Update with the latest changes in JS PR
* Update from latest PR feedback
* Add check for empty path to prevent crashes when creating relative paths
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
- Switch to using rest instead of cli for querying work items
as cli is limited to 1000 items only.
- Fix issues with Generated fields not being set.
- Correctly sort older workitems by version isntead of string.
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
* Check for API review status only if release date is set in changelog
* Change property name to ReleaseStatus
Co-authored-by: Praveen Kuttappan <praveen.kuttappan@gmail.com>
