* Add support for Federated Auth to test resources scripts
* Default -- UseFederatedAuth: false
* Clear secrets if FederatedAuth is set
* Template conditions use AzurePowerShell only when a service connection is needed
* Review feedback and pair with Ben
* Update docs
* http:// -> https://
---------
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
* allow an override FILE to be honored by test-proxy-tool
* update local test-framework to honor proxy override version by default
---------
Co-authored-by: Scott Beddall (from Dev Box) <scbedd@microsoft.com>
* Only make one call to GetPackageInfoJson
* fix copy/paste, it should be packageInfoLocation not packageInfoJsonLocation
---------
Co-authored-by: James Suplizio <jasupliz@microsoft.com>
Moving away from SAS tokens for connecting to storage so switching to using a Workload Identity Federation connection to the container to download the needed files.
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
I could've fixed the quote characters alone, but I think it is better to just capture everything, this way it is compatible with both `REF azure-core_1234` and `REF "azure-core_1234"`, as well as `REF foo`.
* Update vcpkg portfile.cmake of sdk to add parameter HEAD_REF
* Update vcpkg portfile.cmake of sdk to use variable VERSION
* Update eng\scripts\Initialize-VcpkgRelease.ps1 regex for quoted REF
* Re-trigger CI
* Revert unnecessary change
---------
Co-authored-by: Anton Kolesnyk <41349689+antkmsft@users.noreply.github.com>
Co-authored-by: Anton Kolesnyk <antkmsft@users.noreply.github.com>
* Pass flag to suppress errors correctly when verifying release change log
* Changes as per review comments
---------
Co-authored-by: Praveen Kuttappan <prmarott@microsoft.com>
* Pipeline template to validate package and update package work item
* Changes to restructure validations
* Additional fixes as per comments
* Remove explicit exit code
* Set erroractionpreference for change log check
---------
Co-authored-by: Praveen Kuttappan <prmarott@microsoft.com>
* All eng/ folder changes necessary to enable 1es-templates for the azure-sdk-for-cpp
* Includes updates to all ci.yml files to reflect new syntax requirements from 1es-template changes to archetype-sdk-client
In the case there is exactly one previous release PS will return the single object and thus the Count property will not exist. Instead this change ensures that we always have a list.
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Run publish on failed or succeeded
* Expand agent os string detection
* Check agent job status env var for artifact name detection
* Add sbomEnabled flag to publish template
* Fix image and artifact name conditional
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* Add 1es changes for job/matrix generation and publish
* Use more flexible pool filter for prev/next pool matches
* consolidate displayname definition
* use linux pool variables for generate matrix job
* Fix publish artifact
* Use single publish task for publish artifact
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* Support resolving environment variable references in matrix config
* Improve type and null handling
* Fix reference bug
* Change behavior on missing env vars to throw
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* Updated tests to support -UserAuth test resources switch to enable non-corpnet access.
* Try setting sample environment variables in sample script pipeline
* Upload logs on cmake generate/build failure
* Copy relevant files to staging location, clean up after publishing
* Update cspell words
* Test: break cmake generate in a way that'll produce SOME log files.
* Revert "Test: break cmake generate in a way that'll produce SOME log files."
This reverts commit d5a300587b3359ed3f2bf7be5abea40868f958a7.
* Show failure logs instead of uploading artifacts. Only show log files for vcpkg at this time
* Reapply "Test: break cmake generate in a way that'll produce SOME log files."
This reverts commit 582629ad587f3c9b652ac1f8a7eda5e12d46843a.
* Revert "Reapply "Test: break cmake generate in a way that'll produce SOME log files.""
This reverts commit 7a35250c7e598fc6e1c060f896cbc952704e40bf.
* Another type of test
* Revert inadvertent whitespace changes
* Revert "Another type of test"
This reverts commit e38013f95b152e4e2a3b9d27ccf1eb9b63fa1817.
* More logging
* Fix sparse-checkout git command line behavior
* Use command arg parsing behavior in sparse-checkout.yml
---------
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
* Prepare-Release.ps1: Make dateTime.ToString("MM/dd/yyyy") to work on exotic set-ups
On my machine, I experimented with the registry, and the worst part is that I don't remember/know how to reset it back.
The work items that script produces, do have datetimes for the upcoming releases in the `MM-dd-yyyy` format, and then I have to correct them by hand.
`dateTime.ToString("MM/dd/yyyy")` does produce the date in the format of `MM-dd-yyyy` on my machine. This also happens if I write a corresponding .NET app.
The fix that I am proposing makes it work on my specific setup and hopefully breaks no one else. I understand if you are hesitant to take it. Let me know, I'll see how I can restore my setting.
But on the other hand, I don't think it makes anything worse, it only makes things more robust, so maybe take it?
* Use [CultureInfo]::InvarialtCulture
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Update eng/common/scripts/Prepare-Release.ps1
* Update eng/common/scripts/Prepare-Release.ps1
---------
Co-authored-by: Anton Kolesnyk <41349689+antkmsft@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Remove package retrieval when verify pkg version
* Modified description of PackageName parameter
---------
Co-authored-by: Ray Chen <raychen@microsoft.com>
* Correct the name of JS package folder
* Uncomment the package verification
* Logging more info for troubleshooting
* Get sdkType and directory from the package info
---------
Co-authored-by: Ray Chen <raychen@microsoft.com>
* Support creating resources with user auth
* Log warning if TestApplicationId is set
* missing space
* regenerate md file
* Rename
* Update link
---------
Co-authored-by: jolov <jolov@microsoft.com>
* Add ConflictedFile to git-helpers.ps1, add git-helpers.tests.ps1 to exercise basic functionality.
* Add `resolve-asset-conflict.ps1` a script that can autoresolve an assets.json file.
---------
Co-authored-by: Scott Beddall (from Dev Box) <scbedd@microsoft.com>
Co-authored-by: Scott Beddall <45376673+scbedd@users.noreply.github.com>
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* Added script and pipeline for spec location validation
SDK release pipeline would run this validation to ensure the spec comes from
the main branch of Azure/azure-rest-api-specs repo
* Update parameter
* Use github rest api to validate commit
* Added token parameter
* Support more yaml cases and other languages
* Removed the default setting in yaml template
* Only validate in case of GA package
* Follow APIView to retrieve package version for verification
* Get github token from env variable
* Removed obsolete parameter
---------
Co-authored-by: raychen <raychen@microsoft.com>
* Add package filter parameter to select recursively found packages in release
* Handle empty value for packageFilter
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* check for the presence of a compatible powershell. ensure that we always return a list of tags
* allow the script to require pshell6+
* remove the en-us from the link
---------
Co-authored-by: Scott Beddall (from Dev Box) <scbedd@microsoft.com>
* Update organization of the eng/common/testproxy folder
* Add merge-proxy-tags.ps1 and readme to eng/common/testproxy/scripts/tag-merge/
* Extract common assets script functionality to common-asset-functions.ps1
---------
Co-authored-by: Scott Beddall (from Dev Box) <scbedd@microsoft.com>
Http Status code 401 just be unauthorized and not necessarily that the link isn't valid. Removing this from the list of codes to error from.
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* First cut at snippet generation - markdown files only for now.
* Removed trailing blank lines in updated file
* Use snippet generator for keyvault snippets
* Create new pipeline to create package work item
* Update powershell script to pass tag
* Fixes as per review comments
* Update as per review comments
* Login as az devops
* Create new entry point script to create package work item
* Load common ps module
* Add devops login step
* Pipe login token to devops login
* Pipe login token to devops login
* Add looging
* more logging
* Remove duplicate login from new script and add more logging
* More logging to debug create work item
* Remove script added for debugging
* Skip az interactive login when PAT is passed
* Remove debug logging
---------
Co-authored-by: praveenkuttappan <praveenkuttappan@gmail.com>
* Force capitalize all deployment/env outputs in live test
* Update eng/common/TestResources/New-TestResources.ps1
Co-authored-by: Heath Stewart <heaths@outlook.com>
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
Co-authored-by: Heath Stewart <heaths@outlook.com>
* Add CODEOWNERS linter yml file to eng/common/pipelines
* Add filter baseline option to run command line
* Update version of linter to 1.0.0-dev.20231107.2 which will install correctly
---------
Co-authored-by: James Suplizio <jasupliz@microsoft.com>
* update proxy version to 20231030.1
* update test-proxy-tool.yml to record both stderr and stdout to the proxy log
---------
Co-authored-by: Scott Beddall (from Dev Box) <scbedd@microsoft.com>
* Update branch deletion code
- Update to use the gh cli for github operations
- Switch to using graphql for pulling branches as the rest api
starts to fail after a certain number of branches and we can
also pull back all the associated data like pull request info
and commit date all in one query.
- switc to running on ubuntu 1ES pool
Improve query error handling
Fix condition
Fix slash escaping on the command line
clean-up pipeline
* Update eng/common/scripts/Delete-RemoteBranches.ps1
Co-authored-by: Ben Broderick Phillips <ben@benbp.net>
* Update eng/common/scripts/Delete-RemoteBranches.ps1
Co-authored-by: Ben Broderick Phillips <ben@benbp.net>
---------
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Ben Broderick Phillips <ben@benbp.net>
* Split out curl an winhttp testing on WS2022 to work around CURL bias in test proxy tests
* Minor change to storage to trigger CI pipeline
* On testing builds, send a 0 length Content-Length header on Patch verbs if the caller has not provided a Content-Length header.
* Update sdk/core/azure-core/inc/azure/core/http/http.hpp
Co-authored-by: Anton Kolesnyk <41349689+antkmsft@users.noreply.github.com>
---------
Co-authored-by: Anton Kolesnyk <41349689+antkmsft@users.noreply.github.com>
* clang-13 version of cstdlib doesn't define integer types in global namespace
* Added clang-13 and clang-15 variants to mix
* Pull request feedback
* Update sdk/core/azure-core/inc/azure/core/uuid.hpp
* Update sdk/core/azure-core/inc/azure/core/base64.hpp
---------
Co-authored-by: Larry Osterman <larryo@LOTHAIR-HOME.ntdev.corp.microsoft.com>
* Remove ARM deployment after deploy
* Update eng/common/TestResources/New-TestResources.ps1
Co-authored-by: Heath Stewart <heaths@outlook.com>
* Apply suggestions from code review
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
Co-authored-by: Heath Stewart <heaths@outlook.com>
Fixes https://github.com/Azure/azure-sdk-tools/issues/5882
We need to set the difffilter to empty instead of the default of exclude deleted files when we are trying to verify there are no changes under eng/common.
See test PR https://github.com/Azure/azure-sdk-for-python/pull/32348 which demonstrates us not detecting a deleted file under eng/common. I'll use that same test PR to verify this now catches that issue.
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Remove codeowners resolution logic from docs scripts
* Remove parameters that aren't needed
* Remove auth information from update-docsms-metadata.yml
---------
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
* Add autorest-preview pipeline
* Add emitternpminstall
* Always install
* Use shorter leg name
* Add short circuiting to emitter install
* Use language matrix function
* Don't look for subfolders in the matrix package folders
* Revert unnecessary eng/common changes
* Rewrite GetPullRequestUrl to Get-BuildSourceDescription
* Remove alias from Invoke-LoggedCommand
* Use invoke-expression instead of shelling out
* Add better job splitting
* Replace Folder with Directory
---------
Co-authored-by: Patrick Hallisey <pahallis@microsoft.com>
* Add LockDeletionForDays parameter to set PodDisruptionBudget and cleanup job
* Use matrix for parallel tests. PDB improvements+docs.
* Fix kubectl namespace context preservation on login
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* fixes
* ongoing
* ongoing
* add top-level throw/catches showing exception info
* fix handling of cases when there is 1 link and when there is no RetryAfter.Delta
* handle lack of Exception.Headers property
* handle gracefully obtaining status code from $_.Exception.InnerException.ErrorCode
---------
Co-authored-by: Konrad Jamrozik <kojamroz@microsoft.com>
* Docs Onboarding 2 - Add eng/common changes
* Dictionary Syntax
* Remove $UpdatePackageMetadata, it can be handled in other places
* Set exit code to 0 on success. The last executable to run (pip or docker or other) may be passed through
* Add optional _DocsOnboardingOrdinal sorting to work around onboarding sequence problems
* Update-DocsMsPackages2.ps1 -> Update-DocsMsPackages.ps1
---------
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
* Create Epic work item type as Service or product when running prepare release script
* Updated epic type field name
* Additional change to create release plan work item
* Exclude work items created from release planner test
* Fix issue in params
* Updated as per review comments
---------
Co-authored-by: praveenkuttappan <prmarott@microsoft.com>
The previous property was calculated so didn't exist from the REST-based
call. Also updated the api-version to the latest stable control plane
version.
Co-authored-by: Heath Stewart <heaths@microsoft.com>
* Bump test proxy version
* move proxy transition-scripts folder to onboarding
Co-authored-by: Bill Wert <billwert@microsoft.com>
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* Only save package properties for track 2 packages (prevents overwrites of track 2 package info by track 1 packages)
* Only overwrite if the package is track 2
---------
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
* Update-DocsMsMetadata.ps1 can fail the build on invalid packages
* Better error handling and logging
* Review feedback
---------
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
* Check links in generated documentation
* Another try after talking to Daniel
* More doxygen checks 3
* Disable link guidance because it blocks relative URLs
This adds the ability to do a branch merge as part of our mirroring
pipeline. The merge will allow configuration of which files to
move from source to target so it can be scoped to part of the branch.
Instead of a standard push it will create a PR with the merge so it
can be manually reviewed.
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
* doxygen fixes
* enable doxygen errors now that doxygen is updated.
---------
Co-authored-by: Ahson Khan <ahkha@microsoft.com>
Co-authored-by: Anton Kolesnyk <41349689+antkmsft@users.noreply.github.com>
Co-authored-by: Rick Winter <rick.winter@microsoft.com>
* Upgrade Doxygen to 1.9.7 including template and version selector
* Style and logo changes
* Add README.md for docs templates
* Tabs to spaces
* Show installed doxygen version
* choco list
* "choco" is a word
* Choco list
* Require Doxygen 1.9.7
* Revert "Choco list"
This reverts commit 4d283088771874053f5691d2d464c592dae263a8.
* Revert ""choco" is a word"
This reverts commit 3233cfde8676313efbec23b5a278c8c6b148ac24.
* Revert "choco list"
This reverts commit 107499ba7486bdca7dbbedb8ad9dbb0a683ed2ba.
* Review feedback: Use storage account
* Update README.md
* Support localspecrepo if pass in this parameter
* Support regen sdk code based on local typespecs
* Added reference doc to error message
---------
Co-authored-by: raychen <raychen@microsoft.com>
* Revert the change and return tsplocation folder
* Out null for the new-item command
* exit if return failure when call sync script
* Added repo assignment in local path scenario
* Added more output
* Fix path processing in windows
* exit when run generate script with error
---------
Co-authored-by: raychen <raychen@microsoft.com>
* Adding initial commit of TypeSpecE2E common script
* add TODO comment
* Updated per feedback
---------
Co-authored-by: raychen <raychen@microsoft.com>
* Switch to using standard PAT tokens instead of base 64
For most of these we can use the standard System.AccessToken given to
the build instead of maintaining a specific token. However that
token isn't base 64 encoded so we need to encode it.
With this we can stop explicitly passing PAT's unless we need to
access another DevOps org and we also don't have to remember
to keep the PAT's in KV base 64 encoded.
Add error detection for queue build script to fail if we get login response.
* PR Feedback
---------
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
* First cut at migrating Ubuntu 18 to 22 directly in the matricses
* Try 20.04
* Remove azure.list from apt configuration on 22.04
* Parens
* gpp-8 on Ubuntu 20.04
* Remove g++-5
* Move config changes to steps/
* Remove g++-5 from live tests
* Update eng/pipelines/templates/steps/fix-linux-1es-configs.yml
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* Review feedback
* Finish renaming
* Setting line coverage target to 91%
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* ongoing
* Move get-codeowners scripts and tests to their own dirs
* address PR remarks
* Fix CodeOwnerFileLocation path, fix casing, and dedup param defaults
* fix param names
* add todos on needed changes in cpp repo
* Add CodeownersFileLocation to Get-Codeowners and use $null for default param values
* move get-codeowners back to scripts/ and rename -functions to .lib
* fix: use empty string as defaults instead of $nulls, to fix invocation
* fix bug with invocation of Get-Codeowners + add support for passing IncludeNonUserAliases as switch
* fix path iin Metadata-Helpers.ps1
* fix typo
* Update archetype-cpp-release.yml
---------
Co-authored-by: Konrad Jamrozik <kojamroz@microsoft.com>
* rerun flag
* rerun failed stress test
* naming & commenting
* update
* function and var renaming for better readability
* readability & exit on error
---------
Co-authored-by: Albert Cheng <albertcheng@microsoft.com>
* add some default output to see about minimizing any occurrence of the task failing for no reason. perhaps having some output will allow devops to have an easier time with the invocation
* update message
* Update eng/common/scripts/trust-proxy-certificate.ps1
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* typespec renaming
* add back scripts for cadl
* support .cadl and .tsp
* rename
* add newline at the end of file
---------
Co-authored-by: FAREAST\chunyu <chunyu@microsoft.com>
* FIrst cut at removing proxy tests
Changed the proxy tests from opt-out to opt-in so they're disabled by default. Removed lines enabling proxy from ci.yml file.
* Use "npm ci" to install cspell and respect package-lock.json
* Review feedback
* Pipe npm ci output to Write-Host
---------
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
* we encourage folks to place their assets.jsons at the package level
* update generate-assets-json.ps1 to only include src/**/session-records so as to avoid picking up the duplicated 'target' sessionrecords
---------
Co-authored-by: scbedd <45376673+scbedd@users.noreply.github.com>
* Add todos to update packages to pick up the newest CODEOWNERS interpreter
* update
---------
Co-authored-by: Konrad Jamrozik <kojamroz@microsoft.com>
* Add parameter TimeoutInMinutes
* Update pools and images to latest
* Remove all usage of vmImage
---------
Co-authored-by: Mike Harder <mharder@microsoft.com>
