* Fix sparse-checkout git command line behavior
* Use command arg parsing behavior in sparse-checkout.yml
---------
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
* Prepare-Release.ps1: Make dateTime.ToString("MM/dd/yyyy") to work on exotic set-ups
On my machine, I experimented with the registry, and the worst part is that I don't remember/know how to reset it back.
The work items that script produces, do have datetimes for the upcoming releases in the `MM-dd-yyyy` format, and then I have to correct them by hand.
`dateTime.ToString("MM/dd/yyyy")` does produce the date in the format of `MM-dd-yyyy` on my machine. This also happens if I write a corresponding .NET app.
The fix that I am proposing makes it work on my specific setup and hopefully breaks no one else. I understand if you are hesitant to take it. Let me know, I'll see how I can restore my setting.
But on the other hand, I don't think it makes anything worse, it only makes things more robust, so maybe take it?
* Use [CultureInfo]::InvarialtCulture
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Update eng/common/scripts/Prepare-Release.ps1
* Update eng/common/scripts/Prepare-Release.ps1
---------
Co-authored-by: Anton Kolesnyk <41349689+antkmsft@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Remove package retrieval when verify pkg version
* Modified description of PackageName parameter
---------
Co-authored-by: Ray Chen <raychen@microsoft.com>
* Correct the name of JS package folder
* Uncomment the package verification
* Logging more info for troubleshooting
* Get sdkType and directory from the package info
---------
Co-authored-by: Ray Chen <raychen@microsoft.com>
* Support creating resources with user auth
* Log warning if TestApplicationId is set
* missing space
* regenerate md file
* Rename
* Update link
---------
Co-authored-by: jolov <jolov@microsoft.com>
* Add ConflictedFile to git-helpers.ps1, add git-helpers.tests.ps1 to exercise basic functionality.
* Add `resolve-asset-conflict.ps1` a script that can autoresolve an assets.json file.
---------
Co-authored-by: Scott Beddall (from Dev Box) <scbedd@microsoft.com>
Co-authored-by: Scott Beddall <45376673+scbedd@users.noreply.github.com>
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* Added script and pipeline for spec location validation
SDK release pipeline would run this validation to ensure the spec comes from
the main branch of Azure/azure-rest-api-specs repo
* Update parameter
* Use github rest api to validate commit
* Added token parameter
* Support more yaml cases and other languages
* Removed the default setting in yaml template
* Only validate in case of GA package
* Follow APIView to retrieve package version for verification
* Get github token from env variable
* Removed obsolete parameter
---------
Co-authored-by: raychen <raychen@microsoft.com>
* Add package filter parameter to select recursively found packages in release
* Handle empty value for packageFilter
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* check for the presence of a compatible powershell. ensure that we always return a list of tags
* allow the script to require pshell6+
* remove the en-us from the link
---------
Co-authored-by: Scott Beddall (from Dev Box) <scbedd@microsoft.com>
* Update organization of the eng/common/testproxy folder
* Add merge-proxy-tags.ps1 and readme to eng/common/testproxy/scripts/tag-merge/
* Extract common assets script functionality to common-asset-functions.ps1
---------
Co-authored-by: Scott Beddall (from Dev Box) <scbedd@microsoft.com>
Http Status code 401 just be unauthorized and not necessarily that the link isn't valid. Removing this from the list of codes to error from.
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Create new pipeline to create package work item
* Update powershell script to pass tag
* Fixes as per review comments
* Update as per review comments
* Login as az devops
* Create new entry point script to create package work item
* Load common ps module
* Add devops login step
* Pipe login token to devops login
* Pipe login token to devops login
* Add looging
* more logging
* Remove duplicate login from new script and add more logging
* More logging to debug create work item
* Remove script added for debugging
* Skip az interactive login when PAT is passed
* Remove debug logging
---------
Co-authored-by: praveenkuttappan <praveenkuttappan@gmail.com>
* Force capitalize all deployment/env outputs in live test
* Update eng/common/TestResources/New-TestResources.ps1
Co-authored-by: Heath Stewart <heaths@outlook.com>
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
Co-authored-by: Heath Stewart <heaths@outlook.com>
* Add CODEOWNERS linter yml file to eng/common/pipelines
* Add filter baseline option to run command line
* Update version of linter to 1.0.0-dev.20231107.2 which will install correctly
---------
Co-authored-by: James Suplizio <jasupliz@microsoft.com>
* update proxy version to 20231030.1
* update test-proxy-tool.yml to record both stderr and stdout to the proxy log
---------
Co-authored-by: Scott Beddall (from Dev Box) <scbedd@microsoft.com>
* Update branch deletion code
- Update to use the gh cli for github operations
- Switch to using graphql for pulling branches as the rest api
starts to fail after a certain number of branches and we can
also pull back all the associated data like pull request info
and commit date all in one query.
- switc to running on ubuntu 1ES pool
Improve query error handling
Fix condition
Fix slash escaping on the command line
clean-up pipeline
* Update eng/common/scripts/Delete-RemoteBranches.ps1
Co-authored-by: Ben Broderick Phillips <ben@benbp.net>
* Update eng/common/scripts/Delete-RemoteBranches.ps1
Co-authored-by: Ben Broderick Phillips <ben@benbp.net>
---------
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Ben Broderick Phillips <ben@benbp.net>
* Remove ARM deployment after deploy
* Update eng/common/TestResources/New-TestResources.ps1
Co-authored-by: Heath Stewart <heaths@outlook.com>
* Apply suggestions from code review
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
Co-authored-by: Heath Stewart <heaths@outlook.com>
Fixes https://github.com/Azure/azure-sdk-tools/issues/5882
We need to set the difffilter to empty instead of the default of exclude deleted files when we are trying to verify there are no changes under eng/common.
See test PR https://github.com/Azure/azure-sdk-for-python/pull/32348 which demonstrates us not detecting a deleted file under eng/common. I'll use that same test PR to verify this now catches that issue.
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Remove codeowners resolution logic from docs scripts
* Remove parameters that aren't needed
* Remove auth information from update-docsms-metadata.yml
---------
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
* Add autorest-preview pipeline
* Add emitternpminstall
* Always install
* Use shorter leg name
* Add short circuiting to emitter install
* Use language matrix function
* Don't look for subfolders in the matrix package folders
* Revert unnecessary eng/common changes
* Rewrite GetPullRequestUrl to Get-BuildSourceDescription
* Remove alias from Invoke-LoggedCommand
* Use invoke-expression instead of shelling out
* Add better job splitting
* Replace Folder with Directory
---------
Co-authored-by: Patrick Hallisey <pahallis@microsoft.com>
* Add LockDeletionForDays parameter to set PodDisruptionBudget and cleanup job
* Use matrix for parallel tests. PDB improvements+docs.
* Fix kubectl namespace context preservation on login
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* fixes
* ongoing
* ongoing
* add top-level throw/catches showing exception info
* fix handling of cases when there is 1 link and when there is no RetryAfter.Delta
* handle lack of Exception.Headers property
* handle gracefully obtaining status code from $_.Exception.InnerException.ErrorCode
---------
Co-authored-by: Konrad Jamrozik <kojamroz@microsoft.com>
* Docs Onboarding 2 - Add eng/common changes
* Dictionary Syntax
* Remove $UpdatePackageMetadata, it can be handled in other places
* Set exit code to 0 on success. The last executable to run (pip or docker or other) may be passed through
* Add optional _DocsOnboardingOrdinal sorting to work around onboarding sequence problems
* Update-DocsMsPackages2.ps1 -> Update-DocsMsPackages.ps1
---------
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
* Create Epic work item type as Service or product when running prepare release script
* Updated epic type field name
* Additional change to create release plan work item
* Exclude work items created from release planner test
* Fix issue in params
* Updated as per review comments
---------
Co-authored-by: praveenkuttappan <prmarott@microsoft.com>
The previous property was calculated so didn't exist from the REST-based
call. Also updated the api-version to the latest stable control plane
version.
Co-authored-by: Heath Stewart <heaths@microsoft.com>
* Bump test proxy version
* move proxy transition-scripts folder to onboarding
Co-authored-by: Bill Wert <billwert@microsoft.com>
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* Only save package properties for track 2 packages (prevents overwrites of track 2 package info by track 1 packages)
* Only overwrite if the package is track 2
---------
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
* Update-DocsMsMetadata.ps1 can fail the build on invalid packages
* Better error handling and logging
* Review feedback
---------
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
This adds the ability to do a branch merge as part of our mirroring
pipeline. The merge will allow configuration of which files to
move from source to target so it can be scoped to part of the branch.
Instead of a standard push it will create a PR with the merge so it
can be manually reviewed.
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
* Support localspecrepo if pass in this parameter
* Support regen sdk code based on local typespecs
* Added reference doc to error message
---------
Co-authored-by: raychen <raychen@microsoft.com>
* Revert the change and return tsplocation folder
* Out null for the new-item command
* exit if return failure when call sync script
* Added repo assignment in local path scenario
* Added more output
* Fix path processing in windows
* exit when run generate script with error
---------
Co-authored-by: raychen <raychen@microsoft.com>
* Adding initial commit of TypeSpecE2E common script
* add TODO comment
* Updated per feedback
---------
Co-authored-by: raychen <raychen@microsoft.com>
* Switch to using standard PAT tokens instead of base 64
For most of these we can use the standard System.AccessToken given to
the build instead of maintaining a specific token. However that
token isn't base 64 encoded so we need to encode it.
With this we can stop explicitly passing PAT's unless we need to
access another DevOps org and we also don't have to remember
to keep the PAT's in KV base 64 encoded.
Add error detection for queue build script to fail if we get login response.
* PR Feedback
---------
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
* ongoing
* Move get-codeowners scripts and tests to their own dirs
* address PR remarks
* Fix CodeOwnerFileLocation path, fix casing, and dedup param defaults
* fix param names
* add todos on needed changes in cpp repo
* Add CodeownersFileLocation to Get-Codeowners and use $null for default param values
* move get-codeowners back to scripts/ and rename -functions to .lib
* fix: use empty string as defaults instead of $nulls, to fix invocation
* fix bug with invocation of Get-Codeowners + add support for passing IncludeNonUserAliases as switch
* fix path iin Metadata-Helpers.ps1
* fix typo
* Update archetype-cpp-release.yml
---------
Co-authored-by: Konrad Jamrozik <kojamroz@microsoft.com>
* rerun flag
* rerun failed stress test
* naming & commenting
* update
* function and var renaming for better readability
* readability & exit on error
---------
Co-authored-by: Albert Cheng <albertcheng@microsoft.com>
* add some default output to see about minimizing any occurrence of the task failing for no reason. perhaps having some output will allow devops to have an easier time with the invocation
* update message
* Update eng/common/scripts/trust-proxy-certificate.ps1
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* typespec renaming
* add back scripts for cadl
* support .cadl and .tsp
* rename
* add newline at the end of file
---------
Co-authored-by: FAREAST\chunyu <chunyu@microsoft.com>
* Use "npm ci" to install cspell and respect package-lock.json
* Review feedback
* Pipe npm ci output to Write-Host
---------
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
* we encourage folks to place their assets.jsons at the package level
* update generate-assets-json.ps1 to only include src/**/session-records so as to avoid picking up the duplicated 'target' sessionrecords
---------
Co-authored-by: scbedd <45376673+scbedd@users.noreply.github.com>
* Add todos to update packages to pick up the newest CODEOWNERS interpreter
* update
---------
Co-authored-by: Konrad Jamrozik <kojamroz@microsoft.com>
* Add parameter TimeoutInMinutes
* Update pools and images to latest
* Remove all usage of vmImage
---------
Co-authored-by: Mike Harder <mharder@microsoft.com>
* Remove language from profile zip
* Revert "Remove language from profile zip"
This reverts commit 2f670435b25f89955338170a7bdfbe4432cbe23f.
* Remove Services parameter
* Fix path to tests-file
* Run both sync and async tests by default
* Add LanguageVersion to common perf.yml
---------
Co-authored-by: Mike Harder <mharder@microsoft.com>
* Move migration script to eng/common
* Remove localization from link
* Ignore the improper link
---------
Co-authored-by: Timo van Veenendaal <timov@microsoft.com>
* add documentation for cadl sync and generate
* move the file to doc/common and link to it from each script
* update links after file rename
---------
Co-authored-by: m-nash <prognash@gmail.com>
* move scripts to common
* add reference to common script
* address feedback
* couple tweaks to relative path updates
* update copy paste issue with emitter path
* updates to make scripts more generic
---------
Co-authored-by: m-nash <prognash@gmail.com>
* Consolidate naming logic and generate short hash names for local use
* Shorten long lines
* Handle issues with EnvironmentVariable parameter ref being updated
* Warn on env variable overwrite. Base name generation off resource group
* Use SHA256 algorithm for short name hash
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* feature/upload-proxy-artifact
* update template to ensure that having the file in use on windows doesn't break the build
* incoming PR fixes where test-proxy stores its log. as as result, we also need to update the location of where we read out the test-proxy log
Co-authored-by: scbedd <45376673+scbedd@users.noreply.github.com>
* Pass profile flag
* Fix conditional add of '--profile'
* One more attempt at how to enable profile
* Don't publish everything
* Move profile parameter after iterations
* Pass profile parameter as flag option
* Create profile variable
Co-authored-by: Alan Zimmer <48699787+alzimmermsft@users.noreply.github.com>
Co-authored-by: Mike Harder <mharder@microsoft.com>
With new PS command line parsing update in 7.3 they explicitly
pass "" to the native commands which in our usage here we don't
want. However setting the variable to null instead works for our
scenario with both command line parsing strategies so using it here.
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
* Add new check in APIView to verify and enforce package name approval
* Changes as per review feedback
* Apply suggestions from code review
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: praveenkuttappan <praveen.kuttappan@gmail.com>
Co-authored-by: Praven Kuttappan <55455725+praveenkuttappan@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Make the env variable upper case
* Adding debug messages.
* Log out the link
Co-authored-by: sima-zhu <sizhu@microsoft.com>
Co-authored-by: Sima Zhu <48036328+sima-zhu@users.noreply.github.com>
* Service readme index table stays the same even the table content needs to update
* adjust the edge cases tests
* change the helper function to helper folder
* add missing files
* Fixed the service name
Co-authored-by: sima-zhu <sizhu@microsoft.com>
* Fix contains method in stress generate function
* Update stress examples to addons 0.2.0. Script QOL fixes.
* Skip CI print for stress matrix. Minor improvements.
* Fix non-matching Chart.yaml files being evaluated too soon
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* Added yaml support for job matrix creation
* autogen scenario matrix for stress test
* Temporary Working State
* update to default sparse
* pr comments and some error handling
* custom matrixfilename and ordering of generatedValues.yaml
* common module import
* JobMatrix write host
Co-authored-by: Albert Cheng <albertcheng@microsoft.com>
We have hit a few 50x errors while pulling a page to get
links and that has caused the rest of the link checking to
terminate early. To fix that we switching to LogError which will
output an error in devops instead of Write-Error which terminates
immediately. We also add some retry count to the page retrieval
and cache file retrieval to help with these transitent 50x issues.
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
This should fix https://github.com/Azure/azure-sdk-tools/issues/3538
as it will now upgrade the devops extension to the latest version
which has the auth fix needed.
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
* Detect API changes using new snadboxinx approach
* Added strictmode
* Remove unset variable
* Changes as per strict mode 3
* Rervert strict mode to allow language level fixes to merge first
Co-authored-by: praveenkuttappan <prmarott@microsoft.com>
* Add full clone fallback to sparse checkout
* Improve clone handling and overrides for sparse checkout
* Use SkipSparseCheckout variable name
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* someone wants to reference the test-proxy startup scripts externally. to make this easy on them we're parameterizing the root of the eng/common for easy use in that scenario
* two leading $ signs on the definition of the certificate path was causing some issues!
Co-authored-by: scbedd <45376673+scbedd@users.noreply.github.com>
* Fixed the common contract.
* Fix for current regex
* Add new line
* Update eng/common/scripts/Service-Level-Readme-Automation.ps1
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* move compare and merge into gerenerate function
* Fix on new lines
Co-authored-by: sima-zhu <sizhu@microsoft.com>
Co-authored-by: Sima Zhu <48036328+sima-zhu@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* stress test script and docs migration from test to pg cluster
* comments for stress-test-deployment-lib script
Co-authored-by: Albert Cheng <albertcheng@microsoft.com>
* Update verify-readme to take a single or multiple paths
* chance ScanPaths to a comma delimited list and coalesce ScanPath/ScanPaths into a single variable for the script
* VS Code was nice enough to add an extra single quote when adding a quote to the end of the line'
* Capture ScanPaths.Split into an array so we don't have to call Split again
Co-authored-by: James Suplizio <jasupliz@microsoft.com>
We had continueOnError set to true so if the emulator failed to start we would still try to run the tests which all fail but take a long time to run. The timeout for this step is about 10mins of trying so that should be enough time and so we also don't really need another retry step.
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Support local addons path override in stress test deployment
* Support username based deployId in local stress deployment
* Support WhatIf in stress infrastructure provision script
* Simplify stress user detection
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Run helm plugin add with helper
* Add WhatIf support to ps module install helper function
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Update the order of remarks and examples to align with docs.ms
* change all occurance
* Update class.tmpl.partial
Co-authored-by: sizhu <sizhu@microsoft.com>
Co-authored-by: Sima Zhu <48036328+sima-zhu@users.noreply.github.com>
* Use seperate scripts
* address comments.
* do compare and update
* save on the service level readme
* have the helper for reuse function
* remove mgmt table
* changes
* fix
* no return on error
* return if no contents
* Address comments
* change the table
* address wes comments.
* address wes comments.
* address more comments.
Co-authored-by: sima-zhu <sizhu@microsoft.com>
