* updating eng/common/testproxy/test-proxy-tool.yml and eng/common/testproxy/test-proxy-standalone-tool.yml to output a bunch of information about what they're actually running for their commands
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
Co-authored-by: Scott Beddall <45376673+scbedd@users.noreply.github.com>
Co-authored-by: Patrick Hallisey <hallipr@gmail.com>
* add a category to parse triggerPaths when there is no artifact present (this is the go use-case)
* add integration tests for Save-Package-Properties scenarios
* bugfixes around various situations in new PackageProps class
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
We need to use Write-Host for the custom logging commands to get them correctly out into stdout which is what they need to be interpreted correctly.
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* support accessing a different key for the value of the Package during batching in Create-PrJobMatrix
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* Add logic for seting APIView Pull Request comment
* Update script to use source and head commish
* Make API Change Check a header
* Add Fetch URI to APIView comment
* Pass APIView host as a parameter
* Update APIVBiew comment header and add state to search api call
---------
Co-authored-by: Chidozie Ononiwu <chononiw@microsoft.com>
* identify and fix the issue with no results on a matrix that is PRBatched. The platform filters eliminate all the jobs, which is why it only turned up on the second windows leg
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* update generate-matrix function ProcessImport to return a structured result, versus a tuple that is destructured immediately
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
Co-authored-by: Scott Beddall <45376673+scbedd@users.noreply.github.com>
* first stab at a client tsp /tspconfig scripts
* space
* updates to clientTSP
* update for latest specs for KV
* gfd
* gfd
* hghf
* dfgdf
* jhg
* oopsie
* comments
* headers
* clang & param
* coverage
* more comments not gened
* spell
* more comments
* regen for testing
* gfd
* gdf
* tretre
* update to use tsp client
* vcxvx
* fdsfsd
* fds
* round 2
* api view
* dsa
* rte
* update to use code injection
* clang and comments
* gfd
* updates
* clang
* fsd
* spell
* comments
* some cleanup
* first test passing
* dsa
* second test
* startdeletesecret
* update secret ptoeprties
* recover secret
* cleanup clang
* docs
* update recordings and clean up tests
* ggfd
* ooops
* samples build
* gdf
* samples run
* perf tests
* CLANG
* fds
* CLAAAAAAAAANG
* update to latest version of the TSPs
* update to latest codegen
* gdfgdf
* replace method
* client and stuffs
* PR comments
* remove security from path
* clang
* qfe
* gdf
* update to models and to the lastest codegen
* forgot cmake file
* Update sdk/keyvault/azure-security-keyvault-secrets/samples/sample1-basic-operations/sample1_basic_operations.cpp
Co-authored-by: Anton Kolesnyk <41349689+antkmsft@users.noreply.github.com>
* test api view
* dfsfsd
* ccxcvx
* step1
* delete, and lor and paged
* all work
* works and serializers and test for them gone
* cleanup test files
* sample
* tests passing
* dfsfsd
* latest codegen
* after merge
* fsd
* gfd
* gfd
* ApiView
* move to _details namespace
* put back readme
* update readmes
* clang
* dfsd
* get rid of snippets, validation fails without explanation as to what the failure is
* update locations of methods tp cpp files
* format files
* cover
---------
Co-authored-by: Anton Kolesnyk <41349689+antkmsft@users.noreply.github.com>
Add the prefix to identify RGs that we are creating in our TME
tenant to identify them as potentially using local auth and violating
our safe secret standards.
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
* Initial attempt at a Rust AMQP stack, merged against feature/rust_amqp for now. (#5942)
* Enabled building AMQP *without* uAMQP
* Start integrating Rust AMQP Value to C++ AMQP Value
* AMQP Value tests now pass
* Moved AmqpValueType ostream inserter to its original location
* Added Rust AMQP Message implementation
* Added start of message source tests
* Enabled building AMQP *without* uAMQP
* Start integrating Rust AMQP Value to C++ AMQP Value
* Moved AmqpValueType ostream inserter to its original location
* Message target support
* Message source and target support
* Add connection support; restructured tests to fail on RUST AMQP rather than attempting to run; removed some uAMQP-only features (#5986)
* Checkpoint of connection logic
* Started implementing Rust based Connection by pulling out uAMQP artifacts
* Implemented AMQP Connection in Rust; started API surface refactoring for Rust APIs; Refactored tests to remove some uAMQP only elements.
* Don't leak runtime context on calls
* Refactor AMQP logic to better isolate rust AMQP code from uAMQP code. (#6008)
* refactor uAMQP and Rust AMQP into separate implementations for ease of use
* Add connection support; restructured tests to fail on RUST AMQP rather than attempting to run; removed some uAMQP-only features (#5986)
* Checkpoint of connection logic
* Started implementing Rust based Connection by pulling out uAMQP artifacts
* Implemented AMQP Connection in Rust; started API surface refactoring for Rust APIs; Refactored tests to remove some uAMQP only elements.
* Don't leak runtime context on calls
* export UUID from AMQP
* Cleaned up some more elements; reduced scope of doxygen significantly
* runtime context needs to be process global not thread global; all tests pass or fail at this point
* Merged main into branch
* Implement AMQP Session APIs in Rust. (#6033)
* Checkpoint of rust session support
* Session begin/end now works
* Session tests pass
* Removed accidental regression
* Added rust per-call context (#6043)
* Reworked runtime context to be call context
* Added context parameter to blocking AMQP calls
* Added comments around the lifetime of the RustRuntimeContext captured in the CallContext object
* uAMQP changes corresponding to Rust changes
Co-authored-by: Anton Kolesnyk <41349689+antkmsft@users.noreply.github.com>
* sync rust SDK fixes to C++
---------
Co-authored-by: Anton Kolesnyk <41349689+antkmsft@users.noreply.github.com>
* Enable Rust ClaimsBasedSecurity interoperability; Converted EventHubs producer client to work with Rust AMQP changes. (#6059)
* canonicalized return values; initial CBS support
* Implementation of CBS for C++
* Amqp Authentication works; Integrate Rust AMQP into Eventhubs Producer client
* Implemented Rust AMQP message sender. (#6083)
* Initial sender implementation
* PR feedback
* Management APIs work (#6096)
* Management APIs work
---------
Co-authored-by: Anton Kolesnyk <41349689+antkmsft@users.noreply.github.com>
* Converted builders to be compliant with Rust guidelines. (#6102)
* Converted builders to be compliant with Rust guidelines.
* Bring rust changes back to C++ repo
* Improved builder developer experience
* Removed builders from AMQP layer to conform to Rust guidelines; Fixed AMQP bug in message sender tests (#6208)
* Removed builders from AMQP layer to conform to Rust guidelines; Fixed AMQP related bug in message sender tests
* Pr feedback
* Implement receiving messages; Changes to eventhubs so that all eventhubs tests pass (#6254)
* Eventhubs tests pass
* Noise reduction; explain the task which spawns a task
* Update sdk/eventhubs/azure-messaging-eventhubs/src/producer_client.cpp
Co-authored-by: Anton Kolesnyk <41349689+antkmsft@users.noreply.github.com>
* PR feedback
---------
Co-authored-by: Anton Kolesnyk <41349689+antkmsft@users.noreply.github.com>
* Integrate Rust 2 step message receive code to C++ (#6349)
* Integrate Rust 2 step message receive code to C++
* If receiving a delivery failed, transmit the error to the message channel if at all possible
* Enable Rust based AMQP by default (#6362)
* AMQP tests now pass; Integrate TestAmqpBroker with CI pipeline
* Updated changelogs to reflect API changes made during AMQP updates
* Replace uAMQP with Rust AMQP as the default AMQP transport; Updated build configurationj to reflect that
* Test fixes
* PR feedback; test fixes
* Fixed uninitialized variable tanking some tests (#6381)
* Fixed uninitialized variable tanking some tests
* Fixed Rust AMQP tests
* Removed connection string support from Eventhubs and EH tests. (#6391)
* Removed the ability to use connection strings from EventHubs
* Enable Rust AMQP by default!!!
* Update CMakePresets.json
Co-authored-by: Anton Kolesnyk <41349689+antkmsft@users.noreply.github.com>
---------
Co-authored-by: Anton Kolesnyk <41349689+antkmsft@users.noreply.github.com>
* Use PME for esrp release
* Update to EsrpRelease@9
* Remove extra DomainTenantId parameter
* Add PME service connection to pipelines
---------
Co-authored-by: Patrick Hallisey <pahallis@microsoft.com>
* populate triggeringPaths of each artifactdetails with the owning ci.yml
* handle service-directory changes in Get-PrPkgProperties
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* Get-PRPkgProperties now honors artifact-list specific absolute/relative paths to other triggering folders. This will supplant the common "if eng/ changes include template and core" in a follow-up PR.
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* adjustments to iterating across matrix configs for packages that belong to multiple matrices
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* replace PkgProps CIMatrixConfigs property in favor of a more generalized CIParameters property. Original CIMatrixConfigs will exist as an item of same key within CIParameters.
* update Create-PRJobMatrix to honor new CIMatrixConfigs property location
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* update Get-ObjectKey to coalesce boolean-like values to their true boolean value
* update Get-ObjectKey to call Get-ObjectKey on each item of an array or pscustom object - versus utilizing their values directly
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* Pass arm template parameter for TME context
* Use supportsSafeSecretStandard parameter instead
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* allow the indirect matrix to generate non-sparse based on new parameter to generate-job-matrix
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* Added label handle sdk-gen pipeline template
Added common script to delete label from a PR
* Update eng/common/scripts/Invoke-GitHubAPI.ps1
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
---------
Co-authored-by: ray chen <raychen@microsoft.com>
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* Support incrementing semver prereleases with 'zero' versions
* Make tests more explicit
---------
Co-authored-by: Patrick Hallisey <pahallis@microsoft.com>
* updates to dynamic matrix generation. moves indirectly included packages to sparse usage. adds additional parameters to GeneratePRMatrix that enable further filtering of the matrix generated for the indirect packages
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Use -Mail for test resources local user lookup to support TME
* Auto-select TME subscription based on user context
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* ensure that packages that come back from AdditionalValidationPackages are NOT counted as 'includedForValidation: true' if they also exist in the original 'these packages changed' set
Co-authored-by: Ben Broderick Phillips <ben@benbp.net>
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
Co-authored-by: Scott Beddall <45376673+scbedd@users.noreply.github.com>
Co-authored-by: Ben Broderick Phillips <ben@benbp.net>
* add and use new function to PSModule-Helpers.ps1 -- `InstallAndImport-ModuleIfNotInstalled` which smartly short circuits the install and module import to be as efficient as possible. This largely has impact of improving Save-Package-Properties performance on windows machines.
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* Use new generation tool that supports service connections
* Fix condition for generating weekly test pipelines
---------
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* change our verifyChangelogs over to utilizing the artifact details. if we don't have an artifact details at all, don't verify changelog. if we do, but skipverifychangelog is present, don't verify changelog. if we have artifact details, but no disable, verify the changelog
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* ensure that ChangedServices value within the generated pr diff is always properly an array of strings
* upload the changed services as an attachment named AdditionalTags in save-package-properties.yml, to allow PipelineWitness to tag public builds during log processing
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
Co-authored-by: Scott Beddall <45376673+scbedd@users.noreply.github.com>
Co-authored-by: Patrick Hallisey <hallipr@gmail.com>
* Add project skeleton and check-in snapshot of generated SDK for Azure App Configuration.
* Add new line at eof
* Add exclusion to the cspell issue.
* Explicitly use size_t instead of auto for iterating through a vector
* Add another cspell exclusion, since it is case-sensitive
* Address PR feedback, update CL, package version.
* Update readme to remove template project content.
* Exclude appconfig from recordings, since none exist atm.
* Temporarily exclude appconfiguration from docs generation.
* Add line/branch coverage thresholds and skip publishing docs if they don't exist.
* Add step for emitting rate limits for a token
* add step to eng/common sync template
* Fix resource label output
* Use propery name instead of nested 'resource' property
* Add percent metric too
* Add divide by zero safety
---------
Co-authored-by: Patrick Hallisey <pahallis@microsoft.com>
* Common cleanup with the docs docker images no longer being used
* Remove docker-pull-image.yml
---------
Co-authored-by: James Suplizio <jasupliz@microsoft.com>
* Retry container deletion
* Do not try to purge keyvaults with purge protection
* Delete all blobs when container has immutability
* Skip missing blob container properties
* Fix null container
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* identify and resolve missing function GenerateMatrixForConfig
* identify and resolve miss-used null-coalesce operator
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* package-properties are now populated with matrix configurations from their ci.yml if present
* create new code path for generate-job-matrix.yml which combines Create-JobMatrix and the "distribute-packages-to-matrix" action to generate dynamic matrices for PRs
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* Run perf tests under federated auth
Run the tests under the same federated auth used to deploy the tests and setup the variables needed to configure AzurePipelineCredential.
* Fix indention for working directory
---------
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Refactor PackageProps to make the Yaml loading and processing methods common
* Updates for feedback
---------
Co-authored-by: James Suplizio <jasupliz@microsoft.com>
* Remove default sub config sub-config-azure-cloud-test-resources
We no longer want to default the configuration and instead
default from the service connection which had the info we need.
* Fix quoting
---------
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
* save-package-properties.yml only activates PR diff mechanism if the service directory is "auto"
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* Escape parentheses in includes to fix issues in publishing
* Only escape close parens
* Comments
---------
Co-authored-by: Daniel Jurek <djurek@microsoft.com>
* Changes required for sdk-repo based emitter pipelines
* Rename template
* Convert emitter archetype to pipeline template
* Move the archetype template out of the stages folder
---------
Co-authored-by: Patrick Hallisey <pahallis@microsoft.com>
* compact the macros
copy generated files
* some comments
* Cleanup and update to the latest SHAs
* sdffds
* working hybrid
* cleanup
* clang
* Update cmake-modules/TSPCompile.cmake
Co-authored-by: Rick Winter <rick.winter@microsoft.com>
---------
Co-authored-by: Rick Winter <rick.winter@microsoft.com>
* Add CompatibleConvertFrom-Yaml to common Package-Helpers
* Add CI artifact parsing to Save-Package-Properties, now each individual packageinfo.json file contains the relevant ci artifact lines for the package. This short circuits needing to re-parse this information in common checks.
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* Export the subscription data from the service connection
* Update deploy-test-resources.yml
---------
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Use code snippets from managed identity credential samples in identity readme doc.
* Generate readme from snippets.
* Update snippet generation script to remove unnecessary indentation and extra new line at eof.
* Update Snippet Generation doc to show a concrete example on how to run it.
* Update other repo READMEs with the generation fixes.
* Fix KeyVault Secrets sample and use the snippets in its README
* Use the added sample snippet.
* don't verify changelog for packages that aren't actively published anymore
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
Co-authored-by: Scott Beddall <45376673+scbedd@users.noreply.github.com>
Co-authored-by: Ben Broderick Phillips <ben@benbp.net>
* updating package properties with direct/indirect (if named differently) as well as pulling BuildDocs from ci.yml artifact list if it exists
* eliminate the addition of buildDocs property. it requires powershell-yaml to be present on our base function. not good
* remove call to InitializeBuildDocs
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* Add descriptive error when variable groups are not added to pipeline
* Add emoji to deploy step title to make it easier to find
* Move yaml json checking into powershell script
* Fix base sub config empty/string checking
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
* Refactor prepare-pipelines
* Convert pipeling generation single to use template
* Convert to pwsh for setting the variables
* Fix switch expression
* Add path separator in front of devopspath
* Continue on error for individual step failures
* Remove unnecessary parameters
---------
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Apply a few changes to Get-PRPackageProperties to make it more customizable for our users. Remove wonky gap + extra spacing that should pass the patch application process now that we've manually aligned common.ps1
* account for feedback -- rename CanaryPackages to AdditionalValidationPackages
* dependentPackagesforInclusion -> additionalValidationPackages
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* bump proxy version
* bump proxy version
* restore most of the old performance, only fall back to json matching when something goes wrong
* use newest version with protection from invalid json
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
Given we cannot create subfolders under .github\workflow folder
and we have repos that want additional workflows then we have
in the tools repo we need to setup a file pattern matching for
the sync processing to only sync the ones we want to be common.
For now just use the existing *event* pattern but we can move them
to another pattern similar to *common* or *sync* in the future as
we add more common workflows.
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
* Add RestAPISpecsDocsRepos to branch cleanup.
* Remove commented out yml, add pre-message to limit output and output core limit at the start of processing
* updates for feedback
---------
Co-authored-by: James Suplizio <jasupliz@microsoft.com>
* we want the path that we write to to be defined by literalpath, otherwise files with [ or ] will break powershell when writing the file to disk
* slight wording adjustment so the command can be invoked directly
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
* more compatibility with expanding/contracting packages. add ability for packages to have DependentPackages that must be included in the set of packages that should be built given a changeset
---------
Co-authored-by: Scott Beddall <scbedd@microsoft.com>
Co-authored-by: Scott Beddall <45376673+scbedd@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* Do not mark values as secret that are from git-hosted sub config files
* Simplify sub config build yaml
---------
Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>
