Commit Graph

2057 Commits

Author SHA1 Message Date
Anton Kolesnyk
d32aab96eb
Acknowledge community contributions in changelogs (#5850)
* Acknowledge cusomer contributions

* Update cspell

* Add more contributors

* More cspell

* Update contributions

---------

Co-authored-by: Anton Kolesnyk <antkmsft@users.noreply.github.com>
2024-07-31 13:44:28 -07:00
Robert Schulze
401b97e0ac
Remove unneeded <regex> includes (#5851)
* Remove unneeded <regex> includes

* Fix build (hopefully)
2024-07-31 13:07:47 -07:00
George Arama
ec6a66d53e
Keyvault full backup/restore support (#5744)
* first build pass

* move models

Refactor KeyVault backup/restore client

This commit represents a significant refactoring of the Azure Security KeyVault Administration client library, transitioning from a generic `KeyVaultClient` to a more specialized `BackupRestoreClient`. The refactoring includes:

- Renaming `KeyVaultClient` to `BackupRestoreClient` in both `backup_restore_client.hpp` and `backup_restore_client.cpp` to better reflect the focused functionality on backup and restore operations.
- Moving model definitions from `backup_restore_client.hpp` to `rest_client_models.hpp`, including structures for errors, full backup, restore, and selective key restore operations, along with their options. This change centralizes model definitions for improved maintainability and reuse across the client library.
- Updating method implementations in `backup_restore_client.cpp` to align with the `BackupRestoreClient` class name and its specialized backup and restore operations.

These changes aim to provide clearer separation of concerns within the library, making it more manageable and focused on backup and restore functionalities for Azure KeyVault.

* Add BackupRestoreClient with tests and options

This commit introduces the BackupRestoreClient for Azure Key Vault backup and restore operations, along with BackupRestoreClientOptions for configuration. Significant refactoring within the Azure::Security::KeyVault::Administration namespace simplifies namespace usage by directly using `using namespace Models;`. The BackupRestoreClient has been enhanced with a new constructor accepting `vaultUrl`, a `TokenCredential` shared pointer, and optionally `BackupRestoreClientOptions`. Additionally, a copy constructor and a default destructor have been added, alongside a private member `m_apiVersion` for API version management.

The BackupRestoreClientOptions struct extends Azure::Core::_internal::ClientOptions with an `ApiVersion` member initialized to "7.5". Changes in the implementation file include additional headers for internal use and authentication, and updates to the constructor for initializing member variables and setting up the HTTP pipeline.

Test infrastructure has been enhanced with modifications in `CMakeLists.txt` to include test source files, the introduction of a base test class `BackupRestoreClientTest` for setup, and a test case `CreateClient1` demonstrating client creation.

These changes collectively enhance the Azure Key Vault Administration library by adding a new client for backup and restore operations, improving code organization, and setting up a testing framework.

* Enhance BackupRestoreClient and tests

This commit significantly updates the BackupRestoreClient and its associated models, introducing new features for backup and restore operations, and improving error handling with a new Error model. The `BackupRestoreClient` class has been marked as `final` to prevent inheritance, and several new methods such as `FullBackup`, `FullBackupStatus`, `FullRestore`, `RestoreStatus`, and `SelectiveKeyRestore` have been added, along with corresponding data models in `rest_client_models.hpp`. These changes enhance the client's functionality and API clarity. Additionally, the test infrastructure has seen substantial improvements, including refactoring in `backup_restore_client_base_test.hpp` for better encapsulation and updated utility methods for more effective testing. The practical application of these enhancements is demonstrated in `backup_restore_client_test.cpp` through the `CreateClient1` test case, showcasing the use of the `FullBackup` method. Overall, these updates bolster the Azure Key Vault Backup and Restore client library's capabilities and test support.

* Enhance BackupRestoreClient for Azure backups

- Updated `BackupRestoreClient` to support full backups and restores using Azure blob storage SAS tokens, including changes to method signatures for SAS token and job identifier parameters.
- Introduced nullable types for `StatusDetails` and `Error` in operation models to handle optional response fields.
- Removed `FullBackupOptions`, `FullRestoreOperationOptions`, and `RestoreStatusOptions` structures in favor of direct parameter usage in methods.
- Refined HTTP request construction to include necessary headers and dynamically set API version.
- Updated JSON parsing logic to handle nullable fields and adapt to changed response JSON structure.
- Enhanced test infrastructure with methods for creating test-configured `BackupRestoreClient` and generating SAS tokens for backups. Added new tests for full backup, backup status, full restore, and restore status functionalities.
- Adjusted URL path construction and query parameter setting in HTTP requests for updated API endpoints and parameters compatibility.

* Refine SelectiveKeyRestore API and tests

This commit overhauls the SelectiveKeyRestore function in the Azure Key Vault Backup and Restore client library. The function signature in `backup_restore_client.hpp` has been updated to replace the `SelectiveKeyRestoreOperationOptions` parameter with two distinct parameters: `keyName` (std::string) and `restoreBlobDetails` (SelectiveKeyRestoreOperationParameters). This change enhances the API's clarity and usability by explicitly requiring the key name and restore blob details for the operation.

Additionally, the `SelectiveKeyRestoreOperationOptions` structure has been removed from `rest_client_models.hpp`, aligning with the shift towards using direct parameters for the selective key restore operation. Corresponding implementation adjustments in `backup_restore_client.cpp` include changes to request URL and JSON body construction, as well as improved error handling in response parsing.

The test suite in `backup_restore_client_test.cpp` has also been updated. The `CreateClient1` test case was removed, and a new test case, `RestoreSelectiveStatus`, was added to demonstrate the updated API's functionality through a full backup and selective key restore operation scenario.

These changes collectively improve the API's intuitiveness and demonstrate its practical application through updated test cases.

* oops

* fds

* negative tests

Enhance backup/restore tests and cleanups

- Added a new test case `BackupFullErrorStatus` to the `BackupRestoreClientTest` suite to simulate a full backup operation with an invalid URI and verify the transition of backup status from "InProgress" to "Failed" with detailed error status.
- Refined the `RestoreFull` test case by removing unnecessary line breaks, improving code formatting without affecting functionality.
- Adjusted formatting in the `RestoreFullStatus` test by removing unnecessary braces and line breaks, streamlining the code.
- Improved readability in the `RestoreSelectiveStatus` test by adding a space after a comma.
- Introduced a new test case `RestoreSelectiveInvalidKeyStatus` to test the system's handling of invalid key restore requests, ensuring error reporting is accurate when keys or their versions are not found.

* recordings and pipeline

* linux build issues

* comment

* parse the error type

* fix paths

* ewrwe

* fdss

* reencode ansi

* fdsfsd

* Enhance samples and refine project structure

- Adjusted `CMakeLists.txt` for better project clarity, moving `BUILD_SAMPLES_HSM` from `test/samples` to `samples` and adding new sample directories for backup and restore functionalities.
- Updated `sample1_administration.cpp` to use `Azure::Core::_internal::Environment::GetVariable` for environment variable access and removed settings management code to focus on specific functionalities.
- Added new sample applications `sample2_full_backup_restore.cpp` and `sample3_backup_selective_restore.cpp` demonstrating full and selective backup and restore operations using the Azure Key Vault administration client library, including detailed steps and error handling.
- Included `CMakeLists.txt` for each new sample, specifying C++ standards and linking necessary libraries, with a focus on ease of use and configuration through the inclusion of `get-env-helper`.

* restore sample 1

* attempt reformat doc

* gfddgdf

* attempt 2

* are you serious ?

* treter

* docs

* typo

* changelog

* doc

* KHlAAAAAAAAAAANg said Kirk

* error check

* Updates to API

* clangs

* test new macro use in cmake file

* clang again

* missing include

* update1

* LROs

* clang 11

* attempt2

* fhkwerjhfkj

* documentation

* attempt++

* separating the op in its own files

* clang

* docs ++

* docc +++

* clang

* typo

* slkjlkdsjflks

* rename to backupclient

* hoping it's building

* PR comments

* address magic param check

* assets update
2024-07-31 10:29:41 -07:00
Robert Schulze
6960ad1247
XML: Use RAII wrappers instead of manual memory management (#5767)
Co-authored-by: alesapin <alesapin@gmail.com>
Co-authored-by: Anton Popov <anton@clickhouse.com>
2024-07-30 09:48:48 +08:00
Anton Kolesnyk
0f62b3aa63
Add consumption-time value of __cplusplus to User-Agent (#5662)
* Add consumption-time value of __cplusplus to User-Agent

---------

Co-authored-by: Anton Kolesnyk <antkmsft@users.noreply.github.com>
2024-07-29 18:31:51 -07:00
arwell
46757b69b1
Fix exception when an int32 value comes from the json (#5781)
* Fix exception when an int32 value comes from the json

* Update serializers.cpp

Fix Pipeline errors

* Format using clang

* add more tests

---------

Co-authored-by: George Arama <gearama@microsoft.com>
2024-07-29 12:50:34 -07:00
microzchang
175a761dbb
Storage Fix live test failure (#5836) 2024-07-23 14:48:35 +08:00
Robert Schulze
6c9af55bae
Fix libcxx15 build (#5765) 2024-07-23 12:19:55 +08:00
Azure SDK Bot
cce5fc98a6
Increment package version after release of azure-identity (#5835) 2024-07-22 20:26:28 -07:00
Ahson Khan
b1674473eb
Prepare for the July Identity beta release. (#5770)
* Prepare for the July Identity beta release.

* Update release date.

* Update CL - remove empty headings.
2024-07-22 19:06:04 -07:00
Ahson Khan
193bbe6963
Add support for providing a Resource ID to ManagedIdentityCredential. (#5832)
* Add support for providing a Resource ID to ManagedIdentityCredential.

* Fix typo in param name.

* Fix clang format and undo removing of private methods (that are unused).

* Address PR feedback.

* Add back Azure:: to fix doxygen docs issue.
2024-07-22 19:00:13 -07:00
Ahson Khan
00304a0556
Add support for customizing the IMDS endpoint within ManagedIdentityCredential using an env var. (#5834)
* Add support for customizing the IMDS endpoint within
ManagedIdentityCredential using an env var.

* Clean up the impl.

* Add imds as an opt-out for spell checl.

* Address PR feedback.
2024-07-22 18:27:16 -07:00
JinmingHu
4dd4b18af9
UseFederatedAuth storage test (#5810) 2024-07-20 02:00:18 +00:00
Larry Osterman
ab777719ea
Fix credentials used for eventhubs admin tests. Fixes #5820 (#5825)
* Log credential creation


* Fail on exception after dumping it
2024-07-19 13:44:04 -07:00
Scott Addie
4b8d7a05e0
Reformat DefaultAzureCredential flow chart (#5829) 2024-07-18 22:50:26 +00:00
Ahson Khan
9b9f76a737
Re-enable running previously disabled samples using AzureCliCredential. (#5818)
* Re-enable running KeyVault samples on Ubuntu using AzureCliCredential.

* Re-enable running Attestation samples using AzureCliCredential.

* Re-enable running EventHubs samples using AzureCliCredential.

* Re-enable running Identity samples.

* Disable client_secret_credential_sample since it depends on env vars
that don't exist.
2024-07-18 14:45:27 -07:00
Azure SDK Bot
01263ec39b
Increment version for storage releases (#5808)
* Increment package version after release of azure-storage-common

* Increment package version after release of azure-storage-blobs

* Increment package version after release of azure-storage-files-datalake

* Increment package version after release of azure-storage-files-shares

* Increment package version after release of azure-storage-queues
2024-07-18 16:55:22 +08:00
Larry Osterman
90089ad326
Cleaned up Azure::Core::Context API surface (#5676)
* Deprecated Azure::Core::ApplicationContext because its use is confusing and inconsistent with the original design. 

---------

Co-authored-by: Rick Winter <rick.winter@microsoft.com>
Co-authored-by: Anton Kolesnyk <41349689+antkmsft@users.noreply.github.com>
Co-authored-by: Ahson Khan <ahkha@microsoft.com>
2024-07-17 12:38:30 -07:00
Larry Osterman
2b987eeed8
Improve code coverage numbers by excluding some options (#5805)
* Potentially improve code coverage numbers by excluding some options

* Removed dead lines; replicated switches to both gcovr executions

* added link to gcovr branch coverage faq
2024-07-17 18:36:10 +00:00
microzchang
fbccfdd758
Storage July Release (#5753)
* Storage July Release

* Update release date
2024-07-17 17:26:41 +08:00
microzchang
60fbccf8c0
Storage Fix AccessType live test failure (#5807)
* Fix live test failure

* update datalake tests
2024-07-17 17:26:21 +08:00
Larry Osterman
e19dc2371c
Use GetTestCredentials in eventhubs admin client; updated samples to allow sample execution to be disabled (#5799)
* Use GetTestCredentials in eventhubs admin client; updated samples to allow sample execution to be disabled

* Made sample execution optional

* re-enabled building keyvault samples

* Disabled RetrieveMultipleEvents test

* Disabled RetrieveMultipleEvents test

* Pull request feedback
2024-07-16 10:29:54 -07:00
Ahson Khan
7e9906f884
Added ClientAssertionCredential to enable applications to authenticate with custom client assertions. (#5789)
* Added `ClientAssertionCredential` to enable applications to authenticate with custom client assertions.

* Rename test file.

* Update client assertion credential tests.

* Fix typo.

* Address PR feedback - pass in function by value and some comment fixup.

* Update log messages to use credential name as a prefix.
2024-07-15 19:30:56 -07:00
Anton Kolesnyk
e47e3168fe
Acknowledge community contribution in the changelog (#5797)
* Mention community contribution in the changelog

* cspell

* Remove double spaces

---------

Co-authored-by: Anton Kolesnyk <antkmsft@users.noreply.github.com>
2024-07-15 18:45:06 +00:00
Azure SDK Bot
fb6c039f5e
Increment package version after release of azure-core (#5794) 2024-07-12 21:34:14 +00:00
Anton Kolesnyk
6b9e1cc691
Revert commits related to the new RetryPolicy method (#5793)
* Revert "Update the RetryPolicy for the GA release, keeping ShouldRetry extension point hidden. (#5771)"

This reverts commit 9ccd206ff8.

* Revert "Update the RetryPolicy and ShouldRetry customization logic to allow loosening the retry condition. (#5656)"

This reverts commit f1d95520d1.

* Do not remove changelog entry from a previous beta release

* Revert "Add a virtual ShouldRetry method to the RetryPolicy for customization. (#5584)"

This reverts commit ab90ef68b0.

---------

Co-authored-by: Anton Kolesnyk <antkmsft@users.noreply.github.com>
2024-07-12 12:03:43 -07:00
Anton Kolesnyk
e8c7c559d9
Azure Core July GA Release (#5792)
Co-authored-by: Anton Kolesnyk <antkmsft@users.noreply.github.com>
2024-07-12 10:24:12 -07:00
Ahson Khan
9ccd206ff8
Update the RetryPolicy for the GA release, keeping ShouldRetry extension point hidden. (#5771)
* Update the RetryPolicy for the GA release, keeping ShouldRetry extension
point hidden.

* Mark test helper virtual functions private, so they aren't
accessible/callable by callers.

* Update the changelog.

* Update CL.
2024-07-11 17:39:37 -07:00
George Arama
313fb0e58f
Move tests to use azure pipeline credentials (#5754)
* test1

* hgdfchg

* remove the remnants of azure client secret

* test KV with federated auth

* UseFederatedAuth

* fdsa

* kv template with managed

* try try again

* retry permissions

* add net acls

* blunt force replace the resource json

* put back stuff

* trey again with new method

* attempt

* missed something

* flip if else

* Temporarily use empty sub config file path for preview cloud

* remove client secret

* try to fix the identity tests

* live skip failing tests and return in samples

* samples for identity fix

* disable failing samples in identity

* fix winhttp failing test

* comment out code

* remove managed identity

* restore version from main

* revert readme changes

* PR comments

* test 2

* clang

* attempt default creds with pipeline chanined

* clangs

* identity test and clangs

* oops

* live

* cleanup

* reter

* test

* revert the DAC change

* missed one

* taking the samples to a farm upstate

* PR comments

* Fix bad merge

---------

Co-authored-by: Daniel Jurek <djurek@microsoft.com>
Co-authored-by: Anton Kolesnyk <antkmsft@users.noreply.github.com>
Co-authored-by: Anton Kolesnyk <41349689+antkmsft@users.noreply.github.com>
2024-07-11 17:38:19 -07:00
Ahson Khan
aadeca2c5c
Turn federated auth on for Identity tests. (#5785)
* Turn federated auth on for Identity tests.

* Update test resources json.
2024-07-12 00:00:15 +00:00
Anton Kolesnyk
22f5135d4c
Update vcpkg SHA (#5772) 2024-07-11 22:57:06 +00:00
Robert Schulze
8f26ebb8a8
Add missing include (#5766) 2024-07-08 09:17:46 -07:00
Ahson Khan
3e5b7064ec
Add live tests for AzurePipelinesCredential. (#5734)
* Add AzurePipelinesCredential for authenticating an Azure Pipelines service connection with workload identity federation.

* Add unit tests.

* Add comment about not throwing in the ctor, but rather deferring it.

* Order field in order of initialization and fix cspell.

* Fix ambiguous call to EnvironmentOverride in tests.

* Add a live test to AzurePipelinesCredential.

* Add invalid test cases and output response.

* Add access token env var in ci.yml.

* Add identity yml files and EnvVars.

* Fix merge conflicts and print out the oidc response.

* Remove duplicate definition of ServiceDirectory and remove env.

* Revert CI/infra changes.

* Include engsys changes to add federated auth support.

* Update environment variables used.

* Sync recent engsys changes.

* Add invalid tenant id test and re-order them.

* Fail the live test pipeline if a test fails.

* Update tests and revert source changes.

* Debug failing TokenCredentialTest in new live test environment.

* Dont fail test on missing env var.

* Disable federated auth in ci.yml and add back client secret env var.

* Remove test application secret.

* Revert other changes related to infra.
2024-06-26 11:44:01 -07:00
Azure SDK Bot
e488093c19
Increment package version after release of azure-identity (#5742) 2024-06-25 21:33:16 -07:00
Ahson Khan
dd83659802
Prepare for the June Identity beta release, oob. (#5735) 2024-06-21 14:15:48 -07:00
JinmingHu
3102b7847b
Remove client secrets, Re-make recording (#5725)
---------

Co-authored-by: microzchang <zchang@microsoft.com>
2024-06-21 12:13:38 +08:00
Ahson Khan
9b2c3b337e
Add AzurePipelinesCredential for authenticating an Azure Pipelines service connection with workload identity federation. (#5733)
* Add AzurePipelinesCredential for authenticating an Azure Pipelines service connection with workload identity federation.

* Add unit tests.

* Add comment about not throwing in the ctor, but rather deferring it.

* Order field in order of initialization and fix cspell.

* Fix ambiguous call to EnvironmentOverride in tests.

* Address PR feedback, suppress warning, move oidc fetch in token cache,
and update exception message.

* Address PR feedback, use ID and capitalize Azure Pipelines.

* Revert back to the workaround for the warning, rather than suppressing
it.

* Address PR feedback, move getting an assertion to a helper, and add
const.
2024-06-20 21:09:31 -07:00
Ahson Khan
ed933f5dfd
Add a trouble shooting guide for azure-identity, specific to C++. (#5730)
* Add a trouble shooting guide for azure-identity.

* Fix casing.

* Add cspell opt-outs.

* Update section level nesting to 3 instead of 4.
2024-06-19 15:41:08 -07:00
George Arama
0b5b5ac951
Json library update and Xcode15 (#5681)
* Move up macos and xcode version

* first pass

* to trigger builds, some other issue

* rewrwe

* Remove MSVC compiler warning suppressions in KeyVault

The code within the `mocked_transport_adapter_test.hpp` file, specifically within the `Azure::Security::KeyVault::Keys` namespace, has been updated. The `#if defined(_MSC_VER)` preprocessor directive along with its associated `#pragma warning(push)` and `#pragma warning(disable : 4996)` directives have been removed. The corresponding `#pragma warning(pop)` directive has also been removed. The lines of code that were between these preprocessor directives remain unchanged. These changes remove specific compiler warning suppressions for MSVC compilers, but do not alter the functional behavior of the code.

* dassa

* Add UseDotNet@2 where test proxy might run on a macos agent

* test CI

* fsd

* dasdas

* gregre

* gerregr

* ffdsfsd

* fwefew

* fewfwew

* reter

* fewfew

* fwefwe

* rewrw

* rwe

* temp move build step to exec faster

* gdfgd

* fsdfd

* fdsfs

* dgdf

* Roll forward instead of installing .NET 6

* remove manual install

* rewrew

* gfdffg

* rewrwe

* gfdgd

* fdsfs

* gfdgdf

* tertre

* dasda

* try http for test proxy

* revert readmes

* rew

* rew

* fdsfsd

* ewrew

* revert storages

* KV

* opentelemetry

* remove warnbing

* anton comments

* unit tests

* clangs

* clang

* comment out some asserts

* restore originals

* updated tests

* clangs

* restore cmake

* test

* Remove manual workaround (eng/common change fixes this)

* test2

* the fix doesn't work, need to put back

* Remove DOTNET_ROLL_FORWARD

* annotate macro , put back DOTNET_ROLL_FORWARD

* remove leftover files

* reww

* update changelog

* remove left over file

* remove some doxygen comments

---------

Co-authored-by: Daniel Jurek <djurek@microsoft.com>
2024-06-18 08:26:28 -07:00
Ahson Khan
20efadde4c
Add back-ticks around the WWW-Authenticate header name in exception (#5717)
message.
2024-06-13 10:12:05 -07:00
Azure SDK Bot
5264ee4e28
Increment package version after release of azure-identity (#5714) 2024-06-12 08:33:45 -07:00
Azure SDK Bot
cee566307b
Increment version for storage releases (#5715)
* Increment package version after release of azure-storage-common

* Increment package version after release of azure-storage-blobs

* Increment package version after release of azure-storage-files-datalake

* Increment package version after release of azure-storage-files-shares

* Increment package version after release of azure-storage-queues
2024-06-12 16:43:28 +08:00
microzchang
ad744e2a33
Storage June Release (#5702) 2024-06-12 10:14:00 +08:00
Azure SDK Bot
1a81885044
Increment package version after release of azure-messaging-eventhubs (#5713) 2024-06-11 23:09:10 +00:00
Ahson Khan
1e8c9d0c02
Prepare for the June Identity GA release. (#5695)
* Prepare for the June Identity GA release.

* Validate azure arc.

* Update changelog entry.

* Update cspell, fixup gtest skip, and remove unnecessary logging.

* Move gtest_skip call inside the gtest.

* Use system command due to permissions on creating a directory, on linux.

* Pass in a c_str() to system.

* Update permissions to create keys and address pr feedback (rename test
var and method to remove 'valid').

* Address PR feedback - nits.

* Fix remaining rename of local variable.
2024-06-11 15:25:02 -07:00
Azure SDK Bot
4ca2c8f028
Increment package version after release of azure-data-tables (#5710) 2024-06-11 11:58:20 -07:00
Azure SDK Bot
1b78b93274
Increment version for keyvault releases (#5709)
* Increment package version after release of azure-security-keyvault-certificates

* Increment package version after release of azure-security-keyvault-secrets

* Increment package version after release of azure-security-keyvault-keys
2024-06-11 11:43:49 -07:00
Larry Osterman
242162fa48
Prepare for June eventhubs release (#5694) 2024-06-11 10:18:55 -07:00
George Arama
83d3402eb2
Tables 1.0 beta 3 (#5672)
* Tables 1.0 beta 3

* ewrwe
2024-06-11 06:20:25 -07:00
George Arama
7a80e0a594
Keyvault June release (#5673)
* Update release date and modify sections in CHANGELOG.md
 This commit updates the release date for versions 4.3.0-beta.2 and 4.5.0-beta.2 in the CHANGELOG.md file from "Unreleased" to "2024-06-11". It also removes the "Features Added", "Breaking Changes", and "Bugs Fixed" sections for these versions. The "Other Changes" section has been updated to reflect the relocation of samples to the `samples` directory, updates to the `README.md` file, and updates to the samples.

* kashdhjlaks

* rewrwe

* Update sdk/keyvault/azure-security-keyvault-keys/CHANGELOG.md

Co-authored-by: Ahson Khan <ahkha@microsoft.com>

---------

Co-authored-by: Ahson Khan <ahkha@microsoft.com>
2024-06-11 06:20:08 -07:00