Commit Graph

637 Commits

Author SHA1 Message Date
JinmingHu
01d0c8cbcf
Update migration guide, add sample for wraping sync into async (#3469) 2022-03-24 11:16:24 +08:00
Victor Vazquez
feaaf0b414
Update release mode windows + samples (#3451)
* changes

* vs generator

* use macro

* release and samples

* add target to build smoke test

* revert

* reuse macro for samples

* extra legs for samples only

* storage connection string

* format

* order and macro

* update

* samples

* missing
2022-03-22 01:39:39 +00:00
George Arama
13175a4f41
Openssl1.1.1 (#3457)
* open ssl 1.1.1n

* manifests and vcpkg for attestation

* opensll - > git

* oops git

* set 1.1.1n as min version
2022-03-21 13:59:20 -07:00
George Arama
282ed53068
open ssl 1.1.1n (#3447)
* open ssl 1.1.1n

* manifests and vcpkg for attestation

* opensll - > git

* oops git
2022-03-21 11:17:21 -07:00
JinmingHu
71bdfddfee
Fix bug where we cannot use SAS authentication to move file/directory (#3422)
* Fix bug where we cannot use SAS authentication to move file/directory

* CL

* ut
2022-03-18 21:36:13 +08:00
Victor Vazquez
64e3130e2c
Adding Test-Proxy support to [Perf Fw] tests (#3257)
* cp

* cp

* cp

* Enable test proxy for perf tests

* unwanted

* format

* fix docs

* cspell

* parallel support and multi proxy

* update arg name

* more logs

* fix request redirection

* Apply suggestions from code review

Co-authored-by: Ahson Khan <ahkha@microsoft.com>

* run once before record, then after recording on

Co-authored-by: Ahson Khan <ahkha@microsoft.com>
2022-03-11 17:13:15 -08:00
JinmingHu
53f167dc37
Add missing header (#3420) 2022-03-10 12:04:12 -08:00
Azure SDK Bot
21b4b6a7f3
Increment version for storage releases (#3413)
* Increment package version after release of azure-storage-common

* Increment package version after release of azure-storage-files-datalake

* Increment package version after release of azure-storage-files-shares

* Increment package version after release of azure-storage-queues
2022-03-09 23:52:18 +00:00
Azure SDK Bot
2938cd012f
Increment package version after release of azure-storage-blobs (#3415) 2022-03-10 07:38:26 +08:00
JinmingHu
66b9e75010
Storage Mar. Release (#3400)
* Storage Mar. Release

* upgrade vcpkg commit
2022-03-09 08:51:55 +08:00
JinmingHu
73304d6328
SAS token version related change (#3399) 2022-03-08 15:48:00 +08:00
JinmingHu
0af72dfa58
Different storage service has its own sas version(sv) (#3380) 2022-02-28 08:47:59 +00:00
JinmingHu
f628269290
New protocol layer for File Share service (#3371) 2022-02-28 14:12:12 +08:00
JinmingHu
b26df4804d
Blob STG 75 76 77 features (#3367) 2022-02-28 13:17:03 +08:00
Larry Osterman
ddc9eb355f
Fixed Live Tests for Attestation SDK. (#3366)
* Set WinHTTP_OPTION_CLIENT_CERT_CONTEXT to enable connections to attestation service

* Set serviceDirectory from environment variables

Co-authored-by: Victor Vazquez <victor.vazquez@microsoft.com>
2022-02-24 22:48:15 +00:00
Victor Vazquez
9f58947056
use ref instead of pointer (#3364) 2022-02-18 17:15:07 -08:00
Victor Vazquez
74576d640f
Increment gTest version to get rid of warning during cmake-configure (#3351)
* increase version

* update manifest

* fix test
2022-02-17 13:41:04 -08:00
Azure SDK Bot
269f1a3ca1
Increment version for storage releases (#3344)
* Increment package version after release of azure-storage-common

* Increment package version after release of azure-storage-blobs

* Increment package version after release of azure-storage-files-datalake

* Increment package version after release of azure-storage-queues
2022-02-14 03:58:40 +00:00
JinmingHu
116185409e
Storage Feb. Release (#3340)
* Storage Feb. Release

* Update sdk/storage/azure-storage-queues/CHANGELOG.md
2022-02-13 02:30:09 +00:00
JinmingHu
62865323e3
Remove CopyFromUri (#3339) 2022-02-11 04:17:20 +00:00
JinmingHu
077d32ffff
New protocol layer for blob, datalake and queue generated by CodeGen (#3261)
* new protocol layer

* remove some [[deprecated]] to eliminate compiler warnings

* remove unused helper functions

* ut record

* revert [[deprecated]]

* fix typo
2022-02-11 11:02:51 +08:00
Anton Kolesnyk
53e90d81fb
Undo assert internal macro and headers (#3318) 2022-02-07 12:25:22 -08:00
George Arama
5290690be0
Storage src (#3304)
* before sync

* download latest version

* blobs and datalake

* shares and queues

* some cleanup

* for PR
2022-02-07 11:47:08 -08:00
Anton Kolesnyk
c4e95dfcfc
Fix unit test compilation on UWP (#3298)
Co-authored-by: Anton Kolesnyk <antkmsft@users.noreply.github.com>
2022-01-31 18:43:37 -08:00
Anton Kolesnyk
2aae6be7a3
Encapsulate getenv(), and make it work on UWP (#3275)
It all started with UWP. The [docs](https://docs.microsoft.com/en-us/cpp/cppcx/crt-functions-not-supported-in-universal-windows-platform-apps?view=msvc-170) say: "`Environment variables are not available to UWP apps.`". And it truly won't work, I tried: linker error, the function is simply not present.

So, for a year or so, we were `ifdef`ing everything enivoronment-related: console logger, environment credential, managed identity credential.

And then just recently we wanted to enable our CI for UWP, including tests and samples. And it required to do more ifdefs (in vcpkg, we don't build samples or tests, so that problem did not exist).

It just became more messy. Especially in samples - you can see how we would disable warning with `#pragma warning(disable : 4996)` or defining `_CRT_SECURE_NO_WARNINGS` already, but now came UWP, so we would have to add comment that `getenv()` is not available and make the sample compilation to either fail with clear message, or throw an exception. Plus we would have to detect that we are being compiled on UWP, which also adds visual clutter for reader. You can see how such an irrelevant (for a sample) thing as `getenv` was consuming more and more lines of sample code and reader's attention.

But then! I read docs on more APIs for UWP. And I noticed that on .NET you can read environment variables. So I went and checked Win32 API docs for [GetEnvironmentVariable()](https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-getenvironmentvariable) - it says: "`Minimum supported client: ... UWP apps`".

**GetEnvironmentVariable() works on UWP!**
And so does `SetEnvironmentVariable()` (our tests use it, which means we can make all of them work and execute for UWP).

That's good news, but now it would probably be more code: it usually takes more lines to invoke WinAPI, it is no more an one-liner to call `getenv()/setenv()`. So, I encapsulated that into `Azure::Core::_internal::Environment::GetVariable()` and `SetVariable()`. You can see how much less ifdefs is in our code now. Not to mention it works on UWP!

Per team request, that API is SDK-internal. Samples use their own mini-helper project, `get-env-helper` that makes is so that `getenv()` works naturally on Linux and macOS, compiles without warnings and works on Win32, and compiles and works on UWP (using `GetEnvironmentStringsA()`)

If it was for me, I would just make `Azure::Core::Environment::GetVariable()` public and simplify even further, I think it would be beneficial for sample readers (you can see that extra `get-env-helper` stuff adds just a little more visual clutter, compared to nothing). But I can see reasons against that, why team did not want to do it.
2022-01-29 08:22:33 +00:00
Victor Vazquez
c10de791cc
Build ci gates per service (#3260) 2022-01-20 18:09:06 +00:00
t-ransemel
9d1ab66989
fixed typo in MigrationGuide.md (#3262) 2022-01-20 15:52:56 +00:00
Victor Vazquez
20bb1ec276
Recordings for storage (#3194)
* Recordings for storage (#4)
2022-01-18 14:50:15 -08:00
Ahson Khan
47a4c44eaa
Bump dependency on azure core for all upstream SDKs and client libraries to the latest shipped version (#3216)
* Bump azure core dependency to latest shipped (identity)

* Bump azure core for template

* Bump azure core for storage (common, and all)

* Bump config.cmake.in for identity vcpkg

* Bump config.cmake.in for template vcpkg

* Bump config.cmake.in for storage common vcpkg
2022-01-11 17:06:06 -08:00
Victor Vazquez
add1496338
Fix Storage test-resources config - add one extra year (#3208)
use value 2099 for test-resources
2022-01-04 16:33:52 -08:00
Daniel Jurek
fa0fa97155
Generate map files (#3181)
* Generate map files in build

* Add .map file artifact publishing

* Add Xlinker to spelling exceptions

* Remove PublishMapFiles from clang builds

* Generate the map file artifact name

* CXX_COMPILER_ID and some logging for Clang which might be linking using the GNU linker by default

* More logging

* Move logging out of conditional

* Logging up high, use OS to determine which link flags to set

* Use linker options specific to AppleClang's context, publish map files for all platforms
2022-01-03 13:56:41 -08:00
Anton Kolesnyk
b4fe751310
RequestFailedException: decouple Message and what(). (#3201)
Co-authored-by: Anton Kolesnyk <antkmsft@users.noreply.github.com>
2021-12-18 09:00:04 -08:00
Ahson Khan
894cd75198
Update vcpkg and dependencies to the newer commit sha from 11-10-2021, after last SDK release (#3154)
* Update vcpkg to the newest published tag

* Update to latest commit sha from 12-02-2021

* Update to a commit sha from 11-10-2021, after last SDK release

* Treat third-party dependencies as system to avoid triggering warnings in code we don't control
2021-12-08 12:13:57 -08:00
JinmingHu
1b814a8757
restructure storage samples (#3113)
* restructure storage samples

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Victor Vazquez <victor.vazquez@microsoft.com>

* u

* u2

* fix link

* clang-format

Co-authored-by: Victor Vazquez <victor.vazquez@microsoft.com>
2021-11-19 09:59:34 +08:00
Anton Kolesnyk
a831b4b3f6
Exclude test, samples, and 3rd party code from code coverage (#3120)
* Exclude test, samples, and 3rd party code from code coverage

* Update min coverage requirement to a true one

Co-authored-by: Anton Kolesnyk <antkmsft@users.noreply.github.com>
2021-11-18 13:06:12 -08:00
Anton Kolesnyk
764b839dec
Make AZURE_ASSERT and related macros internal (#3059) 2021-11-17 18:31:37 -08:00
JinmingHu
52adaaa057
fix a bug in generated xml code (#3114) 2021-11-18 10:16:10 +08:00
JinmingHu
f9157c8763
New API: CopyFromUri, which copies a blob synchronously (#3098)
* sync copy blob

* CL

* Update sdk/storage/azure-storage-blobs/CHANGELOG.md

Co-authored-by: Ahson Khan <ahkha@microsoft.com>

* Update sdk/storage/azure-storage-blobs/inc/azure/storage/blobs/blob_client.hpp

Co-authored-by: Ahson Khan <ahkha@microsoft.com>

* Update sdk/storage/azure-storage-blobs/inc/azure/storage/blobs/blob_client.hpp

Co-authored-by: Ahson Khan <ahkha@microsoft.com>

Co-authored-by: Ahson Khan <ahkha@microsoft.com>
2021-11-17 10:07:40 +08:00
Anton Kolesnyk
767b03401e
Remove en-us from links in code comments (#3083)
Co-authored-by: Anton Kolesnyk <antkmsft@users.noreply.github.com>
2021-11-08 20:27:32 -08:00
Daniel Jurek
198327e16c
Reduce incidence of build storms (#3079)
* Narrower build trigger matrix, introduce batching so changes to main don't trigger builds for every checkin if there are other builds running, expand core to test everything in CI

* Wire up CtestExcludeRegex

* Improvements including scoping to exclude storage tests

* Use correct trigger settings for storage
2021-11-08 15:01:50 -08:00
Azure SDK Bot
1b19e0e269
Increment package version after release of azure-storage-queues (#3078) 2021-11-08 06:39:32 +00:00
JinmingHu
0a9796cb2d
Storage Nov Release (#3076) 2021-11-08 13:37:38 +08:00
JinmingHu
b6489259ce
Storage Nov release (#3074) 2021-11-08 04:29:12 +00:00
JinmingHu
db85b15f6a
Rename some model types for Queue service (#3028)
* QueueServiceProperties

* QueueProperties

* more

* clangformat

* CL

* Update sdk/storage/azure-storage-queues/CHANGELOG.md

Co-authored-by: Ahson Khan <ahkha@microsoft.com>

* f

* typo

* f

Co-authored-by: Ahson Khan <ahkha@microsoft.com>
2021-11-08 11:57:09 +08:00
JinmingHu
14d700eda2
Changelog for #2968 (#3039) 2021-11-04 10:29:12 -07:00
Victor Vazquez
f90ab8a9b5
Add Storage Performance tests for DownloadTo API (#2987)
Adding 3 new tests for Storage Performance
The next tests are expected to find out the overhead between each layer of the Azure SDK client while performing a download operation. The current performance test is measuring the client + azure core pipeline + http transport.

The next tests removes authentication policy and then each layer up to the HTTP transport adapter directly.

DownloadBlobSas
Generates a SaS token to download a blob. This way the authentication policy "sharedKey" won't run in the pipeline.

DownloadBlobWithPipelineOnly
Use the SaS token as well but send the request without the BlobClient. Instead it uses the Azure Core Pipeline directly

DownloadBlobWithTransportOnly
Same as before, the SaS token is used to download the blob but consuming only the libcurl-transport adapter. This option skips the HttpPipeline. The only elements used from Azure Core are the HttpRequest and the Azure::Context

Note
porting tests from the .NET equivalent:

https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/Azure.Storage.Blobs/perf/Azure.Storage.Blobs.Perf/Scenarios/DownloadSasUriBlobClient.cs

https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/Azure.Storage.Blobs/perf/Azure.Storage.Blobs.Perf/Scenarios/DownloadSasUriHttpClient.cs

https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/Azure.Storage.Blobs/perf/Azure.Storage.Blobs.Perf/Scenarios/DownloadSasUriHttpPipeline.cs
2021-10-25 11:48:55 -07:00
Daniel Jurek
c534a1e160
Cognitive Matrix Consolidation (#2947)
* Refactor template to use multiple builds on the same machine resetting the machine and runtime varaibles along the way

Move template expression out of default matrix parameters

Remove extra reference to ci/jobs.yml

Remove unnecessary ci.yml, change param declaration in devops-variables-set.yml

Heredoc

Fix set/reset scripts after testing with devops outputs

-ErrorAction Ignore

Smaller matrix

OsVMImage comes from machine, not parameters

Add Ubuntu 20 to matrix

Remove checkout: self, it is implied

Move machine-setup steps outside of loop

CODE_COVERAGE handling

tab

Remove creation of CODE_COVERAGE

Use AptDependencies and add apt update to dependency install step

Remove extra preparation steps

Enable code coverage check scenario

Add MacOS 11

Disable vcpkg cache

Coverage configuration

Pass parameter properly

bool -> boolean

Install coverage tools if it's possible that coverage may run

More logging on code coverage

Show contents of Makefile

One vcpkg to build them all

Verbose

SourcesDirectory

Nesting VcpkgInstall

workingDirectory for vcpkg install

AptDependencies

complete the conditional

Enable all matrix entries

sudo

Remove extra chatter

More scrubbing post-build to help with Windows scenarios

sudo echo

Get-ChildItem

ErrorAction

sudo sh -c

More ErrorAction

output sourceListFile

Disable additional sources.list

Install coverage tools when used for coverage

COndition

Do not build testing for Windows arm64

Correct path for coverage tools

ne -> eq

Add more scenarios to the matrix

Remove libxml2

Add MacOS 10.15 to matrix

Revert "Remove libxml2"

This reverts commit 178af8b89abb21ee84dbcd25b4d54bd90ec1d44c.

Add libxml2 to MacOS 10.15 matrix entry

Display name decorator, refine the matrix

Remove OsVmImage variable

Add commentary to tests about matrix

Rebalance matrix, nest EnvVars, DisplayNameDecorator

Correct parameter access

AptDependencies

Ensure building for WindowsStore in UWP builds, Only build tests for debug builds in altered matrix entires, Linux build type specified as environment variable

Rebalance matrix for better timing

Remove all instances of libxml2 in vcpkg dependencies

Move common devops tasks to eng/common

Set eng/common paths in pipelines

Revert "Remove all instances of libxml2 in vcpkg dependencies"

This reverts commit 35ffbc7aac87f400fdf3351076346750c1c750ed.

https://github.com/Azure/azure-sdk-for-cpp/issues/2946

Permit words like iname in more pipeline .yml files

Remove libxml2 from vcpkg dependencies for Windows

Perf test matrix maxParallel: 10

Validate

apt install first

apt install first

maxParallel: 12

-j 8

-j 10

Use matrix generator

Add expected parameters

Move parameters to appropriate file, fix cloud spec

Specify correct platform matrix location

DependsOn

Platform matrix

Remove DisplayNameDecorator, try more clang information

Rapid iteration matrix

clang --version

install clang-11

Use correct pool

Use full matrix, revert additional logic in jobs/ci.yml

* Review feedback: newline

* Review feedback: Test in release, too

* Remove DisplayNameDecorator (it was useful when we were putting multiple builds into a single job, now it is redundant)

* Remove displayNames

* Matrix review feedback

* Conditions

* Rename ci.yml -> ci.tests.yml
2021-10-22 13:49:10 -07:00
Anton Kolesnyk
b477373a9c
Remove extra semicolons (#2979)
Co-authored-by: Anton Kolesnyk <antkmsft@users.noreply.github.com>
2021-10-19 16:08:42 -07:00
JinmingHu
9dac89c675
make DefaultEncryptionScope and PreventEncryptionScopeOverride optional (#2968) 2021-10-16 10:06:18 +08:00
Victor Vazquez
f4e6390dff
Enable no-rtti (#2963)
* Enable no-rtti
2021-10-14 18:12:03 -07:00