Commit Graph

447 Commits

Author SHA1 Message Date
JinmingHu
08e364e1cd
replace std::tolower with Azure::Core::ToLower (#834) 2020-10-23 09:43:30 +08:00
Victor Vazquez
9d47db1d4a
Add RequestFailedException (#717)
* Add RequestFailedException
2020-10-20 22:22:02 -07:00
Victor Vazquez
e1dddce239
remove azure hpp (#809)
Remove Azure.hpp

Make Azure/Core/Strings.hpp public

Move ToLower and InvariantIsEqual to Strings.hpp

Also removing `AZURE_UNREFERENCED_PARAMETER`  

fixes: https://github.com/Azure/azure-sdk-for-cpp/issues/811
2020-10-20 18:08:58 +00:00
Kan Tang
b6ab1d0872
Added date value to changelog (#800) 2020-10-16 20:45:32 +08:00
Kan Tang
ef4b7c5268
Resolved the issue in datalake/file where file/path/directory/share/f… (#796)
* Resolved the issue in datalake/file where file/path/directory/share/filesystem name are not encoded.

* Refined special char string

* Resolve comments.
2020-10-16 14:26:50 +08:00
JinmingHu
95fea7af78
fix url encoding issue (#795)
* fix more

* add ut

* add more ut
2020-10-16 14:08:10 +08:00
JinmingHu
5bad554a06
fix path encoding issue (#794) 2020-10-16 12:43:12 +08:00
Victor Vazquez
f9e6acb358
Update Readme and Contribute to Oct07-2020 release instructions (#712)
* Update Readme and Contribute to Oct07-2020 release instructions
2020-10-14 22:00:34 -07:00
Kan Tang
a7672f67f6
Renamed all continuation related object to ContinuationToken. (#779)
* Renamed all continuation related object to ContinuationToken.

* Refined changelog.
2020-10-14 15:18:38 +08:00
Kan Tang
e0079f0541
Update Changelog and remove breaking changes individual file. (#778)
Update Changelog and remove breaking changes individual file
2020-10-14 14:35:57 +08:00
JinmingHu
443369b057
[Storage] Rename NextMarker -> ContinuationToken (#777)
* rename nextmarker->continuationtoken

* update changelog

* Update sdk/storage/azure-storage-blobs/CHANGELOG.md

Co-authored-by: Ahson Khan <ahkha@microsoft.com>

* fix

Co-authored-by: Ahson Khan <ahkha@microsoft.com>
2020-10-14 14:19:38 +08:00
JinmingHu
e0d717a0d3
revert non-overwriting changes (#773) 2020-10-14 13:23:40 +08:00
JinmingHu
570c774ebd
fix typo in doc (#774) 2020-10-14 12:05:20 +08:00
JinmingHu
b0e5d3d356
update changelog for storage beta3 release (#772) 2020-10-14 10:46:30 +08:00
Anton Kolesnyk
a38de03e7b
Extract part of credentials to Identity module (#748) 2020-10-13 07:33:18 -07:00
Kan Tang
e9f37a5760
Change 4 spaces into 2. (#761) 2020-10-13 13:02:33 +08:00
Kan Tang
9633fbc6be
Modified version, Added changelog/breaking changes for beta.3 (#759) 2020-10-13 04:30:31 +00:00
JinmingHu
f3b324d4f5
changelog (#758) 2020-10-13 04:20:50 +00:00
JinmingHu
a07aa979ce
concurrent upload is overwriting (#760) 2020-10-13 04:02:23 +00:00
JinmingHu
320a22e664
Make sure CI pipelines legs get triggered with cmake modules are modi… (#756)
…fied.
2020-10-13 03:10:51 +00:00
Kan Tang
f43ef4b137
Enabled Get file range diff with snapshot. (#757) 2020-10-13 10:56:46 +08:00
Kan Tang
af71e0167d
Removed Nullable return value that never returns. (#747) 2020-10-13 10:41:23 +08:00
JinmingHu
5d1a8f4cbc
Upload/Create is non-overwriting by default (#745)
* add overwrite option

* fix failed ut

* add ut
2020-10-13 10:41:15 +08:00
JinmingHu
b6032bfe54
Enable tests for setting tiers for snapshots/versions (#743) 2020-10-13 09:42:10 +08:00
Victor Vazquez
9eb132249b
Url use url encoded queries (#730)
* keep url-encoded queries
2020-10-12 09:39:46 -07:00
Kan Tang
966b7892b2
Avoid test case throw meaningless exception when sys env is not set. (#723)
* Avoid test case throw meaningless exception when sys env is not set.

* Update sdk/storage/azure-storage-common/test/test_base.cpp

Co-authored-by: JinmingHu <jinmhu@microsoft.com>

* Update sdk/storage/azure-storage-common/test/test_base.cpp

Co-authored-by: JinmingHu <jinmhu@microsoft.com>

* Update sdk/storage/azure-storage-common/test/test_base.cpp

Co-authored-by: JinmingHu <jinmhu@microsoft.com>

* Apply clang format

Co-authored-by: JinmingHu <jinmhu@microsoft.com>
2020-10-12 14:49:30 +08:00
Kan Tang
582377fc1d
Fixed multichannel bug. (#737)
* Fixed multichannel bug.

* Resolved some comments.
2020-10-12 11:48:48 +08:00
Victor Vazquez
2cc4ecdea8
Validate invalid HTTP headers (#703)
Creating new function to insert headers by validating characters on it are accepted chars for header name
fixes: https://github.com/Azure/azure-sdk-for-cpp/issues/191
2020-10-09 23:35:58 +00:00
Victor Vazquez
dc9d3b0595
update CMake min version to 3.15 (#710)
Storage is using CMake version 3.15 and it is currently required for generating without errors
2020-10-09 23:26:01 +00:00
Kan Tang
f59ab214a6
Integrate storage retry policy. (#722)
* Integrate storage retry policy.

* Removed the logic to make URL empty when '.blob.' is not found in URL.
2020-10-09 14:30:35 +08:00
Kan Tang
1b351de2d8
Added 2020-02-10 features. (#692)
* Added 2020-02-10 features.

* Resolved comments.
2020-10-09 11:05:24 +08:00
Kan Tang
a898fd819a
Added SetExpiry DataLake convenience layer. (#696) 2020-09-30 16:13:37 +08:00
JinmingHu
e238e28b0d
[Storage Common] Storage Retry Policy (#588)
* storage retry policy

* revert changes to azure core

* refactor storage retry policy

* fix typo

* change the logic a little bit

* adapt storage retry policy in blob batch client

* fix build error

* Add StorageRetryWithSecondaryOptions
2020-09-30 13:14:28 +08:00
JinmingHu
8c4c4e75a9
[Storage Blobs Service] STG 74 features (#652)
* SetExpiry protocol layer

* add test

* last access time

* upgrade blob sas to 2020-02-10

* datalake sas

* fix build error

* tag count

* enable tag test

* fix build error

* enable container soft delete test
2020-09-30 11:38:25 +08:00
JinmingHu
fbe95f36c5
fix variable casing (#689) 2020-09-28 11:10:32 +08:00
Sima Zhu
8fed524b96
Replaced relative link with absolute links and remove locale (#668) 2020-09-17 22:28:36 -07:00
Kan Tang
34944552ad
Resolved an issue where path is not encoded. (#657) 2020-09-16 11:13:12 +08:00
JinmingHu
7584c208ab
Fix bug: encode blob name (#655) 2020-09-16 11:08:59 +08:00
JinmingHu
95f4034490
fix bug in blob tags (#653) 2020-09-15 10:52:23 +08:00
JinmingHu
73ba0029bc
Add date in Changelog (#637)
* Add date in Changelog

* Update changelog

* Update changelog
2020-09-10 13:51:31 +08:00
JinmingHu
29c58ca4bf
[Storage Blobs Service] update changelog (#615)
* update changelog

* Add changelog

* Update changelog
2020-09-09 10:32:35 +08:00
JinmingHu
a5d09b5f47
disable blob tags ut (#613)
* disable blob tags ut

* fix ut

* disable tags in sas
2020-09-09 09:04:44 +08:00
JinmingHu
c949400dba
fix bug in file sas (#597) 2020-09-04 17:30:07 +08:00
JinmingHu
bcdeeb1434
[Storage Blobs Service] Blob Tags (#416)
* blob tags

* add TagConditions for CreateSnapshot and GetBlockList

* update tags ut

* remove unsupported conditional headers from PutBlockFromUriOptions

* sas permission ut for tags

* add comment
2020-09-04 14:47:34 +08:00
JinmingHu
911a879a00
[Storage Blobs Service] Blob Batch (#582)
* blob batch

* fix build error

* Update sdk/storage/azure-storage-blobs/test/blob_batch_client_test.cpp
2020-09-04 14:47:11 +08:00
JinmingHu
07208d1720
[Storage Files Service] File sas (#568)
* File sas

* rebase on lastest master
2020-09-04 14:46:24 +08:00
Kan Tang
d12db7f413
Added ChangeLog/Readme for file service. (#573) 2020-09-04 04:31:07 +08:00
Kan Tang
d9b58fd935
Resolved breaking change introduced by URL. (#584) 2020-09-04 04:18:48 +08:00
Victor Vazquez
38d90d62f3
URL encode and URL Unification (#561)
* Update URL from CORE with uri builder from storage and make Request to take URL to be created
2020-09-02 23:57:03 -07:00
Daniel Jurek
919610f429
Use version.hpp for package versions (#572)
* Add version.txt
* Add doc generation for template
* Add version.hpp parsing and update capabilities to cmake and engsys
* Get-SdkVersion -> Get-PkgVersion
* Move Update-PkgVersion.ps1 under eng/scripts
* Get-PackageVersion -> Get-PkgVersion
* Update paths, params, verbosity
* Couple fixes to output and make use of new SemVer version
* Add fallback support for verison.txt in cases where we still use it to unblock release artifact generation
* Use version information in release pipeline
* Add workaround to generate storage pipeline artifacts
* eng/scripts/
* Write warning
* Haven't released storage-file-shares yet according to releases on GitHub
* Set release date on changelog.md
* Update CHANGELOG.md to indicate this is a test release
* Remove fallback exception for storage
* Re-add Rick's suggestions
* Revert "Remove fallback exception for storage"
2020-09-02 18:53:46 -07:00
Kan Tang
97bfdf91d2
Rewritten the user agent. (#555) 2020-08-28 21:23:31 +08:00
Kan Tang
f9fee4f56a
Added file/datalake changelog (#554) 2020-08-28 18:26:29 +08:00
JinmingHu
9a1f84fbfb
Update changelog (#551) 2020-08-28 17:09:05 +08:00
JinmingHu
66c331e52b
Resolve review comments (#545)
* Rename UndeleteContainer->Undelete

* Remove concurrent upload page blob

* move offset to the first parameter

* Rename: TransactionalMd5 TransactionalCrc64

* add batch protocol layer
fix x-ms-server-encrypted
transactional-md5

* transcational md5

* Remove Transactional headers from ClearPagesResult

* parse x-ms-blob-content-md5

* Remove redundant space
2020-08-27 22:44:29 +08:00
Kan Tang
62dd39c1be
Resolved review comments. (#544)
* Fixed review comments.

* Resolved some review comments
2020-08-27 16:54:35 +08:00
JinmingHu
3a4e1f3b49
Restructure storage directory (#536)
* Restructure azure storage directory

* Add tests and sample

* reformat with clang-format

* Reorder files in CMake

* load versions from version.txt

* fix crash

* Update README.md

* Use relative links

* fix CI

* Fix CI

* Try to fix CI

* Move option to the top of the file
2020-08-27 16:16:08 +08:00
Kan Tang
53bd00034b
Added samples and more test cases to file service and resolved some issues. (#539) 2020-08-26 19:56:15 -07:00
Kan Tang
2c84e3f094
Resolve some casing issue (#534) 2020-08-26 12:09:06 +08:00
Rick Winter
c5438d623b
Restructure the azure inc folder (#514)
* Restructure the azure inc folder
2020-08-25 20:08:34 -07:00
Kan Tang
ae6ada5cfb
Added lease support for DataLake's Path (#521) 2020-08-26 07:43:29 +08:00
Kan Tang
b87bd0816c
Added GetSubDirectory. (#524) 2020-08-26 07:43:08 +08:00
Kan Tang
9c9e631c0d
Fixed datalake/file service protocol layer remaining issues. (#523)
* Fixed datalake build warnings on linux

* Pass context by reference instead.

* Added metadata key case check

* Fixed acronym casing.

* Fixes more acronym casing.
2020-08-26 07:42:46 +08:00
JinmingHu
cf08d39d14
Remove context from StorageError::CreateFromResponse (#511)
* Remove context from StorageError::CreateFromResponse

* Resolved issues in datalake/file protocol layer

Co-authored-by: Tank Tang <kat@microsoft.com>
2020-08-25 14:38:18 +08:00
Kan Tang
38241bb152
Added support for directory/file client for file service. (#510)
* Added support for directory/file client for file service.

* Resolve multiple comments

* Resolved some further comments

* Removed ShareSnapshot

* Fixed protocol layer and integrated refined concurrent download test from blob service.

* Fixed CI issue on Linux/Mac

* Adapted to change of Storage per-retry policy.

* Removed inconsistency comparing towards other SDKs.
2020-08-25 12:17:21 +08:00
JinmingHu
f0ece5fd7a
[Storage Blobs] Optimize blob ut (#513) 2020-08-25 10:14:20 +08:00
JinmingHu
b2dfbabd04
StoragePerRetryPolicy (#512) 2020-08-25 09:52:11 +08:00
JinmingHu
755085f25d
GetBlobServiceStatistics (#496) 2020-08-21 09:29:26 +08:00
JinmingHu
b9299478ee
[Storage Blob Service] Concurrent upload for Page Blob (#486) 2020-08-20 10:54:28 +08:00
JinmingHu
070772e7c5
Remove Content-MD5 and ContentCRC64 from Create{PageBlob|AppendBlob}Result (#487) 2020-08-20 10:52:21 +08:00
Mitch Denny
420a72a9d5
Adjust CI YAML headers (#448) 2020-08-19 13:49:45 +10:00
JinmingHu
4c5286803b
[Storage Blob Service] ORS (#467) 2020-08-19 09:13:13 +08:00
Kan Tang
e0bb037cdd
Fixed test failure due to network latency too low. (#466) 2020-08-18 11:13:39 +08:00
JinmingHu
42ffbc853e
[Storage Blobs Service] Seal Append Blob (#465)
* seal append blob

* copy seal append blob
2020-08-18 09:55:12 +08:00
Kan Tang
3042f69d5b
Rename return types for Datalake and file services. (#464) 2020-08-18 09:28:31 +08:00
JinmingHu
1dce74c642
[Storage Blobs Service] Undelete Container (#462)
* Undelete Container
2020-08-18 09:11:59 +08:00
JinmingHu
24959f30da
[Storage Blobs Service] Rename Results and Options (#460)
* rename UploadFrom

* Rename CommitBlockListResult/Options GetBlockListResult/Options

* Rename DownloadBlobTo{Options/Result} UploadBlockBlobFrom{Options/Result}
2020-08-17 10:06:58 +08:00
JinmingHu
86a22901e9
[Storage Blobs Service] Transactional MD5 and CRC64 (#456)
* MD5

* Crc64

* crc64 concatenate
2020-08-16 00:30:49 +08:00
JinmingHu
069e359562
Rename return types and option types (#449)
* rename return types and option types

* define ConcurrentUploadBlockBlobFromBufferOptions and ConcurrentUploadBlockBlobFromFileOptions

* use std::move

* Rename DownloadTo

* update doc

* Resolved datalake build issues.

* Resolved some cmake issue when linking

Co-authored-by: Tank Tang <kat@microsoft.com>
2020-08-14 13:02:10 +08:00
Kan Tang
d77ecf7917
Added support for share service (#450) 2020-08-14 13:01:51 +08:00
JinmingHu
698c4f5848
Replace old RFC1123 impl which cannot be built until GCC 5 (#440) 2020-08-13 08:17:27 +08:00
JinmingHu
a848c65933
x-ms-client-request-id header (#439) 2020-08-13 08:16:52 +08:00
JinmingHu
4c549b1a29
Fix build error (#438) 2020-08-13 08:16:21 +08:00
JinmingHu
2027d5102f
add test cases for base64 encode/decode, hash wrapper (#431) 2020-08-12 10:13:16 +08:00
JinmingHu
65bd5c6bf4
use #include "" instead of #include <> (#418) 2020-08-12 10:12:48 +08:00
JinmingHu
ee0b8d07b8
pass Context by const ref (#432) 2020-08-12 07:02:28 +08:00
Rick Winter
8821aadb80
Rename UUID -> Uuid (#436)
* Rename UUID -> Uuid
Update header references
2020-08-11 13:23:14 -07:00
JinmingHu
68643e3ee2
[Storage Blobs Service] Add test cases for LastModifiedTime and ETag and lease id (#425)
* add test cases for LastModifiedTime and ETag and lease id
2020-08-11 10:40:59 +08:00
JinmingHu
8372d39cc1
[Storage Blobs Service] Customer-Provided Key (CPK-V) and Encryption Scope (CPK-N) (#427)
* cpk-v and cpk-n
2020-08-11 09:45:35 +08:00
JinmingHu
a47789c311
Fix bug: CopyBlob always sends empty access-tier header (#426) 2020-08-11 07:25:09 +08:00
JinmingHu
329a30f0e3
[Storage Blob Service] integrate reliable stream (#421)
* integrate reliable stream

* define c_reliableStreamRetryCount
2020-08-10 16:07:38 +08:00
JinmingHu
cc0ff27eec
[Storage Blobs Service] Only make one HTTP request if the blob size is small for Concurrent Upload API (#419)
* Only make one HTTP request if the blob size is small for Concurrent Upload API
2020-08-08 08:44:03 +08:00
JinmingHu
c700b7811c
[Storage Blobs Service] Blob Versioning (#409)
* blob versioning
2020-08-07 12:14:26 +08:00
JinmingHu
355184ef4f
[Storage Blob Service] Container Lease and Blob Lease (#402)
* container lease

* blob lease
2020-08-06 13:50:15 +08:00
Kan Tang
46c0af9a70
Added service prefix to file names. (#399) 2020-08-06 13:09:02 +08:00
JinmingHu
1081ed6979
Fix compiler warning (#404) 2020-08-06 07:47:36 +08:00
JinmingHu
5e596b723e
Fix compiler warning (#403) 2020-08-06 07:47:20 +08:00
JinmingHu
52dbc6b2f2
Pascal case for acronyms (#391) 2020-08-05 10:32:03 +08:00
Kan Tang
626a8856f0
Added support for get/set file service properties. (#384) 2020-08-05 10:31:48 +08:00
Victor Vazquez
eeffd6248f
update context to const ref (#389) 2020-08-04 09:45:13 -07:00
Kan Tang
3de57667d4
Added support for create/delete file shares. (#382)
* Added support for create/delete file shares.

* Removed nullable for metadata.
2020-08-04 09:54:04 +08:00
JinmingHu
8867c7abcb
[Storage Blob Service] Get/SetBlobContainerAccessPolicy (#381)
* Get/SetBlobContainerAccessPolicy

* Remove ResetPermissions()
2020-08-04 09:53:34 +08:00
JinmingHu
18b02094ed
Parse error message only from bodyBuffer (#380) 2020-08-04 09:53:22 +08:00
Victor Vazquez
5426cf7a07
Retriable stream (#369)
* Adding retriable_stream
2020-08-03 10:09:12 -07:00
JinmingHu
22f4ba12fc
GetAccountInfo (#371) 2020-08-03 10:32:43 +08:00
Kan Tang
1b4ea7ac70
Added file protocol layer. (#367)
* Added file protocol layer.

* Resolved some comments.
2020-08-01 20:23:50 -07:00
JinmingHu
b9212922e7
GetBlobServiceProperties and SetBlobServiceProperties (#366) 2020-07-30 16:39:06 +08:00
JinmingHu
cbec5c6af6
Add unit test cases that require SAS token (#359) 2020-07-29 17:42:14 +08:00
JinmingHu
b0b5a18d18
User Delegation SAS (#358) 2020-07-29 16:11:21 +08:00
JinmingHu
b0facd3dd5
adapt to breaking change in Azure Core (#356)
* adapt to breaking change in Azure Core

* Adapt breaking change for Datalake service.

* Adapt to the breaking change

* Adapt breaking change for Datalake service.

* fix ut

* Fixed UT.

Co-authored-by: Tank Tang <kat@microsoft.com>
2020-07-29 14:55:53 +08:00
JinmingHu
9d1566c5ea
Add Account SAS and Blob SAS (#353)
* Add Account SAS and Blob SAS

* add doc for sas

* remove Nones

* remove Nones
2020-07-28 13:43:04 +08:00
JinmingHu
b380dfb76a
Fix inconsistent RFC1123 date format in different system locales (#352) 2020-07-27 18:49:42 -07:00
Kan Tang
573ffdf1f7
Added Readme/Changelog and 3rd party notice. (#340) 2020-07-24 11:14:10 +08:00
JinmingHu
ef03d3b3ec
Rename XxxRespone->XxxInfo (#337)
* Rename XxxRespone->XxxInfo

* fix doc, remove unused include headers, remove fab4

* Rename SetAccessTierInfo->SetBlobAccessTierInfo
2020-07-24 09:40:15 +08:00
Kan Tang
8568327a56
Refined DataLake sample (#336)
* Refined DataLake sample

* Added DirectoryClient::GetFileClient, resolved some comment and test failures.

* Resolved an issue introduced by lowercasing response headers resulting in error code/message not being passed, and refined sample.

* Refined the structure according to comments.

* Changed buffer initialziation to stack allocated.

* Resolved some further comments.
2020-07-23 15:40:16 +08:00
JinmingHu
29f5e30ad3
integrate TelemetryPolicy into Blob Service (#328)
* integrate TelemetryPolicy into Blob Service

* Also add telemetry support to ADLS Gen 2.

Co-authored-by: Tank Tang <kat@microsoft.com>
2020-07-23 15:39:51 +08:00
Kan Tang
b0b348e9cb
Resolved API review comments and added FileClient and DirectoryClient… (#308)
* Added API to extract raw response.

* Resolved API review comments and added FileClient and DirectoryClient for DataLake.

* Added some blob/adls Gen2 interop functionality.

* Resolved some issues/comments

* Adds more interop, support retry and concurrent upload/download.

* Added support for Azure Core's token credential to support bearer token credential.

* Added integration on RawResponse/Response<T> and resolved some test issues.

* Added validation for Client secret authentication.

* Resolved some review comments and resolved CI issue.

* Resolved a UT failure in Azure Core.

* Resolved some further comments.
2020-07-22 16:03:44 +08:00
JinmingHu
cd023c49f0
Remove MD5 and CRC64 function interface (#324) (#329) 2020-07-22 11:27:27 +08:00
JinmingHu
4d19f3b078
Integrate Response<T> into Storage Blobs Service (#327)
* Revert changes to storage code

* adapt to breaking changes in Azure::Core, Response->RawResponse, pipeline clone returns unique_ptr

* just to make build pass, remove this commit after https://github.com/Azure/azure-sdk-for-cpp/pull/326 is merged

* adapt to Response<T>

* concurrent upload/download also return Response<T>

* add BlobInfo and BlobContainerInfo
2020-07-21 10:31:00 +08:00
Daniel Jurek
1a9918c4db
add CMakeLists.txt and eng to CI and PR triggers (#314) 2020-07-20 10:09:50 -07:00
JinmingHu
2c21d4b018
Remove MD5 and CRC64 function interface (#324) 2020-07-20 10:35:09 +08:00
Anton Kolesnyk
b259db5a3b
Telemetry policy (#264) 2020-07-17 17:03:47 -07:00
Victor Vazquez
6122529bac
Core/response t (#316)
* rename http response to RawResponse

* Adding Response T
2020-07-17 14:43:17 -07:00
JinmingHu
0f37994e44
add perf benchmark code (#319)
* Add multithread perf testcase

* add large scale upload testcase

* Fix compiler error on GCC 9

* Fix failed testcase
2020-07-17 11:16:18 +08:00
JinmingHu
204c025d09
suggested changes in API review (#312)
* suggested changes in API review

* Rename UrlBuilder->UriBuilder

* add perf test case

* Integrate retrypolicy into blob clients

* remove `Content-Length` header for GET/HEAD/DELETE http requests

* Remove temp files in ut
2020-07-16 00:49:55 -07:00
JinmingHu
3db59439c9
[Storage Blobs Service] ListBlobsByHierarchy (#265)
* add API ListBlobsByHierarchy

* fix compiler error on GCC 8

* Update Blob API version to 2019-12-12

* use lowercase header key
2020-07-15 10:45:22 +08:00
Kan Tang
a1ec5119a6
Datalake full feature support PR. (#257)
* Added test cases.

* Adapt stream breaking change and adding usage for Nullable.

* Resolve CI issue.

* Resolved comments from review.

* Integrate access condition support and resolve more comments.

* Resolved some further comments.

* Resolved some further comments and removed GetMetadata.
2020-07-13 05:31:02 -07:00
JinmingHu
83406f23dd
Storage Blob Service Access Conditions, Bitwise enum class (#252)
* access conditions

* bitwise enum class

* Update x-ms-date on each retry

* add a header for all constant strings

* fix some ut

* rename ModifiedTimeAccessConditons->LastModifiedTimeAccessConditions
2020-07-09 21:03:32 -07:00
JinmingHu
95aa6361ef
Concurrent upload block blob from file, concurrent download blob to file (#247)
* concurrent download to file

* concurrent upload block blob from file

* update sample

* Change default download chunk size from 8MB to 4MB

* fix CI
2020-07-08 13:26:22 +08:00
JinmingHu
66c7518dce
Suggested changes in API review (#244)
* Rename Properties->HttpHeaders

* Rename BlobXxxItems->Items

* Remove MaxResults in response of list operations

* specify time format in comments for GetUSerDelegationKey API

* per operation pipelines and per retry pipelines

* Define some response types and rename ListBlobsFlat

* Add different return types for some APIs

* Rename BlobDownloadInfo->BlobDownloadResponse

* Assign default value to BlobContentLength to supporess compiler warnings

* add concurrent download to buffer

* concurrent upload block blob from buffer
2020-07-07 10:54:05 +08:00
Victor Vazquez
9fb4119ccb
Memory Stream - File Stream (win/lin) - HTTP Request / Response - Updates (#236)
* make stream use int64_t instead of uint64_t
2020-07-03 22:22:16 -07:00
Chidozie Ononiwu
c22e4f8386
Clean up azure-sdk-tools and reduce dependency on external resources (#228) 2020-07-02 14:20:12 -07:00
JinmingHu
e61c3ff51a
Documentation for BlockBlobClient and Blob options (#235)
* Documentation for blob options

* Rename struct UploadBlobOptions -> UploadBlockBlobOptions

* documentation for BlockBlobClient
2020-07-02 10:01:27 +08:00
JinmingHu
74ce80659b
Documentation for Storage Blob Service, Nullable, signed int, and others (#213)
* use Nullable

* use C++ style cast

* Add documentation

* use signed int instead of unsigned

* disable storage ut

* add timeout in protocol layer

* enable stroage ut

* undelete test case

* Support copy status in properties

* Remove default value for SequenceNumber in CreatePageBlobOptions
2020-06-30 03:03:42 -07:00
Victor Vazquez
2877c959e1
Enable storage unit tests (#227)
* enable core unit tests to be compiled
* Avoid running core unit tests in CI
2020-06-29 14:19:46 -07:00
Victor Vazquez
d58cbad14d
Unique ptr for body stream (#214)
* change bodyStream to unique ptr
2020-06-29 10:33:31 -07:00
Anton Kolesnyk
ad2e63afe1
Remake ClientSecretCredential (#212) 2020-06-27 15:27:11 -07:00
Daniel Jurek
0cfcdca725
Unified Pipelines for storage (#184)
* Add ci.yml for storage

* OSVmImage

* Use correct name for storage

* Update ci.ymls

* Documentation generation

* Fix tabbing

* More tabbing

* Use correct service directory for storage ci.yml

* Add docs assets

* Use cpp for dropdown generation

* Use cmake to generate documentation

* Use CMake to generate doxygen docs

* BuildArgs -> GenerateArgs

* Correct template path

* More CURL options

* Install curl to satisfy minimum build requirements

* use dependency variable name

* Add VCPKG_DEFAULT_TRIPLET

* Always generate documentaion when -DBUILD_DOCUMENTATION=YES

* Build docs at top level

* Change variable names, simplify cmake-build.yml

* -DBUILD_DOCUMENTATION

* Try using MathJax for formula rendering instead of Latex

* Add version.txt for storage

* artifact.Name -> artifact.Path

* Build docs by target

* Remove Doxyfile

* Remove Doxyfile.template

* Remove generate_docs.py, we are using cmake

* Pass CtestRegex to archetype-sdk-client job template)
2020-06-26 10:42:48 -07:00
Kan Tang
1564e5106e
Resolved comments and rewrote StorageError. (#199)
* Resolved comments and rewrote StorageError.

* Changed Upn name, removed ';', refined error output, moved local function to unamed namespace.
2020-06-24 08:54:06 -07:00
JinmingHu
f74011a2e0
Add Unittest for Blob Service, other fixes and enhancements found during validation (#181)
* Give AccessTier a default value to eliminate warning

* Add blob clients conversion functions

* Fix bug in shared key authentication

* Add GetUri().

* make lease-status and lease-state optional

* Check duplicates in metadata

* add tests for blob service

* AppendBlocb doesn't support AccessTier

* fix bug in "RangeFromXml"

* redefine PageRange in convenience layer

* suppress warnings from gtest

* add API AppendBlockFromUri

* connection string parsing

* Add memorystream

* Rename connection string constants

* Adapt to new bodystream, this will be rewritten when bosystream is changed to unique_ptr

* fix compiler errors

* Change MemoryStream interface

* samples framework

* Add newline at the end of file

* fix compiler error
2020-06-24 01:17:09 -07:00
Kan Tang
a8a200ef26
Added convenience layer for datalake service (#178) 2020-06-20 19:55:41 -07:00
Victor Vazquez
90e57362bf
Victor/http/add streams (#183)
* Adding body Stream for HTTP
* Adding libcurl session with custom send and receive
2020-06-18 21:43:00 -07:00
JinmingHu
5758bedc5c
Export symbols when building dll (#177) 2020-06-16 13:28:37 +08:00
JinmingHu
4366159c50
Add more APIs in storage sdk (#169)
* Add xml serializer/deserializer

* Add blob service client, add undelete, set access tier, start copy, abort copy, create snapshot, stage block from uri

* add support for conditional headers

* support range download

* Change the helper functions order to alphabetical

so that we always get the same result in different environemnts.

* use c++ casting

* Add support for append blob and page blob

* remove redundant ; added by auto-generator

* Remove BlobType in BlockBlob::Upload

* Rename blob_client_options.hpp->blob_options.hpp

* Remove API version in request options
2020-06-15 17:31:36 +08:00
Kan Tang
18408f848a
Adding ADLS gen2 protocol layer code. (#153)
* Adding test framework for azure storage.

* Adding ADLS gen2 protocol layer code.
2020-06-09 06:31:12 -07:00
JinmingHu
401298577e
Shared Key Authentication Policy (#160)
* Remove Date in request options, assign x-ms-version a default value

* reformat files with clang-format

* Add shared key policy and basic request policy

* Add openssl dependency for linux and macOS

* add some comments, some slight fixes

* Rename CommonRequestPolicy -> CommonHeaderRequestPolicy
2020-06-09 03:38:35 -07:00
Kan Tang
6e01498038
Adding test framework for azure storage. (#152) 2020-06-09 02:45:11 -07:00
JinmingHu
5fc6fea9f4
Auto-generated protocol layer of Container and BlockBlob (#144)
* add blob service auto generated protocol layer

* Add storage uri builder

* integrate with pipeline

* Add libxml2 dependency

* Fix variables name

* Rename variable and function name

* Move xxx_options.hpp out of internal directory

* Rename FromConnectionString -> CreateFromConnectionString
2020-06-04 13:21:08 +08:00