Commit Graph

48 Commits

Author SHA1 Message Date
JinmingHu
b9299478ee
[Storage Blob Service] Concurrent upload for Page Blob (#486) 2020-08-20 10:54:28 +08:00
JinmingHu
4c5286803b
[Storage Blob Service] ORS (#467) 2020-08-19 09:13:13 +08:00
Kan Tang
e0bb037cdd
Fixed test failure due to network latency too low. (#466) 2020-08-18 11:13:39 +08:00
JinmingHu
42ffbc853e
[Storage Blobs Service] Seal Append Blob (#465)
* seal append blob

* copy seal append blob
2020-08-18 09:55:12 +08:00
Kan Tang
3042f69d5b
Rename return types for Datalake and file services. (#464) 2020-08-18 09:28:31 +08:00
JinmingHu
1dce74c642
[Storage Blobs Service] Undelete Container (#462)
* Undelete Container
2020-08-18 09:11:59 +08:00
JinmingHu
24959f30da
[Storage Blobs Service] Rename Results and Options (#460)
* rename UploadFrom

* Rename CommitBlockListResult/Options GetBlockListResult/Options

* Rename DownloadBlobTo{Options/Result} UploadBlockBlobFrom{Options/Result}
2020-08-17 10:06:58 +08:00
JinmingHu
86a22901e9
[Storage Blobs Service] Transactional MD5 and CRC64 (#456)
* MD5

* Crc64

* crc64 concatenate
2020-08-16 00:30:49 +08:00
JinmingHu
069e359562
Rename return types and option types (#449)
* rename return types and option types

* define ConcurrentUploadBlockBlobFromBufferOptions and ConcurrentUploadBlockBlobFromFileOptions

* use std::move

* Rename DownloadTo

* update doc

* Resolved datalake build issues.

* Resolved some cmake issue when linking

Co-authored-by: Tank Tang <kat@microsoft.com>
2020-08-14 13:02:10 +08:00
Kan Tang
d77ecf7917
Added support for share service (#450) 2020-08-14 13:01:51 +08:00
JinmingHu
2027d5102f
add test cases for base64 encode/decode, hash wrapper (#431) 2020-08-12 10:13:16 +08:00
JinmingHu
68643e3ee2
[Storage Blobs Service] Add test cases for LastModifiedTime and ETag and lease id (#425)
* add test cases for LastModifiedTime and ETag and lease id
2020-08-11 10:40:59 +08:00
JinmingHu
8372d39cc1
[Storage Blobs Service] Customer-Provided Key (CPK-V) and Encryption Scope (CPK-N) (#427)
* cpk-v and cpk-n
2020-08-11 09:45:35 +08:00
JinmingHu
cc0ff27eec
[Storage Blobs Service] Only make one HTTP request if the blob size is small for Concurrent Upload API (#419)
* Only make one HTTP request if the blob size is small for Concurrent Upload API
2020-08-08 08:44:03 +08:00
JinmingHu
c700b7811c
[Storage Blobs Service] Blob Versioning (#409)
* blob versioning
2020-08-07 12:14:26 +08:00
JinmingHu
355184ef4f
[Storage Blob Service] Container Lease and Blob Lease (#402)
* container lease

* blob lease
2020-08-06 13:50:15 +08:00
Kan Tang
46c0af9a70
Added service prefix to file names. (#399) 2020-08-06 13:09:02 +08:00
JinmingHu
52dbc6b2f2
Pascal case for acronyms (#391) 2020-08-05 10:32:03 +08:00
Kan Tang
626a8856f0
Added support for get/set file service properties. (#384) 2020-08-05 10:31:48 +08:00
Kan Tang
3de57667d4
Added support for create/delete file shares. (#382)
* Added support for create/delete file shares.

* Removed nullable for metadata.
2020-08-04 09:54:04 +08:00
JinmingHu
8867c7abcb
[Storage Blob Service] Get/SetBlobContainerAccessPolicy (#381)
* Get/SetBlobContainerAccessPolicy

* Remove ResetPermissions()
2020-08-04 09:53:34 +08:00
JinmingHu
22f4ba12fc
GetAccountInfo (#371) 2020-08-03 10:32:43 +08:00
JinmingHu
b9212922e7
GetBlobServiceProperties and SetBlobServiceProperties (#366) 2020-07-30 16:39:06 +08:00
JinmingHu
cbec5c6af6
Add unit test cases that require SAS token (#359) 2020-07-29 17:42:14 +08:00
JinmingHu
b0b5a18d18
User Delegation SAS (#358) 2020-07-29 16:11:21 +08:00
JinmingHu
b0facd3dd5
adapt to breaking change in Azure Core (#356)
* adapt to breaking change in Azure Core

* Adapt breaking change for Datalake service.

* Adapt to the breaking change

* Adapt breaking change for Datalake service.

* fix ut

* Fixed UT.

Co-authored-by: Tank Tang <kat@microsoft.com>
2020-07-29 14:55:53 +08:00
JinmingHu
9d1566c5ea
Add Account SAS and Blob SAS (#353)
* Add Account SAS and Blob SAS

* add doc for sas

* remove Nones

* remove Nones
2020-07-28 13:43:04 +08:00
Kan Tang
8568327a56
Refined DataLake sample (#336)
* Refined DataLake sample

* Added DirectoryClient::GetFileClient, resolved some comment and test failures.

* Resolved an issue introduced by lowercasing response headers resulting in error code/message not being passed, and refined sample.

* Refined the structure according to comments.

* Changed buffer initialziation to stack allocated.

* Resolved some further comments.
2020-07-23 15:40:16 +08:00
JinmingHu
29f5e30ad3
integrate TelemetryPolicy into Blob Service (#328)
* integrate TelemetryPolicy into Blob Service

* Also add telemetry support to ADLS Gen 2.

Co-authored-by: Tank Tang <kat@microsoft.com>
2020-07-23 15:39:51 +08:00
Kan Tang
b0b348e9cb
Resolved API review comments and added FileClient and DirectoryClient… (#308)
* Added API to extract raw response.

* Resolved API review comments and added FileClient and DirectoryClient for DataLake.

* Added some blob/adls Gen2 interop functionality.

* Resolved some issues/comments

* Adds more interop, support retry and concurrent upload/download.

* Added support for Azure Core's token credential to support bearer token credential.

* Added integration on RawResponse/Response<T> and resolved some test issues.

* Added validation for Client secret authentication.

* Resolved some review comments and resolved CI issue.

* Resolved a UT failure in Azure Core.

* Resolved some further comments.
2020-07-22 16:03:44 +08:00
JinmingHu
4d19f3b078
Integrate Response<T> into Storage Blobs Service (#327)
* Revert changes to storage code

* adapt to breaking changes in Azure::Core, Response->RawResponse, pipeline clone returns unique_ptr

* just to make build pass, remove this commit after https://github.com/Azure/azure-sdk-for-cpp/pull/326 is merged

* adapt to Response<T>

* concurrent upload/download also return Response<T>

* add BlobInfo and BlobContainerInfo
2020-07-21 10:31:00 +08:00
JinmingHu
0f37994e44
add perf benchmark code (#319)
* Add multithread perf testcase

* add large scale upload testcase

* Fix compiler error on GCC 9

* Fix failed testcase
2020-07-17 11:16:18 +08:00
JinmingHu
204c025d09
suggested changes in API review (#312)
* suggested changes in API review

* Rename UrlBuilder->UriBuilder

* add perf test case

* Integrate retrypolicy into blob clients

* remove `Content-Length` header for GET/HEAD/DELETE http requests

* Remove temp files in ut
2020-07-16 00:49:55 -07:00
JinmingHu
3db59439c9
[Storage Blobs Service] ListBlobsByHierarchy (#265)
* add API ListBlobsByHierarchy

* fix compiler error on GCC 8

* Update Blob API version to 2019-12-12

* use lowercase header key
2020-07-15 10:45:22 +08:00
Kan Tang
a1ec5119a6
Datalake full feature support PR. (#257)
* Added test cases.

* Adapt stream breaking change and adding usage for Nullable.

* Resolve CI issue.

* Resolved comments from review.

* Integrate access condition support and resolve more comments.

* Resolved some further comments.

* Resolved some further comments and removed GetMetadata.
2020-07-13 05:31:02 -07:00
JinmingHu
83406f23dd
Storage Blob Service Access Conditions, Bitwise enum class (#252)
* access conditions

* bitwise enum class

* Update x-ms-date on each retry

* add a header for all constant strings

* fix some ut

* rename ModifiedTimeAccessConditons->LastModifiedTimeAccessConditions
2020-07-09 21:03:32 -07:00
JinmingHu
95aa6361ef
Concurrent upload block blob from file, concurrent download blob to file (#247)
* concurrent download to file

* concurrent upload block blob from file

* update sample

* Change default download chunk size from 8MB to 4MB

* fix CI
2020-07-08 13:26:22 +08:00
JinmingHu
66c7518dce
Suggested changes in API review (#244)
* Rename Properties->HttpHeaders

* Rename BlobXxxItems->Items

* Remove MaxResults in response of list operations

* specify time format in comments for GetUSerDelegationKey API

* per operation pipelines and per retry pipelines

* Define some response types and rename ListBlobsFlat

* Add different return types for some APIs

* Rename BlobDownloadInfo->BlobDownloadResponse

* Assign default value to BlobContentLength to supporess compiler warnings

* add concurrent download to buffer

* concurrent upload block blob from buffer
2020-07-07 10:54:05 +08:00
Victor Vazquez
9fb4119ccb
Memory Stream - File Stream (win/lin) - HTTP Request / Response - Updates (#236)
* make stream use int64_t instead of uint64_t
2020-07-03 22:22:16 -07:00
JinmingHu
e61c3ff51a
Documentation for BlockBlobClient and Blob options (#235)
* Documentation for blob options

* Rename struct UploadBlobOptions -> UploadBlockBlobOptions

* documentation for BlockBlobClient
2020-07-02 10:01:27 +08:00
JinmingHu
74ce80659b
Documentation for Storage Blob Service, Nullable, signed int, and others (#213)
* use Nullable

* use C++ style cast

* Add documentation

* use signed int instead of unsigned

* disable storage ut

* add timeout in protocol layer

* enable stroage ut

* undelete test case

* Support copy status in properties

* Remove default value for SequenceNumber in CreatePageBlobOptions
2020-06-30 03:03:42 -07:00
Victor Vazquez
2877c959e1
Enable storage unit tests (#227)
* enable core unit tests to be compiled
* Avoid running core unit tests in CI
2020-06-29 14:19:46 -07:00
Victor Vazquez
d58cbad14d
Unique ptr for body stream (#214)
* change bodyStream to unique ptr
2020-06-29 10:33:31 -07:00
Anton Kolesnyk
ad2e63afe1
Remake ClientSecretCredential (#212) 2020-06-27 15:27:11 -07:00
JinmingHu
f74011a2e0
Add Unittest for Blob Service, other fixes and enhancements found during validation (#181)
* Give AccessTier a default value to eliminate warning

* Add blob clients conversion functions

* Fix bug in shared key authentication

* Add GetUri().

* make lease-status and lease-state optional

* Check duplicates in metadata

* add tests for blob service

* AppendBlocb doesn't support AccessTier

* fix bug in "RangeFromXml"

* redefine PageRange in convenience layer

* suppress warnings from gtest

* add API AppendBlockFromUri

* connection string parsing

* Add memorystream

* Rename connection string constants

* Adapt to new bodystream, this will be rewritten when bosystream is changed to unique_ptr

* fix compiler errors

* Change MemoryStream interface

* samples framework

* Add newline at the end of file

* fix compiler error
2020-06-24 01:17:09 -07:00
Kan Tang
a8a200ef26
Added convenience layer for datalake service (#178) 2020-06-20 19:55:41 -07:00
Kan Tang
18408f848a
Adding ADLS gen2 protocol layer code. (#153)
* Adding test framework for azure storage.

* Adding ADLS gen2 protocol layer code.
2020-06-09 06:31:12 -07:00
Kan Tang
6e01498038
Adding test framework for azure storage. (#152) 2020-06-09 02:45:11 -07:00