* Improve link checking for github links
Use locally cloned repos to check github links to avoid rate-limiting.
* Add default clone path to the link checker template
---------
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
Http Status code 401 just be unauthorized and not necessarily that the link isn't valid. Removing this from the list of codes to error from.
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
* fixes
* ongoing
* ongoing
* add top-level throw/catches showing exception info
* fix handling of cases when there is 1 link and when there is no RetryAfter.Delta
* handle lack of Exception.Headers property
* handle gracefully obtaining status code from $_.Exception.InnerException.ErrorCode
---------
Co-authored-by: Konrad Jamrozik <kojamroz@microsoft.com>
We have hit a few 50x errors while pulling a page to get
links and that has caused the rest of the link checking to
terminate early. To fix that we switching to LogError which will
output an error in devops instead of Write-Error which terminates
immediately. We also add some retry count to the page retrieval
and cache file retrieval to help with these transitent 50x issues.
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
This will allow us to point our verify-link script at a raw
MD file in a github repo and have it parsed correctly for
links.
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
- Update link checking pipeline to enable caching
- Add loading cache file from a http endpoint
- Add retry logic when hitting 429
Fix verify link renaming scenarios
- Correctly error if link format is incorrect
- Handle case where the replace regex is not passed
- Improve the error logging so it is easier to
identify the broken links
* Make the user agent configurable.
* Address comments
* Default to current Chrome version
* Revert the yaml file changes
Co-authored-by: Sima Zhu <sizhu@microsoft.com>
