Skip to content

archive-index: make wait-time for cache download configurable #3068

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
syphar wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

archive-index: make wait-time for cache download configurable #3068

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions src/config.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,8 @@ pub struct Config {
// So we can preallocate the DashMap with this number to avoid resizes.
pub(crate) local_archive_cache_expected_count: usize,

pub(crate) local_archive_cache_download_wait_ms: u64,

// Where to collect metrics for the metrics initiative.
// When empty, we won't collect metrics.
pub(crate) compiler_metrics_collection_path: Option<PathBuf>,
Expand Down Expand Up @@ -233,6 +235,10 @@ impl Config {
"DOCSRS_ARCHIVE_INDEX_EXPECTED_COUNT",
100_000usize,
)?)
.local_archive_cache_download_wait_ms(env(
"DOCSRS_ARCHIVE_INDEX_DOWNLOAD_WAIT_MS",
1000,
)?)
.compiler_metrics_collection_path(maybe_env("DOCSRS_COMPILER_METRICS_PATH")?)
.temp_dir(temp_dir)
.rustwide_workspace(env(
Expand Down
8 changes: 4 additions & 4 deletions src/storage/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -582,12 +582,11 @@ impl AsyncStorage {
// Someone else is already downloading/repairing. Don't queue on write(); just wait
// a bit and poll the fast path until it becomes readable or we give up.
const STEP_MS: u64 = 10;
const ATTEMPTS: u64 = 50; // = 500ms total wait
const TOTAL_WAIT_MS: u64 = STEP_MS * ATTEMPTS;
let attempts = self.config.local_archive_cache_download_wait_ms / STEP_MS;

let mut last_err = None;

for _ in 0..ATTEMPTS {
for _ in 0..attempts {
sleep(Duration::from_millis(STEP_MS)).await;

match archive_index::find_in_file(local_index_path.clone(), path_in_archive).await {
Expand All @@ -603,7 +602,8 @@ impl AsyncStorage {
Err(last_err
.unwrap_or_else(|| anyhow!("archive index unavailable after repair wait"))
.context(format!(
"no archive index after waiting for {TOTAL_WAIT_MS}ms"
"no archive index after waiting for {} ms",
self.config.local_archive_cache_download_wait_ms
)))
}

Expand Down
Loading