Repository navigation
feat: shrink an idle adaptive TSI series-id-set cache - #27698
Open
davidby-influx wants to merge 1 commit into
Open
davidby-influx wants to merge 1 commit into
davidby-influx wants to merge 1 commit into
Conversation
An adaptively sized series-id-set cache only shrinks while it is being read, so a shard that stops being queried keeps the memory it grew into forever. The new option series-id-set-cache-idle-timeout (default 2h, 0 disables) fixes that. Once a shard's cache has served no reads for the timeout, it gives memory back in steps until it is back at series-id-set-cache-size. Any read stops the shrinking, and the cache grows again on demand. Shed entries are reported in a new idle_eviction statistic.
There was a problem hiding this comment.
🟡 Changes recommended
The changed exported constructor signature breaks existing downstream callers at compile time.
1 open finding
What changed in this PR
Adds idle shrinking to adaptive TSI series-ID-set caches so inactive shards release borrowed capacity.
Changes:
- Adds configurable idle timeout with validation and defaults.
- Implements idle sweeping, eviction statistics, and lifecycle handling.
- Adds policy, concurrency, integration, and configuration tests.
| File | Description |
|---|---|
tsdb/index/tsi1/index.go |
Wires timeout and sweeper lifecycle into TSI indexes. |
tsdb/index/tsi1/index_test.go |
Tests index-level shrinking and lifecycle behavior. |
tsdb/index/tsi1/export_test.go |
Exposes cache state to external tests. |
tsdb/index/tsi1/cache.go |
Implements idle shrinking and statistics. |
tsdb/index/tsi1/cache_test.go |
Extensively tests policy and concurrency behavior. |
tsdb/config.go |
Defines and validates the timeout setting. |
tsdb/config_test.go |
Tests timeout validation and TOML handling. |
etc/config.sample.toml |
Documents the new configuration option. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
davidby-influx
marked this pull request as ready for review
October 13, 2026 19:22
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An adaptively sized series-id-set cache only shrinks while it is being read, so a shard that stops being queried keeps the memory it grew into forever.
The new option series-id-set-cache-idle-timeout (default 2h, 0 disables) fixes that. Once a shard's cache has served no reads for the timeout, it gives memory back in steps until it is back at series-id-set-cache-size. Any read stops the shrinking, and the cache grows again on demand. Shed entries are reported in a new idle_eviction statistic.
Closes https://github.com/influxdata/feature-requests/issues/699