You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Online read: columnar store return and proto-direct to_arrow to cut client CPU on wide reads #7072
For wide online reads against the Redis store (4 feature views, ~90 features, 100 to 500 entities per call) the fetch is now GIL-free with client: glide (#6870), but most of get_online_features wall time is still Python after the fetch:
online_read returns one dict[str, ValueProto] per entity, built value by value.
_get_online_features walks those into GetOnlineFeaturesResponse one value at a time.
OnlineResponse.to_arrow() is pa.Table.from_pydict(self.to_dict()), so it costs more than to_dict, not less.
get_feature_service(allow_cache=True) rebuilds the FeatureService from proto on every call, about 2 ms per request.
Measured locally (Apple silicon, Valkey 9.1, feast 0.67.0, 1500 entities x 11 feature views, service of 90 features from 4 views, p50 ms):
k
SDK client: glide + proto walk to Arrow
SDK to_arrow()
raw GLIDE HMGET fetch only
fetch + per-column decode straight to Arrow
100
20
18
3.6
6.7
500
106
112
18
33
1500
316
330
53
96
100, one busy Python thread
143
99
n/a
70
The fetch is about 17% of the SDK read. The same shape holds on the precompute_online=True fast path: one GET per entity, yet 34 to 72 ms at k=360 to 500 in an earlier run because response assembly dominates.
Proposal
An optional columnar return on OnlineStore, e.g. online_read_columns(...) -> dict[feature, list[bytes | None]] plus per-view timestamps, with the base class falling back to today's online_read. Redis can fill it straight from the pipeline replies with no per-value proto objects.
OnlineResponse.to_arrow() builds one pa.array per column from response.proto instead of going through to_dict.
Key the registry_proto_cache decorators in proto_registry_utils on the registry's version_id instead of id(registry_proto). Every registry refresh produces a new proto object, so today each refresh rebuilds every FeatureService from proto (about 35 ms for 14 services, most of it typeguard checks on Field.__init__), and get_feature_service has no object cache at all (about 2 ms per read).
A note on the tail: at k=500 the precompute_online fast path measured p50 63 ms with the default collector and 34 ms after gc.freeze(), because each read allocates tens of thousands of proto wrappers and trips gen2 collections. Not a Feast change, but worth a line in the feature server performance tuning guide.
Each is backwards compatible. I have working drafts of 2 and 3 (3 measured at 35 ms to 0.1 ms per refresh) and can open PRs if the direction is agreed. Fetch-side repro: see the script on #6856.
Correction on proposal 2: I measured a proto-direct to_arrow() (one WhichOneof + getattr per value into a typed pa.array per column) against today's from_pydict(to_dict()) on a 90-column response and it is a wash, 21.1 vs 20.7 ms at 500 rows, 60.6 vs 60.9 ms at 1500. The cost is the per-value proto wrapper access, not feast_value_type_to_python_type, so skipping the converter buys nothing. I am not sending that PR. The gain is only available if the store returns raw bytes per column (proposal 1) so the decode can run column-wise.
Hey @chandlerok,
About your note on the tail, where gc.freeze() took p50 from 63 to 34 ms: we see the same thing in the feature server. There it shows up as a 50–100 ms tail on online reads. I don't think a docs line is enough for feast serve, because today there is no way to turn it on. Feast starts gunicorn from its own code, so gunicorn config files and hooks are not used, and Python has no env var for gc.freeze(). So it needs a small code change. I opened #7088 for it, with repro steps and an opt-in --gc-freeze flag.
Activity
chandlerok commentedon Oct 26, 2026
Opened #7073 for proposal 3.
Correction on proposal 2: I measured a proto-direct
to_arrow()(oneWhichOneof+getattrper value into a typedpa.arrayper column) against today'sfrom_pydict(to_dict())on a 90-column response and it is a wash, 21.1 vs 20.7 ms at 500 rows, 60.6 vs 60.9 ms at 1500. The cost is the per-value proto wrapper access, notfeast_value_type_to_python_type, so skipping the converter buys nothing. I am not sending that PR. The gain is only available if the store returns raw bytes per column (proposal 1) so the decode can run column-wise.perf: Stamp SQL and Snowflake registry protos with a content-hash ver…
aborgatin commentedon Oct 29, 2026
Hey @chandlerok,
About your note on the tail, where
gc.freeze()took p50 from 63 to 34 ms: we see the same thing in the feature server. There it shows up as a 50–100 ms tail on online reads. I don't think a docs line is enough for feast serve, because today there is no way to turn it on. Feast startsgunicornfrom its own code, sogunicornconfig files and hooks are not used, and Python has no env var forgc.freeze(). So it needs a small code change. I opened #7088 for it, with repro steps and an opt-in--gc-freezeflag.