Skip to content
Commit d1dc66d9 authored by Ruty Rinott's avatar Ruty Rinott Committed by Facebook Github Bot
Browse files

optimizations for token_block_dataset

Summary:
optimizing memory use of token_block_dataset by replacing python data structures with numpy arrays.
applying needed parts from D13498973, instead of rebasing it on changes

Reviewed By: edunov

Differential Revision: D13678485

fbshipit-source-id: c0c827a8b95834a6a5456476040ebdc8e42136d4
parent cefe3f8a
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment