stitch preprocessing pipeline
Summary: 1. add call to binarization to complete preprocessing pipeline 2. add ability to specify task to select the dictionary, and add a bert task 3. Get rid of function calls that are no longer needed after moving functions from fairseq here Reviewed By: jingfeidu Differential Revision: D13977842 fbshipit-source-id: ec9bbb4e98e62e12c20ba68bb52b8bcc94aee91d
Loading
Please register or sign in to comment