Bellok
trying to allow ingesting in the background. Current throughput on ingest is 17+ documents per secon, that means only 25k documents out of over a million will load within HuggingFace's required time limit. If we want to have access to all 1.5m documents, we need to run the ingest in the background and allow Warbler-CDA to report healthy before the time-limit is up.
d90df2b