Q: Apache Pig How to whitelist or blacklist in Load function? I am wondering if it's possible to maintain a whitelist or blacklist in Pig's Load function. Say I am doing the following:
AllData = LOAD '/path/to/dir/CAT*' USING AvroStorage();

This would load all the files that starts with the CAT prefix.
e.g. CAT1, CAT2, CAT3, CAT4, CAT5, CAT6

I am wondering if it's possible to maintain a blacklist to filter out let's say CAT2, CAT3 or to maintain a whitelist that keeps CAT1, CAT4, CAT5, CAT6 only. Thanks!

A: You can do a whitelist by listing all the filename suffixes in curly brackets, like:
AllData = LOAD '/path/to/dir/CAT{1,4,5,6}' USING AvroStorage();

