Iteratively building a classifier requires a mix of skill, diagnostic ability and guesswork. I'll lay out a framework that helps you build reliable classifiers with greater confidence and less random guesswork. Tools demonstrated will include sklearn, YellowBrick, Shapley and pandas_profiling.
Yellowbrick is a suite of visual diagnostic tools called "Visualizers" that extend the Scikit-Learn API to allow human steering of the model selection process. In a nutshell, Yellowbrick combines scikit-learn with matplotlib in the best tradition of the scikit-learn documentation, but to produce visualizations for your models! For more on Yellowbrick, please see the About.