Q: Training a classifier on 2 catalogs combined I'm currently trying to train a machine to identify flashes of light in the sky as supernova (SN) or not. To do this, I'm combining 2 different catalogs:


*

*A big catalog (99% SN) with brighter flashes on average

*A small catalog (75% SN) with dimmer flashes on average


[Edit: To clarify, "brightness" is just a number capturing the astronomical magnitude of first detection. The data is in the form of numerical properties, not images]
The difference in brightness is just a sampling artefact. ALL examples (SN or non-SN) are brighter in the big catalog. However, right now the classifier can use "brightness" to figure out which catalog an example came from, and this strongly influences the probability that it is or isn't a supernova.
How can I avoid this problem? What's the best way to alter / pare down this dataset to eliminate statistical artefacts from features like "brightness"?

The above is a complete description of the problem. Below, I'll include my own two ideas on how to solve it; however, I'm not very happy with either of them.
1) Eliminate supernovae from the big catalog at random until the label proportion matches.
This seems like the simplest solution, but it would require changing the SN - non-SN ratio from 99-1 to about 3-1. This is roughly a 30-fold drop in supernova examples, which is not ideal!
2) Eliminate supernovae and non-supernovae from the big catalog selectively until the feature spaces are similar
In other words, drop brighter supernovae from the big catalog until the average SN brightness matches the small catalog's SN brightness. I'm not sure how to do this. Has this been done before? Is there an established algorithm for it?

A: What format is your data in? If it's pictures then and the relative brightness of the supernova compared to the background is the same then normalization/whiteninng will help you. You should probably be normalizing your data anyway since most ML models expect mean 0 data. 
