  Vast amounts of astronomical photometric data are generated from various
projects, requiring significant efforts to identify variable stars and other
object classes. In light of this, a general, widely applicable classification
framework would simplify the task of designing custom classifiers. We present a
novel deep learning framework for classifying light curves using a weakly
supervised object detection model. Our framework identifies the optimal windows
for both light curves and power spectra automatically, and zooms in on their
corresponding data. This allows for automatic feature extraction from both time
and frequency domains, enabling our model to handle data across different
scales and sampling intervals. We train our model on datasets obtained from
both space-based and ground-based multi-band observations of variable stars and
transients. We achieve an accuracy of 87% for combined variables and transient
events, which is comparable to the performance of previous feature-based
models. Our trained model can be utilized directly to other missions, such as
ASAS-SN, without requiring any retraining or fine-tuning. To address known
issues with miscalibrated predictive probabilities, we apply conformal
prediction to generate robust predictive sets that guarantee true label
coverage with a given probability. Additionally, we incorporate various anomaly
detection algorithms to empower our model with the ability to identify
out-of-distribution objects. Our framework is implemented in the Deep-LC
toolkit, which is an open-source Python package hosted on Github and PyPI.
