  Bound states in the continuum (BICs), a concept from quantum mechanics, are
ubiquitous physical phenomena where waves will be completely locked inside
physical systems without energy leaky. Such a physical phenomenon in optics
will provide a platform for optical mode confinement to strengthen local field
enhancement in nonlinear optics. Here we utilize an optical system consisting
of asymmetric nanogratings and waveguide of thin film lithium niobate (LiNbO3)
material to enhance second harmonic response near BICs. By breaking the
symmetry of grating periodicity, we realize strong local field confined inside
waveguide up to 25 times normalized to incident field (with dissymmetric factor
of 0.2), allowing strong light-matter interaction in nonlinear material. From
the numerical simulation, we theoretically demonstrate that such an optical
system can greatly enhance second harmonic intensity enhancement of about 104
compared with undersigned LiNbO3 film and conversion efficiency reaching
1.53e-5 for dissymmetric factor=0.2 under illumination of 1.33 GW/(suqare cm).
Surprisingly, we can predict that a giant enhancement of second harmonic
conversion efficiency will exceed 8.13e-5 for dissymmetric factor=0.1 when the
optical system is extremely close to BICs. We believe that such an optical
system to trap local field inside is also accessible to promote the application
of thin film lithium niobate in the field of integrated nonlinear optics.
