  The treatment and criteria for development of unstable Roche lobe overflow
(RLOF) that leads to the common envelope (CE) phase have hindered the
evolutionary predictions for decades. In particular, the formation of black
hole-black hole (BH-BH), black hole-neutron star (BH-NS), and neutron
star-neutron star (NS-NS) merging binaries depends sensitively on the CE phase
in classical isolated binary evolution model. All these mergers are now
reported as LIGO/Virgo sources or source candidates. CE is even considered by
some as a mandatory phase in the formation of BH-BH, BH-NS or NS-NS mergers in
binary evolution. At the moment, there is no full first-principles model for
development of CE. We employ the Startrack population synthesis code to test
the current advancements in studies on stability of RLOF for massive donors to
assess their effect on LIGO/Virgo source population. In particular, we allow
for more restrictive CE development criteria for massive donors. We also test a
modified condition for switching between different types of stable mass
transfer. Implemented modifications significantly influence basic properties of
merging double compact objects, sometimes in non-intuitive way. For one of
tested models with restricted CE development criteria local merger rate density
for BH-BH systems increased due to emergence of a new dominant formation
scenario without any CE phase. We find that the changes in highly uncertain
assumptions on RLOF physics may significantly affect (i) local merger rate
density, (ii) shape of the mass and mass ratio distributions, and (iii)
dominant evolutionary formation (with and without CE) scenarios of LIGO/Virgo
sources. Our results demonstrate that without sufficiently strong constraints
on RLOF physics, one is not able to draw fully reliable conclusions about the
population of double compact object systems based on population synthesis
studies.
