  Machine programming (MP) is an emerging field at the intersection of
deterministic and probabilistic computing, and it aims to assist software and
hardware engineers, among other applications. Along with powerful compute
resources, MP systems often rely on vast amount of open-source code to learn
interesting properties about code and programming and solve problems in the
areas of debugging, code recommendation, auto-completion, etc. Unfortunately,
several of the existing MP systems either do not consider quality of code
repositories or use atypical quality measures than those typically used in
software engineering community to select them. As such, impact of quality of
code repositories on the performance of these systems needs to be studied.
  In this preliminary paper, we evaluate impact of different quality
repositories on the performance of a candidate MP system. Towards that
objective, we develop a framework, named GitRank, to rank open-source
repositories on quality, maintainability, and popularity by leveraging existing
research on this topic. We then apply GitRank to evaluate correlation between
the quality measures used by the candidate MP system and the quality measures
used by our framework. Our preliminary results reveal some correlation between
the quality measures used in GitRank and ControlFlag's performance, suggesting
that some of the measures used in GitRank are applicable to ControlFlag. But it
also raises questions around right quality measures for code repositories used
in MP systems. We believe that our findings also generate interesting insights
towards code quality measures that affect performance of MP systems.
