<p>Genetic Programming (GP) has been used to modify source code for performance improvement.</p>

<p>e.g. <a href="http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=5688317" rel="nofollow">Evolutionary Improvement of Programs</a></p>

<p>(This is close to my PhD topic, would be very interested to hear your approach)</p>

CC BY-SA 3.0
2012-03-15T00:28:23.613Z
2012-03-15T00:28:23.613Z
Brendan Cody-Kenny
<p>I am looking into ML for generating more efficient code (i.e. compile time and run time heuristics). I have a phd (compilers, hpc), but very little ML experience. </p>

<p>I would appreciate any references to existing work.</p>

<p>More importantly, which Machine Learning techniques should i be exploring ?</p>

<ul>
<li>each data sample is a code + test run + performance data</li>
<li>large number of data samples, with correct labeling</li>
<li>large number of parameters to modify </li>
<li>any sample can be rerun with any parameters (and get correct labeling)</li>
<li>large amount of cpu to run and re run samples</li>
</ul>

<p>And also some of my very suspect assumptions...</p>

<ul>
<li>much of the code is already locally-optimal (basic compiler optimization)</li>
<li>some of the parameters are "high level concepts" e.g. replace array with linked list. (so a small parameter change will have varied effects)</li>
<li>the desired improvement is bounded (a xK improvement is "good enough", K is well known)</li>
<li>the parameters are sparse i.e. mostly zero (i suspect that most code is basically good, and there are relatively few beneficial changes)</li>
</ul>

<p>Any pointers to set me on my way would be appreciated </p>

CC BY-SA 3.0
2012-03-13T13:57:54.170Z
Sounds like you are asking for references to any papers in compiler design.  There are many.
Tsuyoshi Ito
CC BY-SA 3.0
2012-03-13T14:44:00.243Z
Absolutely not! I looking at heuristics which are completely outside compiler design. These heuristic constitute a very large problem space, and cannot be be easily analyzed even in runtime, hence ML.
OA1
CC BY-SA 3.0
2012-03-13T17:38:43.923Z
you should probably go to metaoptimize.com. This question is out of scope here.
Suresh Venkat
CC BY-SA 3.0
2012-03-13T17:49:44.283Z
I realized that “ML” in the question does not mean [what I thought it meant](http://en.wikipedia.org/wiki/ML_%28programming_language%29).
Tsuyoshi Ito
CC BY-SA 3.0
2012-03-13T20:25:56.493Z
@TsuyoshiIto I would say that these technics could be landed into Virtual Machine, not the compiler itself
om-nom-nom
CC BY-SA 3.0
2012-03-14T15:29:57.293Z
@TsuyoshiIto funny: it never even occurred to me to interpret ML the way you did :)
Suresh Venkat
CC BY-SA 3.0
2012-03-15T02:04:57.767Z
I agree with Suresh, this does not look like a theory question.
Kaveh
CC BY-SA 3.0
2012-03-15T08:32:11.517Z
Since you asked for "*Any pointers to set me on my way*". 1. [Google](http://www.google.com/search?q=machine%20learning%20compiler%20optimization&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a) 2. [Google Scholar](http://scholar.google.com/scholar?q=machine%20learning%20compiler%20optimization&hl=en&as_sdt=0&as_vis=1&oi=scholart&sa=X&ei=RqdhT_WxMIGXigKXxISCCQ&ved=0CB0QgQMwAA) 3. [Statistical machine learning techniques in compiler design](http://www.crcnetbase.com/doi/abs/10.1201/9781420043839.
Vijay D
<reference-request><optimization><machine-learning><compilers>
machine learning for code and compiler optimization?
OA1
# machine learning for code and compiler optimization?

(Asked by: username_0 on 2012-03-13)

I am looking into ML for generating more efficient code (i.e. compile time and run time heuristics). I have a phd (compilers, hpc), but very little ML experience.

I would appreciate any references to existing work.

More importantly, which Machine Learning techniques should i be exploring ?

* each data sample is a code + test run + performance data
* large number of data samples, with correct labeling
* large number of parameters to modify
* any sample can be rerun with any parameters (and get correct labeling)
* large amount of cpu to run and re run samples

And also some of my very suspect assumptions...

* much of the code is already locally-optimal (basic compiler optimization)
* some of the parameters are "high level concepts" e.g. replace array with linked list. (so a small parameter change will have varied effects)
* the desired improvement is bounded (a xK improvement is "good enough", K is well known)
* the parameters are sparse i.e. mostly zero (i suspect that most code is basically good, and there are relatively few beneficial changes)

Any pointers to set me on my way would be appreciated

**Tags:** reference-request, optimization, machine-learning, compilers
> **Comments:**
> **username_1**: Sounds like you are asking for references to any papers in compiler design.  There are many.  
> **username_0**: Absolutely not! I looking at heuristics which are completely outside compiler design. These heuristic constitute a very large problem space, and cannot be be easily analyzed even in runtime, hence ML.  
> **username_2**: you should probably go to metaoptimize.com. This question is out of scope here.  
> **username_1**: I realized that “ML” in the question does not mean [what I thought it meant](http://en.wikipedia.org/wiki/ML_%28programming_language%29).  
> **username_3**: @TsuyoshiIto I would say that these technics could be landed into Virtual Machine, not the compiler itself  
> **username_2**: @TsuyoshiIto funny: it never even occurred to me to interpret ML the way you did :)  
> **username_4**: I agree with Suresh, this does not look like a theory question.  
> **username_5**: Since you asked for "*Any pointers to set me on my way*". 1. [Google](http://www.google.com/search?q=machine%20learning%20compiler%20optimization&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a) 2. [Google Scholar](http://scholar.google.com/scholar?q=machine%20learning%20compiler%20optimization&hl=en&as_sdt=0&as_vis=1&oi=scholart&sa=X&ei=RqdhT_WxMIGXigKXxISCCQ&ved=0CB0QgQMwAA) 3. [Statistical machine learning techniques in compiler design](http://www.crcnetbase.com/doi/abs/10.1201/9781420043839.  

---

## 1 answers

### Answer by username_6 on 2012-03-15. Score: 3

Genetic Programming (GP) has been used to modify source code for performance improvement.

e.g. [Evolutionary Improvement of Programs](http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=5688317)

(This is close to my PhD topic, would be very interested to hear your approach)


---

