<p>Another good introductory book is "Foundations of Machine Learning" by Mohri et al.: <a href="https://rads.stackoverflow.com/amzn/click/com/0262039400" rel="noreferrer" rel="nofollow noreferrer">https://www.amazon.com/Foundations-Machine-Learning-Mehryar-Mohri/dp/0262039400/</a>.  It has a large overlap with the Shai and Shai book, but also quite a bit of content that they don't cover.</p>

<p>There are also good books and surveys on more advanced or specialized topics:</p>

<ul>
<li>Szepesvári and Lattimore have a nice book on bandits:
<a href="https://tor-lattimore.com/downloads/book/book.pdf" rel="noreferrer">https://tor-lattimore.com/downloads/book/book.pdf</a></li>
<li>Cesa-Bianchi and Lugosi have a difficult but rewarding book on
learning and games: <a href="https://rads.stackoverflow.com/amzn/click/com/0521841089" rel="noreferrer" rel="nofollow noreferrer">https://www.amazon.com/Prediction-Learning-Games-Nicolo-Cesa-Bianchi/dp/0521841089/</a></li>
<li>Schapire and Freund have a wonderful book on boosting: <a href="https://mitpress.mit.edu/books/boosting" rel="noreferrer">https://mitpress.mit.edu/books/boosting</a></li>
</ul>

<p>There are more, but I hope this is a good start.</p>

CC BY-SA 4.0
2020-04-17T18:19:15.100Z
Thank you this seems like a great list! I'm thinking it would be best to additionally work on computational complexity, high dimensional probability, and convex optimization. This also seems to confirm my suspicion that there are no more 'general' books on the subject after this. Please let me know if you have anymore tips on learning this subject.  Additionally it would be nice to know about how far I am from being able to read and understand journals like COLT.
PMaynard
CC BY-SA 4.0
2020-04-17T18:23:36.737Z
For complexity, you should look up Arora/Barak.  For various results in high dimensional probability, the "data science" book by Blum, Hopcroft, and Kannan is good: https://www.cs.cornell.edu/jeh/book.pdf.  For convex optimization, books and surveys by Vempala are good.

As for reading COLT/ALT/JMLR papers, some are easier than others, and you can learn some topics before others.  There's no reason not to try reading the papers as soon as you want and to consult books and references as needed.
Lev Reyzin
CC BY-SA 4.0
2020-04-17T18:29:34.513Z
Good idea to maybe start with some more approachable papers earlier on. Thanks again for the book recommendations (funny you should mention it Arora/Barak is on my nightstand right now).
PMaynard
CC BY-SA 4.0
2020-04-17T18:55:48.103Z
For high-dimensional probability, the... "High-Dimensional Probability" book by Vershynin is really good. https://www.math.uci.edu/~rvershyn/papers/HDP-book/HDP-book.html @PMaynard
Clement C.
CC BY-SA 4.0
2020-04-21T03:25:39.647Z
*Foundations* is excellent.  I was about to comment with this and then immediately saw your answer.  Its treatment is quite mathematically mature, in my opinion.
Max von Hippel
CC BY-SA 4.0
2020-04-17T17:32:21.243Z
2020-04-17T17:32:21.243Z
Lev Reyzin
<p>I have a list of references (incomplete) that may interest you: <a href="https://kiranvodrahalli.github.io/links/#resources-notes-textbooks-monographs-classes-etc" rel="nofollow noreferrer">https://kiranvodrahalli.github.io/links/#resources-notes-textbooks-monographs-classes-etc</a></p>

<p>(second all the existing suggestions). </p>

CC BY-SA 4.0
2020-04-17T21:04:11.487Z
2020-04-17T21:04:11.487Z
kiranv
<p>I would recommend the book:</p>

<ul>
<li><a href="https://cs.nyu.edu/~mohri/mlbook/" rel="nofollow noreferrer">Foundations of Machine Learning</a> from Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar.</li>
</ul>

CC BY-SA 4.0
2020-04-17T17:26:42.010Z
2020-04-17T17:26:42.010Z
Borja Rodríguez Gàlvez
<p>Pretty soon I will be finishing up Understanding Machine Learning by Shai Ben-David and Shai Shalev-Shwartz. I absolutely love the subject and want to learn more, the only issue is I'm having trouble finding a book that could come after this. Ultimately, my goal is to read papers in <a href="http://proceedings.mlr.press" rel="nofollow noreferrer">JMLR's COLT</a>.</p>

<ol>
<li>Is there a book similar to Understanding Machine Learning that would progress my knowledge further and would go well after reading UML?</li>
<li>Is there any other materials (not a book) that could allow me to learn more or prepare me for reading a journal like the one mentioned above?</li>
</ol>

<p>(also taking courses in this is not really an option so this will be for self-study)</p>

<p>This question was also asked <a href="https://ai.stackexchange.com/q/20355/2444">here</a> on AI SE from suggestion of comments.</p>

CC BY-SA 4.0
2020-04-17T16:51:32.043Z
See e.g., this: https://cstheory.stackexchange.com/a/46574/13319 (the Kearns-Vazirani book is a bit old, but still very relevant).
Clement C.
CC BY-SA 4.0
2020-04-17T16:53:26.153Z
These recent [lecture notes on Statistical Learning Theory](http://maxim.ece.illinois.edu/teaching/SLT/) (organized as a single, organized file) by Bruce Hajek and Maxim Raginsky may also be helpful.
Clement C.
CC BY-SA 4.0
2020-04-18T00:21:35.603Z
PMaynard, I didn't realize that you posted this question elsewhere.  Our site policy prohibits simultaneous crossposting.
Lev Reyzin
CC BY-SA 4.0
2020-04-18T05:05:46.487Z
@LevReyzin My understanding is that the crosspost on AI.SE followed the suggestion in the first comment (it wasn't crossposted prior to it). (Also, nbro, not sure what the explicit rules are, but on Math.SE, Mathoverflow (and here) it's at the very least frowned upon.)
Clement C.
CC BY-SA 4.0
2020-04-19T01:03:54.133Z
@nbro, as a moderator, you should know that different stackexchange sites have their own site-specific rules. You should probably also not go to other stackexchanges encouraging people to ignore their policies because you "disagree" with them.

Our policy is spelled out here: https://cstheory.meta.stackexchange.com/questions/225/official-faq-for-theoretical-computer-science/231#231.

I will not close this post because we already have upvoted answers and because PMaynard is a new user here.
Lev Reyzin
CC BY-SA 4.0
2020-04-20T15:33:01.740Z
We should probably make our policy clearer.  Hours are not what we had in mind as the right amount of time to wait.  Also, we do routinely close cross-posted questions.
Lev Reyzin
<reference-request><machine-learning><lg.learning>
Reference Request: Computational Learning Theory
PMaynard
# Reference Request: Computational Learning Theory

(Asked by: username_0 on 2020-04-17)

Pretty soon I will be finishing up Understanding Machine Learning by Shai Ben-David and Shai Shalev-Shwartz. I absolutely love the subject and want to learn more, the only issue is I'm having trouble finding a book that could come after this. Ultimately, my goal is to read papers in [JMLR's COLT](http://proceedings.mlr.press).

1. Is there a book similar to Understanding Machine Learning that would progress my knowledge further and would go well after reading UML?
2. Is there any other materials (not a book) that could allow me to learn more or prepare me for reading a journal like the one mentioned above?

(also taking courses in this is not really an option so this will be for self-study)

This question was also asked [here](https://ai.stackexchange.com/q/20355/2444) on AI SE from suggestion of comments.

**Tags:** reference-request, machine-learning, lg.learning
> **Comments:**
> **username_1**: See e.g., this: https://cstheory.stackexchange.com/a/46574/13319 (the Kearns-Vazirani book is a bit old, but still very relevant).  
> **username_1**: These recent [lecture notes on Statistical Learning Theory](http://maxim.ece.illinois.edu/teaching/SLT/) (organized as a single, organized file) by Bruce Hajek and Maxim Raginsky may also be helpful.  
> **username_2**: PMaynard, I didn't realize that you posted this question elsewhere.  Our site policy prohibits simultaneous crossposting.  
> **username_1**: @LevReyzin My understanding is that the crosspost on AI.SE followed the suggestion in the first comment (it wasn't crossposted prior to it). (Also, nbro, not sure what the explicit rules are, but on Math.SE, Mathoverflow (and here) it's at the very least frowned upon.)  
> **username_2**: @nbro, as a moderator, you should know that different stackexchange sites have their own site-specific rules. You should probably also not go to other stackexchanges encouraging people to ignore their policies because you "disagree" with them.  Our policy is spelled out here: https://cstheory.meta.stackexchange.com/questions/225/official-faq-for-theoretical-computer-science/231#231.  I will not close this post because we already have upvoted answers and because PMaynard is a new user here.  
> **username_2**: We should probably make our policy clearer.  Hours are not what we had in mind as the right amount of time to wait.  Also, we do routinely close cross-posted questions.  

---

## 3 answers

### Answer by username_2 on 2020-04-17. Score: 6 | Accepted answer

Another good introductory book is "Foundations of Machine Learning" by Mohri et al.: [https://www.amazon.com/Foundations-Machine-Learning-Mehryar-Mohri/dp/0262039400/](https://rads.stackoverflow.com/amzn/click/com/0262039400). It has a large overlap with the Shai and Shai book, but also quite a bit of content that they don't cover.

There are also good books and surveys on more advanced or specialized topics:

* Szepesvári and Lattimore have a nice book on bandits:
  <https://tor-lattimore.com/downloads/book/book.pdf>
* Cesa-Bianchi and Lugosi have a difficult but rewarding book on
  learning and games: [https://www.amazon.com/Prediction-Learning-Games-Nicolo-Cesa-Bianchi/dp/0521841089/](https://rads.stackoverflow.com/amzn/click/com/0521841089)
* Schapire and Freund have a wonderful book on boosting: <https://mitpress.mit.edu/books/boosting>

There are more, but I hope this is a good start.

> **Comments:**
> **username_0**: Thank you this seems like a great list! I'm thinking it would be best to additionally work on computational complexity, high dimensional probability, and convex optimization. This also seems to confirm my suspicion that there are no more 'general' books on the subject after this. Please let me know if you have anymore tips on learning this subject.  Additionally it would be nice to know about how far I am from being able to read and understand journals like COLT.  
> **username_2**: For complexity, you should look up Arora/Barak.  For various results in high dimensional probability, the "data science" book by Blum, Hopcroft, and Kannan is good: https://www.cs.cornell.edu/jeh/book.pdf.  For convex optimization, books and surveys by Vempala are good.  As for reading COLT/ALT/JMLR papers, some are easier than others, and you can learn some topics before others.  There's no reason not to try reading the papers as soon as you want and to consult books and references as needed.  
> **username_0**: Good idea to maybe start with some more approachable papers earlier on. Thanks again for the book recommendations (funny you should mention it Arora/Barak is on my nightstand right now).  
> **username_1**: For high-dimensional probability, the... "High-Dimensional Probability" book by Vershynin is really good. https://www.math.uci.edu/~rvershyn/papers/HDP-book/HDP-book.html @PMaynard  
> **username_3**: *Foundations* is excellent.  I was about to comment with this and then immediately saw your answer.  Its treatment is quite mathematically mature, in my opinion.  

---

### Answer by username_4 on 2020-04-17. Score: 4

I have a list of references (incomplete) that may interest you: <https://kiranvodrahalli.github.io/links/#resources-notes-textbooks-monographs-classes-etc>

(second all the existing suggestions).


---

### Answer by username_5 on 2020-04-17. Score: 2

I would recommend the book:

* [Foundations of Machine Learning](https://cs.nyu.edu/~mohri/mlbook/) from Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar.


---

