With the new tab list, it tries to expand to handle any number of players. The previous limit was 60.
Attached is a screenshot of what 200 players looks like.
This was discussed with Dinnerbone in IRC and either a hard-coded limit of 60 (the limit in 1.7.10 that displays names nicely) or a configurable limit were suggested.