In reality the large list tends to have more than 2000 items in root folder. By default, creating folders in list is hidden from users. Most of the users won't bother/know to create new folders to store their data. I see customers' large lists easily have 50000 items without using any folders.
My test shows that you can work with list with 2 millions items in root folder, but performance is not good. To load a page with 100 item per page, it takes around 16 seconds.
The performance is acceptable for list has 100,000 items in root folder, the page load is less than 1 second. Of course, the number of item per page should be fewer than 2000 for better performance.
The performance degrade about linearly for list with items from 100,000 to 1 milliion. For example, 500,000 item list take about 5s to load in browser.
My hardward and software combination are
1. Intel Xeon CPU X5355 2.66GHz
2. Memory 10 GB
3. Windows 2008 R2 64bit
4. SharePoint 2007 SP2 + MS SQL 2008 SP2 single box installation.
The total memory consumption for a list with 2 millions items under root will be 4.5 GB. (The list is a default custom list with one column "Title")
The articles also says:
Performance for deleting items degrades significantly when a list becomes very large. Deleting a single item from a very large list takes much more time than deleting an item from a smaller list. In the test case, a single item was deleted from a site that was not under load. As the data shows, whether there was an indexed column or not, performance when changing list items degrades as the size of the list grows. It’s more likely that a batch process would need to be built to delete items during off-peak periods. If that is not an option, the performance of delete functionality alone could conceivably force you to abandon plans to use very large lists in Office SharePoint Server 2007.
The test results shows deleting item from a 100K list takes about 1 minutes, however in my test deleting an item from 2 million item list takes only 16 seconds.