Mutt can cache headers of messages so they need to be downloaded just once. This greatly improves speed when opening folders again later. Simply create an empty directory and add
set header_cache = "/path/to/folder"
to your configuration. For this to work, mutt needs to have header cache support (1.5.7 or later) which must be enabled at compile-time.
Starting with mutt 1.5.12, it can also locally cache messages. As a result, each message needs to be downloaded only once. Furthermore, all operations on messages (like searching and limiting) can use the local copy. Since it is automatically enabled when building mutt with POP and/or IMAP support, all that needs to be configured is
set message_cachedir = "/path/to/folder"
The very same directory can also be used for header caching.