IMAP - IOERROR : Cannot allocate memory

[Andy Kopciuch]

Hi :

I have a client who contacted me with problems on their Kolab 3.4 server.
It seems as though they can not create / rename folders any longer.

They sent me this snippet from the log :

<mail.log>
Sep 13 10:52:24 mail imap[32214]: IOERROR: mapping /var/lib/imap/mailboxes.db.NEW file: Cannot allocate memory
Sep 13 10:52:24 mail imap[32214]: Fatal error: failed to mmap /var/lib/imap/mailboxes.db.NEW file
Sep 13 10:52:24 mail imap[32214]: twoskip: /var/lib/imap/mailboxes.db closed while still locked
Sep 13 10:52:24 mail master[2168]: process type:SERVICE name:imap path:/usr/lib/cyrus-imapd/imapd age:56.459s pid:32214 exited, status 75
</mail.log>


I have no idea what is going on here. There is virtually no information I can find
that directly relates to this. This closest I found was this relating to logging region :

https://www.novell.com/support/kb/doc.php?id=3377220

This was relating ro dovecot, seemed like a parallell issue :
http://www.dovecot.org/list/dovecot/2012-August/085148.html

This seemed relevant to mupdate, but with the same error message
https://lists.andrew.cmu.edu/pipermail/info-cyrus/2005-September/019660.html

--

I did some experimenting, and even added a DB_CONFIG to /var/lib/imap/db, and set the
cache size to 250MB ... that still did not fix the problem.

This looks like some kind of resource limit has been reached ... but which one?
And how does it get increased.

Any help or advice on this?



Thanks,



Andy

[m***@chrisfleming.org]

Post by Andy Kopciuch
I have a client who contacted me with problems on their Kolab 3.4 server.
It seems as though they can not create / rename folders any longer.
<mail.log>
Sep 13 10:52:24 mail imap[32214]: IOERROR: mapping /var/lib/imap/mailboxes.db.NEW file: Cannot allocate memory
Sep 13 10:52:24 mail imap[32214]: Fatal error: failed to mmap /var/lib/imap/mailboxes.db.NEW file
Sep 13 10:52:24 mail imap[32214]: twoskip: /var/lib/imap/mailboxes.db closed while still locked
Sep 13 10:52:24 mail master[2168]: process type:SERVICE name:imap path:/usr/lib/cyrus-imapd/imapd age:56.459s pid:32214 exited, status 75
</mail.log>

Having a look at the source code I think the code generating the error
is:



*base = (char *)mmap((caddr_t)0, newlen, PROT_READ, flags, fd, 0L);
if (*base == (char *)MAP_FAILED) {
syslog(LOG_ERR, "IOERROR: mapping %s file%s%s: %m", name,
mboxname ? " for " : "", mboxname ? mboxname : "");
snprintf(buf, sizeof(buf), "failed to mmap %s file", name);
fatal(buf, EC_IOERR);
}


This means that the mmap system call is failing. I would check actual free
memmory - On my server the imapd process never gets big. But I would start
looking at overall memory use.

Cheers
Chris

Post by Andy Kopciuch
I have no idea what is going on here. There is virtually no information I can find
https://www.novell.com/support/kb/doc.php?id=3377220
http://www.dovecot.org/list/dovecot/2012-August/085148.html
This seemed relevant to mupdate, but with the same error message
https://lists.andrew.cmu.edu/pipermail/info-cyrus/2005-September/019660.html
--
I did some experimenting, and even added a DB_CONFIG to /var/lib/imap/db, and set the
cache size to 250MB ... that still did not fix the problem.
This looks like some kind of resource limit has been reached ... but which one?
And how does it get increased.
Any help or advice on this?
Thanks,
Andy
_______________________________________________
users mailing list
https://lists.kolab.org/mailman/listinfo/users

[Andy Kopciuch]

Post by m***@chrisfleming.org
Having a look at the source code I think the code generating the error
*base = (char *)mmap((caddr_t)0, newlen, PROT_READ, flags, fd, 0L);
if (*base == (char *)MAP_FAILED) {
syslog(LOG_ERR, "IOERROR: mapping %s file%s%s: %m", name,
mboxname ? " for " : "", mboxname ? mboxname : "");
snprintf(buf, sizeof(buf), "failed to mmap %s file", name);
fatal(buf, EC_IOERR);
}
This means that the mmap system call is failing. I would check actual free
memmory - On my server the imapd process never gets big. But I would start
looking at overall memory use.
Cheers
Chris

Yes,

I found some information eventually, and something in the cyrus release notes
for version 2.5.1 that there was a memory leak involving mmap that had been
fixed.

However, kolab 3.4 has the custom cyrus 2.5, and this obviously is not (or
expected) to be back ported.

This memory leak seems to be trigger when generating a mailboxes database to
disk when using the twoskip format. Up until cyrus 2.4, the default format
for databases was skiplist, and in 2.5 it was changed to twoskip (next gen
twoskip implementation it seems).

After some investigation, and experimentation I found that I could convert the
database from twoskip into flat, or skiplist (or other supported formats) with
the conversion utility. However ... any conversion from another format
_into_ twoskip would result in the same mmap error during conversion.

So I solved this by converting mailboxes.db back into the previous skiplist
format, and changed the config mailboxes_db = skiplist option in imap.conf.

There have been no issues since.


thanks,


Andy