A lesson in never letting your filesystem fill up too much

June 26th, 2008 by atomic

A good rule of thumb is that a DBA should not let the size of his/her database grow to much more than 50% of the available disk space. I’m sure everyone has a story of being burned in some way by not following this advice.

Before promoting 5.1 to some beefier hardware, I’ve got a few large tables sitting on a small SATA-based machine. Noticing that a few were growing faster than I expected, i decided to pause my loading and convert another one of them to the archive storage engine to save some space.

I certainly saved space.

By trying to do too much at one time, I had some other temp files lying around on the same filesystem, and my conversion to the archive storage engine failed at about 80% through with my disk full.

mysql> alter table w_stats engine = Archive;
ERROR 1030 (HY000): Got error -1 from storage engine

mysql 5.1 has an unfortunate bug that I discovered, and reported, that causes the unfinished archive files to remain after the failed conversion, and not the original data. My disk space problems are no longer much of a problem :)

Now I know that this was basically my fault, and that I should have been more careful with my system, but I would have preferred maybe an embarassing error message, a slap on the wrist, not the loss of a 150GB table! For example:

mysql> alter table w_stats engine = Archive;
ERROR 1030 (HY000):  df -h is your friend

Well, it sounded funny in my head at least…

Posted in mysql |

One Response

  1. Shlomo Priymak Says:

    Good to know, thanks for the heads-up!

    I personally don’t think it was your fault at all - this kind of a bug is a *serious* problem in the engine.
    On the other hand, if it were to do a full rollback, it would take a tremendous amount of time.

    Hope you had backups :)

Leave a Comment

Please note: Comment moderation is enabled and may delay your comment. There is no need to resubmit your comment.