Doublebinlog for MySQL

Background

         In some projects that require very strict data safety, DBAs will except there is a solution to ensure the safety of binlog. That means after the worst case, OS crash or disk damage, there is a way to find out a whole copy of binlog.

        Doublebinlog patch is done for this requirement. It offers a switch to enable the behavior: any write and fsync operation on binary logs are duplicated in another file. So if the original binlog file is destroyed by disk damage, the bak file can be used.

 

Usage

         Two static variables and a dynamic variable:

       If “log-bin” is set “/log/mysql3306/binlog/binlog”, add two line under [mysqld] in my.cnf

    double_file_src_prefix = /log/mysql3306/binlog/binlog

    double_file_dst_dir    = /log/mysql3306/extra_binlog

 

    a) Be sure mysqld has the privilege of /log/mysql3306/extra_binlog

    b) "set global double_file_switch=1" can enable the strategy, default is off

c) After set double_file_switch=0, "flush logs" is recommended.

 

         The logic is that, if an file operation on a file and the filename has the prefix as double_file_src_prefix, all the operaion will duplicated in another file, which locates in the directory double_file_dst_dir.

 

Design

  In order to avoid modifying the logic, we hook in the file operations such as write/pwrite/fsync/lseek.

1)    When a file is open or create, we check whether it matches the prefix, if yes, the same operation is done in the duplicated file. This will return two file description (fd), src_fd defined as key, dest_fd defined as value, a hash table is used to save the relationship.

2)     When wrte/pwrite/fsync/lseek operate on an fd, search it in the hash table, if there is an item match, get the value as dest_fd, the same operation is done on it.

3)    When close(src_fd), we close(dest_fd) too, and then remove the relationship from hash table.

4)     It is confirmed that when doing fsync, duplicated file is done before original file, so if the server crash, duplicated file will be larger than original, which means the slave will not get “dirty” binary events.

 

 Patch File Download  (based on Percona Server 5518)