replication updates

mysql> select INET_ATON("209.207.224.40");

       ->  3520061480

@end example

@findex MASTER_POS_WAIT()

@item MASTER_POS_WAIT(log_name, log_pos)

Blocks until the slave reaches the specified position in the master log during

replication. If master information is not initialized, returns NULL. If the

slave is not running, will block and wait until it is started and goes to or

past

the specified postion. If the slave is already past the specified postion,

returns immediately. The return value is the number of log events it had to

wait to get to the specified position, or NULL in case of error. Useful for

control of master-slave synchronization, but was originally written to

facilate replication testing.

@end table

@findex GROUP BY functions

* Replication Options::         Replication Options in my.cnf

* Replication SQL::             SQL Commands related to replication

* Replication FAQ::             Frequently Asked Questions about replication

* Troubleshooting Replication:: Troubleshooting Replication

@end menu

@node Replication Intro, Replication Implementation, Replication, Replication

internally. One server acts as the master, while the other acts as the

slave. Note that one server could play the roles of master in one pair

and slave in the other. The master server keeps a binary log of updates

and an index file to binary logs to keep track of log rotation.  The

slave upon connecting informs the master where it left off since the

last successfully propagated update, catches up on the updates, and then

blocks and waits for the master to notify it of the new updates.

(@xref{Binary log}.) and an index file to binary logs to keep track of

log rotation.  The slave, upon connecting, informs the master where it

left off since the last successfully propagated update, catches up on

the updates, and then blocks and waits for the master to notify it of

the new updates.

Note that if you are replicating a database, all updates to this

database should be done through the master!

@node Replication Implementation, Replication HOWTO, Replication Intro, Replication

@section Replication Implementation Overview

@strong{MySQL} internal replication uses the master-slave approach. One

server is designated as the master, while the other (or others) as

slave(s). The master keeps a binary log of updates. @xref{Binary log}.

The slave connects to the master, catches up on the missed updates, and

then starts receiving updates immediately as they come to the master. If

the connection is lost, the slave will reconnect. If the master goes

down, the slave will keep trying to connect every

@code{master-connect-retry} seconds until the master comes back up and

the connection can be established. The slave keeps track of where it

left off in the replication process, so it can use the information in case

it goes down and gets restarted later.

@strong{MySQL} replication is based on the server keeping track of all

changes to your database (updates, deletes, etc) in the binary

log. (@xref{Binary log}.) and the slave server(s) reading the saved

queries from the master server's binary log so that the slave can

execute the same queries on its copy of the data.

It is @strong{very important} to realize that the binary log is simply a

record starting from a fixed point in time (the moment you enable binary

logging). Any slaves which you set up will need copies of all the data

from your master as it existed the moment that you enabled binary

logging on the master. If you start your slaves with data that doesn't

agree with what was on the master @strong{when the binary log was

started}, your slaves may fail.

A future version (4.0) of @strong{MySQL} will remove the need to keep a

(possibly large) snapshot of data for new slaves that you might wish to

set up through the live backup functionality with no locking required.

However, at this time, it is necessary to block all writes either with a

global read lock or by shutting down the master while taking a snapshot.

Once a slave is properly configured and running, it will simply connect

to the master and wait for updates to process. If the master goes away

or the slave loses connectivity with your master, it will keep trying to

connect every @code{master-connect-retry} seconds until it is able to

reconnect and resume listening for updates.

Each slave keeps track of where it left off. The master server has no

knowledge of how many slaves there are or which ones are up-to-date at

any given time.

The next section explains the master/slave setup process in more detail.

@node Replication HOWTO, Replication Features, Replication Implementation, Replication

@section HOWTO

Below is a quick HOWTO on how to set up replication on your current

system:

Below is a quick description of how to set up complete replication on

your current @strong{MySQL} server. It assumes you want to replicate all

your databases and have not configured replication before. You will need

to shutdown your master server briefly to complete the steops outlined

below.

@itemize @bullet

@enumerate

@item

Upgrade both slave and master to Version 3.23.15 or higher. We recommend

that you always use the latest release of Version 3.23 on both the slave

and the master. The binary log format has also changed a couple of times

during development (before 3.23.29) so you should delete old binary logs

when upgrading @strong{MySQL}. In addition, the newer version will fix

some bugs and add new features. Please, do not report bugs until you

have verified that the problem is present in the latest release.

Make you have a recent version of @strong{MySQL} installed on the master

and slave(s).

Use Version 3.23.29 or higher. Previous releases used a different binary

log format and had bugs which have been fixed in newer releases. Please,

do not report bugs until you have verified that the problem is present

in the latest release.

@item

Set up special replication user(s) on the master with the @code{FILE}

Set up special a replication user on the master with the @code{FILE}

privilege and permission to connect from all the slaves. If the user is

only doing replication, you don't need to grant him other privileges.

only doing replication (recommended), you don't need to grant any

additional privileges.

For example, to create a user named @code{repl} which can access your

master from any host, you might use this command:

@example

GRANT FILE ON *.* TO repl@@"%" IDENTIFIED BY '<password>';

@end example

@item

Take a snapshot of all the tables/databases on the master that could

possibly be involved in the update queries before taking the next step.

Starting in Version 3.23.21, there is a command that allows you to take

a snapshot of a table on the master and copy it to the slave, called

@code{LOAD TABLE FROM MASTER}. Until Version 3.23.23, though, it has a

serious bug, and we recommend that you do not use it until you have

upgraded.

Shut down @strong{MySQL} on the master.

@example

mysqladmin -u root -p<password> shutdown

@end example

@item

Snapshot all the data on your master server.

The easiest way to do this (on Unix) is to simply use @strong{tar} to

produce an archvie of your entrie data directory. The exact data

directory location depends on your installation.

@example

tar -cvf /tmp/mysql-snapshot.tar /path/to/data-dir

@end example

Windows users can use WinZip or similar software to create an archive of

the data directory.

@item

In @code{my.cnf} on the master add @code{log-bin} and

@code{server-id=unique number} and restart it. Make sure there are no

important updates to the master between the time you have taken the

snapshot and the time the master is restarted with @code{log-bin}

option.

@code{server-id=unique number} to the @code{[mysqld]} section and

restart it. It is very important that the id of the slave is different from

the id of the master. Think of @code{server-id} as something similar

to the IP address - it uniquely identifies the server instance in the

comminity of replication partners.

@example

[mysqld]

log-bin

server-id=1

@end example

@item

Load the snapshot of the master to all the slaves.

Restart @strong{MySQL} on the master.

@item

Add the following to @code{my.cnf} on the slave(s):

master-host=<hostname of the master>

master-user=<replication user name>

master-password=<replication user password>

server-id=<some unique number between 1 and 2^32-1>

server-id=<some unique number between 2 and 2^32-1>

@end example

replacing the values in <> with what is relevant to your system.

@code{server-id} must be different for each server participating in

replication.  If you don't specify a server-id, it will be set to

1 if you have not defined @code{master-host}, else it will be set to 2.

replication.  If you don't specify a server-id, it will be set to 1 if

you have not defined @code{master-host}, else it will be set to 2. Note

that in the case of @code{server-id} omission the master will refuse

connections from all slaves, and the slave will refuse to connect to a

master. Thus, omitting @code{server-id} is only good for backup with a

binary log.

@item

Copy the snapshot data into your data directory on your slave(s). Make

sure that the privileges on the files and directories are correct. The

user which @strong{MySQL} runs as needs to be able to read and write to

them, just as on the master.

@item Restart the slave(s).

@end itemize

@end enumerate

After you have done the above, the master and the slave(s) should be in

sync.

After you have done the above, the slave(s) should connect to the master

and catch up on any updates which happened since the snapshot was taken.

If you have forgot to set server-id for the slave you will get the following

error in the error log file:

If you have forgotten to set @code{server-id} for the slave you will get

the following error in the error log file:

@example

Warning: one should set server_id to a non-0 value if master_host is set.

The server will not act as a slave.

@end example

If you have forgot to do this for the master, the slaves will not be able to

connect to the master.

If you have forgot to do this for the master, the slaves will not be

able to connect to the master.

If a slave is not able to replicate for any reason, you will find error

messages in the error log on the slave.

Once a slave is replicating, you will find a file called

@code{master.info} in the same directory as your error log. The

@code{master.info} file is used by the slave to keep track of how much

of the master's binary log is has processed. @strong{Do not} remove or

edit the file, unless you really know what you are doing. Even in that case,

it is preferred that you use @code{CHANGE MASTER TO} command.

@cindex options, replication

@cindex @code{my.cnf} file

Replication will be done correctly with @code{AUTO_INCREMENT},

@code{LAST_INSERT_ID}, and @code{TIMESTAMP} values.

@item

@code{RAND()} in updates does not replicate properly. Use

 @code{RAND(some_non_rand_expr)} if you are replcating updates with

@code{RAND()}. You can, for example, use @code{UNIX_TIMESTAMP()} for the

argument to @code{RAND()}.

@item

@code{LOAD DATA INFILE} will be handled properly as long as the file

still resides on the master server at the time of update

propagation. @code{LOAD LOCAL DATA INFILE} will be skipped.

something goes wrong and you want a clean start with @code{FLUSH MASTER}

and @code{FLUSH SLAVE} commands. In Version 3.23.26 we have renamed them to

@code{RESET MASTER} and @code{RESET SLAVE} respectively to clarify

what they do. The old @code{FLUSH} variants still work, though for

what they do. The old @code{FLUSH} variants still work, though, for

compatibility.

@item

Starting in Version 3.23.21, you can use @code{LOAD TABLE FROM MASTER} for

network backup and to set up replication initially.

network backup and to set up replication initially. We have recently

received a number of bug reports concerning it that we are investigating, so

we recommend that you use it only in testing until we make it more stable.

@item

Starting in Version 3.23.23, you can change masters with @code{CHANGE MASTER

TO}.

Starting in Version 3.23.23, you can change masters and adjust log position

with @code{CHANGE MASTER TO}.

@item

Starting in Version 3.23.23, you tell the master that updates in certain

databases should not be logged to the binary log with @code{binlog-ignore-db}.

@node Replication Options, Replication SQL, Replication Features, Replication

@section Replication Options in my.cnf

If you are using replication, we recommend you to use MySQL Version 3.23.28 or

If you are using replication, we recommend you to use MySQL Version 3.23.30 or

later. Older versions work, but they do have some bugs and are missing some

features.

@end multitable

@node Replication FAQ,  , Replication SQL, Replication

@node Replication FAQ,Troubleshooting Replication, Replication SQL, Replication

@section Replication FAQ

@cindex @code{Binlog_Dump}

all the updates, the slave will be able to catch up once it is up and

can connect.

We plan to make post 3.23.26 versions to be backwards compatible

for replication down to Version 3.23.26, so upgrade should be just a matter

of plug and play. Of course, as one joke goes, plug and play works

usually only 50% of the time - just the plug part. We hope to do much

better than that, though.

After 3.23.26, we have locked the replication protocol for modifications, so

you can upgrade masters and slave on the fly to a newer 3.23 version and you

can have different versions of @code{MySQL} running on the slave and the

master, as long as they are both newer than 3.23.26. 

@cindex replication, two-way

@strong{Q}: What issues should I be aware of when setting up two-way

can safely happen in any order, or unless you take care of mis-ordered

updates somehow in the client code.

Until we implement @code{server_id} variable, you cannot have more than

two servers in a co-master replication relationship, and you must

run @code{mysqld} without @code{log-slave-updates} (default) to avoid

infinite update loops.

You must also realize that two-way replication actually does not improve

performance very much, if at all, as far as updates are concerned. Both

So if N = 0, which means we have no replication, our system can handle

1200/11, about 109 writes per second (which means we will have 9 times

as many reads to the nature of our application).

as many reads due to the nature of our application).

If N = 1, we can get up to 184 writes per second.

system into @strong{MySQL}, but until it is ready, you will have to

create your own monitoring tools.

@node Troubleshooting Replication, ,Replication FAQ, Replication

@section Troubleshooting Replication

If you have followed the instructions, and your replication setup is not

working, first elliminate the user error factor by checking the following:

@itemize @bullet

@item

Is the master logging to the binary log? Check with @code{SHOW MASTER STATUS}.

If it is, @code{Position} will be non-zero. If not, verify that you have

given the master @code{log-bin} option and have set @code{server-id}.

@item

Is the slave running? Check with @code{SHOW SLAVE STATUS}. The answer is found

in @code{Slave_running} column. If not, verify slave options and check the

error log for messages.

@item

If the slave is running, did it establish connection with the master? Do

@code{SHOW PROCESSLIST}, find the thread with @code{system user} value in

@code{User} column and @code{none} in the @code{Host} column, and check the

@code{State} column. If it says @code{connecting to master}, verify the

privileges for the replication user on the master, master host name, your

DNS setup, whether the master is actually running, whether it is reachable

from the slave, and if all that seems ok, read the error logs.

@item

If the slave was running, but then stopped, check the error logs. It usually

happens when some query that succeeded on the master fails on the slave. This

should never happen if you have taken a proper snapshot of the master, and

never modify the data on the slave outside of the slave thread. If it does,

it is a bug, read below on how to report it.

@item

Make sure you are not running into an old bug by upgrading to the most recent

version.

@item

If all else fails, read the error logs. If they are big,

@code{grep -i slave /path/to/your-log.err} on the slave. There is no

generic pattern to search for on the master, as the only errors it logs

are general system errors - if it can, it will send the error to the slave

when things go wrong.

@end itemize

When you have determined that there is no user error involved, and replication

still either does not work at all or is unstable, it is time to start working

on a bug report. We need to get as much info as possible from you to be able

to track down the bug. Please do spend some time and effort preparing a good

bug report. Ideally, we would like to have a test case in the format found in

@code{mysql-test/t/rpl*} directory of the source tree. If you submit a test

case like that, you can expect a patch within a day or two in most cases,

although, of course, you mileage may vary depending on a number of factors.

Second best option is a just program with easily configurable connection

arguments for the master and the slave that will demonstrate the problem on our

systems. You can write one in Perl or in C, depending on which language you

know better.

If you have one of the above ways to demonstrate the bug, use

@code{mysqlbug} to prepare a bug report and send it to

@email{bugs@@lists.mysql.com}. If you have a phantom - a problem that

does occur but you cannot duplicate "at will":

@itemize @bullet

@item

Verify that there is no user error involved. For example, if you update the

slave outside of the slave thread, the data will be out of sync, and you can

have unique key violations on updates, in which case the slave thread will

stop and wait for you to clean up the tables manually to bring them in sync.

@item

Run slave with @code{log-slave-updates} and @code{log-bin} - this will keep

a log of all updates on the slave.

@item

Save all evidence before reseting the replication. If we have no or only

sketchy information, it would take us a while to track down the problem. The

evidence you should collect is:

@itemize @bullet

@item

all binary logs on the master

@item

all binary log on the slave

@item

the output of @code{SHOW MASTER STATUS} on the master at the time

you have discovered the problem

@item

the output of @code{SHOW SLAVE STATUS} on the master at the time

you have discovered the problem

@item

Error logs on the master and on the slave

@end itemize

@item

Use @code{mysqlbinlog} to examine the binary logs. The following should

be helpful

to find the trouble query, for example:

@example

mysqlbinlog -j pos_from_slave_status /path/to/log_from_slave_status | head

@end example

@end itemize

Once you have collected the evidence on the phantom problem, try hard to

isolate it into a separate test case first. Then report the problem to

@email{bugs@@lists.mysql.com} with as much info as possible.

@cindex performance, maximizing

@cindex optimization

though, so Version 3.23 is not released as a stable version yet.

@menu

* News-3.23.32::                Changes in release 3.23.32

* News-3.23.31::                Changes in release 3.23.31

* News-3.23.30::                Changes in release 3.23.30

* News-3.23.29::                Changes in release 3.23.29

* News-3.23.0::                 Changes in release 3.23.0

@end menu

@node News-3.23.31, News-3.23.30, News-3.23.x, News-3.23.x

@node News-3.23.32, News-3.23.31, News-3.23.x, News-3.23.x

@appendixsubsec Changes in release 3.23.32

@itemize @bullet

@item

Added MASTER_POS_WAIT()

@end itemize

@node News-3.23.31, News-3.23.30, News-3.23.32, News-3.23.x

@appendixsubsec Changes in release 3.23.31

@itemize @bullet

@item