- 浏览: 1319230 次
- 性别:
- 来自: 广州
文章分类
- 全部博客 (351)
- Java General (37)
- .net General (2)
- Linux Toy (55)
- Oracle (81)
- Mysql (11)
- Programer Career (12)
- Oh, my living ! (2)
- Shell Script (8)
- Web Service (0)
- Linux Server (22)
- Php/Python/Perl (3P) (2)
- Javascript General (5)
- Saleforce Apex Dev (2)
- Web General (5)
- Xen & VM tech. (17)
- PSP (13)
- OpenSolaris (34)
- php (1)
- RAI/flex/action script (16)
- asterisk/CTI (7)
- 交互设计 (6)
- English (3)
- Lucene (1)
最新评论
-
GuolinLee:
markmark
JVM调优总结 -Xms -Xmx -Xmn -Xss -
di1984HIT:
写的太好啊。
JVM调优总结 -Xms -Xmx -Xmn -Xss -
javajdbc:
javajdbc 写道
JVM调优总结 -Xms -Xmx -Xmn -Xss -
javajdbc:
...
JVM调优总结 -Xms -Xmx -Xmn -Xss -
alvin198761:
非常感谢,国外的被封杀了,你这里还有一份
How to Convert An Image-Based Guest To An LVM-Based Guest
Mysql Cluster: The definitive HOWTO
This guide was written over a year ago and is an excellent introduction however it has not been updated since! If you have any questions please email me or Buy my book (US).
Introduction
You MUST have a third server as a managment node but this can be shut down after the cluster starts. Also note that I do not recommend shutting down the managment server (see the extra notes at the bottom of this document for more information). You can not run a mysql cluster with just two servers And have true redundancy.
You should also disable SELinux or use the following rules (many thanks to Robin Bowes for letting me know of these):
allow mysqld_t port_t:tcp_socket name_connect; allow mysqld_t var_lib_t:file append; allow mysqld_t var_lib_t:sock_file create; allow mysqld_t var_lib_t:file read; allow mysqld_t var_lib_t:sock_file unlink; allow mysqld_t var_lib_t:file { getattr write };
Although it is possible to set the cluster up on two physical servers you WILL NOT GET the ability to "kill" one server and for the cluster to continue as normal. For this you need a third server running the managment node.
I am going to talk about three servers,
mysql1.domain.com 192.168.0.1 mysql2.domain.com 192.168.0.2 mysql3.domain.com 192.168.0.3
Servers 1 and 2 will be the two that end up "clustered". This would be perfect for two servers behind a loadbalancer or using round robin DNS and is a good replacement for replication. Server 3 needs to have only minor changes made to it and does NOT require a mysql install. It can be a low-end machine and can be carrying out other tasks.
STAGE 1: Install mysql on the first two servers:
Complete the following steps on both mysql1 and mysql2:
groupadd mysql useradd -g mysql mysql cd /usr/local/ wget http://dev.mysql.com/get/Downloads/MySQL-5.0/mysql-max-5.0.15-linux-i686-glibc23.tar.gz /from/http://mirror.trouble-free.net/mysql_mirror/ tar -zxvf mysql-max-5.0.15-linux-i686-glibc23.tar.gz rm mysql-max-5.0.15-linux-i686-glibc23.tar.gz ln -s mysql-max-5.0.15-linux-i686-glibc23 mysql cd mysql scripts/mysql_install_db --user=mysql chown -R root . chown -R mysql data chgrp -R mysql . cp support-files/mysql.server /etc/rc.d/init.d/ chmod +x /etc/rc.d/init.d/mysql.server chkconfig --add mysql.server
Do not start mysql yet.
STAGE 2: Install and configure the managment server
You need the following files from the bin/ of the mysql directory: ndb_mgm and ndb_mgmd. Download the whole mysql-max tarball and extract them from the bin/ directory.
mkdir /usr/src/mysql-mgm cd /usr/src/mysql-mgm http://dev.mysql.com/get/Downloads/MySQL-4.1/mysql-max-5.0.15-pc-linux-gnu-i686.tar.gz/ from/http://www.signal42.com/mirrors/mysql/ tar -zxvf mysql-max-5.0.15-pc-linux-gnu-i686.tar.gz rm mysql-max-5.0.15-pc-linux-gnu-i686.tar.gz cd mysql-max-5.0.15-pc-linux-gnu-i686 mv bin/ndb_mgm . mv bin/ndb_mgmd . chmod +x ndb_mg* mv ndb_mg* /usr/bin/ cd rm -rf /usr/src/mysql-mgm
You now need to set up the config file for this managment:
mkdir /var/lib/mysql-cluster cd /var/lib/mysql-cluster vi [or emacs or any other editor] config.ini
Now, insert the following (changing the bits as indicated):
[NDBD DEFAULT] NoOfReplicas=2 [MYSQLD DEFAULT] [NDB_MGMD DEFAULT] [TCP DEFAULT] # Managment Server [NDB_MGMD] # the IP of THIS SERVER HostName=192.168.0.3 # Storage Engines [NDBD] # the IP of the FIRST SERVER (STORAGE NODE) HostName=192.168.0.1 DataDir= /var/lib/mysql-cluster [NDBD] # the IP of the SECOND SERVER (STORAGE NODE) HostName=192.168.0.2 DataDir=/var/lib/mysql-cluster # 2 MySQL Clients # I personally leave this blank to allow rapid changes of the mysql clients; # you can enter the hostnames of the above two servers here. I suggest you dont. [MYSQLD] [MYSQLD]
Now, start the managment server:
ndb_mgmd
This is the mysql managment server, not maganment console. You should therefore not expect any output (we will start the console later).
STAGE 3: Configure the storage/SQL servers and start mysql
On each of the two storage/SQL servers (192.168.0.1 and 192.168.0.2) enter the following (changing the bits as appropriate):
vi /etc/my.cnf
Enter i to go to insert mode again and insert this on both servers (changing the IP address to the IP of the managment server that you set up in stage 2):
[mysqld] ndbcluster # the IP of the MANAGMENT (THIRD) SERVER ndb-connectstring=192.168.0.3 [mysql_cluster] # the IP of the MANAGMENT (THIRD) SERVER ndb-connectstring=192.168.0.3
Be aware that anything in the [mysql_cluster] section will override the defaults in [mysql], so if you introduce a nodeid and then try to run multiple daemons on the same machines you should be aware of this!
Now, we make the data directory and start the storage engine:
mkdir /var/lib/mysql-cluster cd /var/lib/mysql-cluster /usr/local/mysql/bin/ndbd --initial /etc/rc.d/init.d/mysql.server start
If you have done one server now go back to the start of stage 3 and repeat exactly the same procedure on the second server.
NOTE that you should ONLY use --initial if you are either starting from scratch or have changed the config.ini file on the managment.
STAGE 4: Check its working
You can now return to the managment server (mysql3) and enter the managment console:
/usr/local/mysql/bin/ndb_mgm
Enter the command SHOW to see what is going on. A sample output looks like this:
[root@mysql3 mysql-cluster]# /usr/local/mysql/bin/ndb_mgm -- NDB Cluster -- Management Client -- ndb_mgm> show Connected to Management Server at: localhost:1186 Cluster Configuration --------------------- [ndbd(NDB)] 2 node(s) id=2 @192.168.0.1 (Version: 5.0.15, Nodegroup: 0, Master) id=3 @192.168.0.2 (Version: 5.0.15, Nodegroup: 0) [ndb_mgmd(MGM)] 1 node(s) id=1 @192.168.0.3 (Version: 5.0.15) [mysqld(API)] 2 node(s) id=4 (Version: 5.0.15) id=5 (Version: 5.0.15) ndb_mgm>
If you see
not connected, accepting connect from 192.168.0.[1/2/3]
in the first or last two lines they you have a problem. Please email me with as much detail as you can give and I can try to find out where you have gone wrong and change this HOWTO to fix it.
If you are OK to here it is time to test mysql. On either server mysql1 or mysql2 enter the following commands: Note that we have no root password yet.
mysql use test; CREATE TABLE ctest (i INT) ENGINE=NDBCLUSTER; INSERT INTO ctest () VALUES (1); SELECT * FROM ctest;
You should see 1 row returned (with the value 1).
If this works, now go to the other server and run the same SELECT and see what you get. Insert from that host and go back to host 1 and see if it works. If it works then congratulations.
The final test is to kill one server to see what happens. If you have physical access to the machine simply unplug its network cable and see if the other server keeps on going fine (try the SELECT query). If you dont have physical access do the following:
ps aux | grep ndbd
You get an output like this:
root 5578 0.0 0.3 6220 1964 ? S 03:14 0:00 ndbd root 5579 0.0 20.4 492072 102828 ? R 03:14 0:04 ndbd root 23532 0.0 0.1 3680 684 pts/1 S 07:59 0:00 grep ndbd
In this case ignore the command "grep ndbd" (the last line) but kill the first two processes by issuing the command kill -9 pid pid:
kill -9 5578 5579
Then try the select on the other server. While you are at it run a SHOW command on the managment node to see that the server has died. To restart it, just issue
ndbd
NOTE no
--inital!
Further notes about setup
I strongly recommend that you read all of this (and bookmark this page). It will almost certainly save you a lot of searching.
The Managment Server
I strongly recommend that you do not stop the managment server once it has started. This is for several resons:
- The server takes hardly any server resources
- If a cluster falls over, you want to be able to just ssh in and type ndbd to stat it. You dont want to have to start messing around with another server
- If you want to take backups then you need the managment server up
- The cluster log is sent to the management server so to check what is going on in the cluster or has happened since last this is an important tool
- All commands from the ndb_mgm client is sent to the management server and thus no management commands without management server.
- The managment server is required in case of cluster reconfiguration (crashed server or network split). In the case that it is not running, "split-brain" scenario will occure. The management server arbitration role is required for this type of setup to provide better fault tollerance.
However you are welcome to stop the server if you prefer.
Starting and stopping ndbd automatically on boot
To achieve this, do the following on both mysql1 and mysql2:
echo "ndbd" > /etc/rc.d/init.d/ndbd chmod +x /etc/rc.d/init.d/ndbd chkconfig --add ndbd
Note that this is a really quick script. You ought really to write one that at least checks if ndbd is already started on the machine.
Use of hostnames
You will note that I have used IP addresses exclusively throught this setup. This is because using hostnames simply increases the number of things that can go wrong. Mikael Ronström of MySQL AB kindly explains: "Hostnames certainly work with MySQL Cluster. But using hostnames introduces quite a few error sources since a proper DNS lookup system must be set-up, sometimes /etc/hosts must be edited and their might be security blocks ensuring that communication between certain machines is not possible other than on certain ports". I strongly suggest that while testing you use IP addresses if you can, then once it is all working change to hostnames.
RAM
Use the following formula to work out the amount of RAM that you need on each storage node:
(Size of database * NumberofReplicas * 1.1) / Number of storage nodes
NumberofReplicas is set to two by default. You can change it in config.ini if you want. So for example to run a 4GB database with NoOfReplicas set to 2 you need just under 9GB of RAM in total (4 * 2 * 1.1), so if you had two storage nodes you would need 4.5GB ram per storage node. For the SQL nodes and managment nodes you dont need much RAM at all.Bear in mind that if you have variable-width fields in MySQL Cluster 4.0 or 5.0 you will find that you will need a LOT more RAM than this formula predicts.
Note: A lot of people have emailed me querying the maths above! Remember that the cluster is fault tollerant, and each piece of data is stored on at least 2 nodes. (2 by default, as set by NumberOfReplicas). So you need TWICE the space you would need just for one copy, multiplied by 1.1 for overhead.
Adding storage nodes
If you decide to add storage nodes, bear in mind that 3 is not an optimal numbers. If you are going to move from two (above) then move to 4.
Adding SQL nodes
If you want to add another SQL node (i.e. you have another server that you want to add to the cluster but you dont need it to act as a storage node), then just add the following to /etc/my.cnf on the server (it must be a mysql-max server):
[mysqld] ndbcluster # the IP of the MANAGMENT (THIRD) SERVER ndb-connectstring=192.168.0.3 [mysql_cluster] # the IP of the MANAGMENT (THIRD) SERVER ndb-connectstring=192.168.0.3
Then you need to make sure that there is another [MYSQLD] line at the end of config.ini on the managment server. Restart the cluster (see below for an important note) and restart mysql on the new API. It should be connected.
Important note on changing config.ini
If you ever change config.ini you must stop the whole cluster and restart it to re-read the config file. Stop the cluster with a SHUTDOWN command to the ndb_mgm package on the managment server and then restart all the storage nodes.
Some useful configuration options that you will need if you have large tables:
DataMemory: defines the space available to store the actual records in the database. The entire DataMemory will be allocated in memory so it is important that the machine contains enough memory to handle the DataMemory size. Note that DataMemory is also used to store ordered indexes. Ordered indexes uses about 10 bytes per record. Default: 80MB
IndexMemory The IndexMemory is the parameter that controls the amount of storage used for hash indexes in MySQL Cluster. Hash indexes are always used for primary key indexes, unique indexes, and unique constraints. Default: 18MB
MaxNoOfAttributes This parameter defines the number of attributes that can be defined in the cluster. Default: 1000
MaxNoOfTables Obvious (bear in mind that each BLOB field creates another table for various reasons so take this into account). Default: 128
View this page at mysql.com for further information about the things you can put in the [NDBD] section of config.ini
发表评论
-
A sample to update mysqm column charecter set
2010-10-21 12:58 1281mysql> SHOW CHARACTER SET LI ... -
MySQL dba Script
2009-10-17 11:56 1843MySQL DBA scripts The followin ... -
使用MySQL的23个注意事项
2009-09-26 16:55 11841.如果客户端和服务器端的连接需要跨越并通过不可信任的网络,那 ... -
linux shell 下 用 mysql的中文问题
2009-09-18 10:30 2750虽然很少有人用linux下的mysql 客户端来维护数据,毕竟 ... -
mysql 打竖显示结果
2008-05-28 10:31 1542加个\G在最后现就可以了. mysql> select ... -
dump data into txt file
2008-05-10 15:42 1456mysql> select * from Directo ... -
Using Master/Slave Replication with ReplicationCon
2008-04-28 15:51 125025.4.4.6. Using Master/Slav ... -
How To Control Mysql Replication
2008-04-28 15:46 1671How To Control Mysql Replicatio ... -
Implementing High Availability in MySQL
2008-04-16 08:04 2348MySQL provides a built-in data ... -
innodb data file per table
2008-04-16 08:04 2148One very interesting thing I no ...
相关推荐
《JavaScript: The Definitive Guide, 5th Edition》是JavaScript学习领域中一本经典的参考书,由David Flanagan撰写,全面且深入地介绍了JavaScript语言。这本书面向已经有一定编程基础的读者,旨在帮助他们掌握...
`Hadoop: The Definitive Guide`中可能会讲解如何创建、读取和操作HDFS上的文件,以及如何配置HDFS参数以优化性能。 MapReduce是Hadoop处理大数据的主要计算模型,它将大规模数据处理任务分解为小的“映射”和...
Spark: The Definitive Guide: Big Data Processing Made Simple 1st Edition Spark: The Definitive Guide: Big Data Processing Made Simple 1st Edition Spark: The Definitive Guide: Big Data Processing Made ...
With the latest edition of this comprehensive resource, you’ll learn how to use Apache Hadoop to build and maintain reliable, scalable, distributed systems. It’s ideal for programmers looking to ...
《PNG:The Definitive Guide》这本书深入解析了PNG图像格式的各个方面,是理解、创建和优化PNG图像的重要资源。以下是关于PNG格式的一些关键知识点: 1. **PNG格式的起源**:PNG格式是在GIF图像格式受到专利限制后...
MongoDB: The Definitive Guide MongoDB is a powerful, flexible, and scalable generalpurpose database. It combines the ability to scale out with features such as secondary indexes, range queries, ...
Hadoop: The Definitive Guide, 4th Edition Get ready to unlock the power of your data. With the fourth edition of this comprehensive guide, you’ll learn how to build and maintain reliable, scalable,...
Put everthing you need to know about HTML & XHTML at your fingertips. For nearly a decade, hundreds of thousands of web developers have turned to HTML & XHTML: The Definitive Guide to master ...
在给定文件内容中提及的《Closure: The Definitive Guide》是由Michael Bolin撰写,涵盖了Google开发的Closure工具集,这个工具集包括了Closure Library、Closure Templates、Closure Compiler以及Closure Testing ...
[英文文字版]As the title suggests, HTTP: The Definitive Guide explains the HTIP protocol: how it works and how to use it co develop web-based applications. However, this book is not just about HTIP; ...
### HBase权威指南知识点概述 #### 一、引言与背景 - **大数据时代的来临**:随着互联网技术的发展,人类社会产生了前所未为的数据量。这些数据不仅数量巨大,而且种类繁多,传统的数据库系统难以应对这样的挑战。...
《Spark: The Definitive Guide: Big Data Processing Made Simple》是大数据处理领域的经典著作,由Databricks的创始人之一Michael Armbrust等专家撰写。这本书深入浅出地介绍了Apache Spark的核心概念、架构以及...
CSS(The Cascading Style Sheets, 层叠样式表)是构建网页和用户界面的一种样式表语言,它定义了如何在浏览器中显示HTML文档,从而实现内容与表现的分离。《CSS权威指南》第四版是CSS领域的重要著作,作者Eric A. ...