Is the cluster data still available to the API node?
mysql-api-1# mysql -u root
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 258552 to server version: 4.1.9-max
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql> use foo;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> select * from test3;
+------+
| i |
+------+
| 54 |
| 91 |
| 79 |
| 52 |
| 92 |
| 20 |
| 18 |
| 84 |
| 49 |
| 22 |
+------+
10 rows in set (0.02 sec)
Now, plug the disconnected storage node back into the network. It will attempt to rejoin the cluster, but probably will be shutdown by the management server, and something similar to the following will appear in the error log (/var/lib/mysql-cluster/mdb_2_error.log):
Date/Time: Saturday 12 February 2005 - 12:46:21
Type of error: error
Message: Arbitrator shutdown
Fault ID: 2305
Problem data: Arbitrator decided to shutdown this node
Object of reference: QMGR (Line: 3796) 0x0000000a
ProgramName: /usr/local/mysql/bin/ndbd
ProcessID: 1185
TraceFile: /var/lib/mysql-cluster/ndb_2_trace.log.3
***EOM***
Restart the ndb storage engine process on that node and verify that it rejoins the cluster properly:
mysql-ndb-1# /usr/local/mysql/bin/ndbd
ndb_mgm> show
Cluster Configuration
---------------------
[ndbd(NDB)] 2 node(s)
id=2 @192.168.0.33 (Version: 4.1.9, Nodegroup: 0)
id=3 @192.168.0.34 (Version: 4.1.9, Nodegroup: 0, Master)
[ndb_mgmd(MGM)] 1 node(s)
id=1 @192.168.0.32 (Version: 4.1.9)
[mysqld(API)] 4 node(s)
id=4 (Version: 4.1.9)
id=5 (Version: 4.1.9)
id=6 @192.168.0.35 (Version: 4.1.9)
id=7 (not connected, accepting connect from any host)
Miscellaneous
- Remember that in order for SQL data to be stored (replicated) on the cluster, database tables must be created specifying engine=NDBCLUSTER (as shown the in the examples above). It is possible to use this mechanism to specify different storage engines for different tables within the same database, depending on individual performance and reliability requirements. Non-critical database tables need not be stored on the cluster.
- Occasionally, after abnormal cluster node termination (for example, a system crash) we see "hung" connections, and upon restart the failed node is unable to join the cluster. In this case, the session should be manually cleared on the management console using the command, "purge stale sessions":
ndb_mgm> purge stale sessions
Purged sessions with node id's: 3
ndb_mgm>
Please direct questions, comments, and suggestions regarding this document to consult@lod.com
LOD Communications, Inc.
(800) 959-6641
http://www.lod.com
