Linux / Open Source Forum: June 2010

Wednesday, June 30, 2010

MYSQL - REPLICATION Master-Slave

MYSQL - Replication Master-Slave

*Using this we can replicate our mysql database server to one or more slave
* This is asynchronous , slave server need not be be connected permenantly to the master server.
* all the writes and updates occured on master server. Reads can takes place in slave box

Mysql-Master server:-
* must enable server unique ID and master bin log, Binary login is sending the master data to slave. so this is the main important thing.
* For configuring Master first we need to shutdown mysql-server and the edit my.cnf file. Details are included under [mysqld] section.
* To enable binary log using log-bin represented the name as "mysql-bin" or db-bin and serve id can mention as server-id=1(positive integer),default value is 0.
after that server needs to be reboot.

Mysql-Slave Server
* first need to specify the server ID

Replication user creation:[On Master Box]
*For replication slave need to connect to master box using a user name and password. So we need to create a user and password on Master box and grand privilege for the slave box on master box.

Setting the Master configuration on Slave
To make our slave box communicate with master box we need to setup some connection information using ' CHANGE MASTER TO"

Here I am going to describe how to database replication using Mysql.
Mysql replication is a asynchronous data transfer(ie, database on the master server are not immediately replicated to the database on the slave server).
This Replication Type is Master-Slave
A. MASTER SET UP

1. Edit /etc/my.cnf and enter the following details under mysqld
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
server-id = 1
log-bin=db-bin
relay-log=db-relay-bin

2.enter into the mysql database

mysql> GRANT REPLICATION SLAVE ON *.* TO 'replication'@'10.10.2.24' IDENTIFIED BY 'password';

Suppose that your slave server is '10.10.2.24' and that you want to create an account with a user name of 'replication' such that slave servers can use the account to access the master server using a password of 'password'.

3.mysql> FLUSH TABLES WITH READ LOCK;
Flush all the tables and block write statements

4.mysql> SHOW MASTER STATUS;
+---------------+----------+--------------+------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+---------------+----------+--------------+------------------+
| db-bin.000005 | 792 | | |
+---------------+----------+--------------+------------------+

OR
mysql> SHOW MASTER STATUS \G;
*************************** 1. row ***************************
File: db-bin.000005
Position: 792
Binlog_Do_DB:
Binlog_Ignore_DB:
1 row in set (0.00 sec)

So MASTER Side configuration has over ..

B SLAVE SERVER CONFIGURATION

1. Edit the file /etc/my.cnf and enter the details as follows
[mysqld]
server-id=2
master-host = 10.10.3.21
master-user = replication
master-password = password
master-port = 3306

10.10.3.21 is the MASTER server IP.

These information will be stored to /var/lib/mysql/master.info

Enter into the Mysql server
2. mysql> stop slave;
3. mysql> reset slave;
4. mysql> CHANGE MASTER TO MASTER_HOST='10.10.3.21', MASTER_USER='replication', MASTER_PASSWORD='password', MASTER_LOG_FILE='db-bin.000005';

Here 10.10.3.21 is your MASTER Server IP 'replication' is the master user 'password' is the password for your master server and 'db-bin.000005' is the Master log file this one you wll get while running the command mysql> SHOW SLAVE STATUS \G; in Master server

5. mysql> START SLAVE;

6. mysql> show slave status \G;
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 10.10.3.21
Master_User: replication
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: db-bin.000005
Read_Master_Log_Pos: 792
Relay_Log_File: mysqld-relay-bin.000002
Relay_Log_Pos: 934
Relay_Master_Log_File: db-bin.000005
Slave_IO_Running: Yes
Slave_SQL_Running: Yes

Last_Errno: 0

Exec_Master_Log_Pos: 792
Relay_Log_Space: 1090
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No

Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:

The process:- In Master Server(Alter,Delete) all details execute and 'I/O Thread' save to a log file called Binary Log(ie,db-bin.000005) And in slave(only Read) read the changes from Binary log to relay log and update to slave SQL Thread.

You can verify by checking the Seconds_Behind_Master details and Slave_IO_Running and Slave_SQL_Running

REPLICATION ERRORS and FIX

1. 'Last_IO_Error' => 'Got a packet bigger than > \'max_allowed_packet\' bytes'

This is due to 'max_allowed_packet' size. Increasing max_allowed_packet will fix the issue

Comments are welcomed

2. > show slave status \G
Slave_IO_State:
Master_Host: 10.10.210.198
Master_User: slave
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000916
Read_Master_Log_Pos: 863345946
Relay_Log_File: relay-bin.000012
Relay_Log_Pos: 374486719
Relay_Master_Log_File: mysql-bin.000916
......
;;;;;;;
;;;;;;;;

Last_Errno: 1064
Last_Error: Error 'You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near FrT;@28|$9|main_init$1|1$11|cookies_tmp%0|$10|mobipocket$1|0$7|cookies%0|$11|la' at line 3' on query. Default database: 'ebooks'. Query: 'REPLACE INTO session
(module, id, contents, timestamp) VALUES
('main', 'ee1db08d8f93bc69bbe0a1dd9a9cc301', 'FrT;@28|$9|main_init$1|1$11|cookies_tmp%0|$10|mobipocket$1|0$7|cookies%0|$11|last_viewed@5|%6|$5|brand$5|model$3|sku$26|Bankes, Tim, Hatter, David$82|Lotus Notes and Domino R6 Application Development Exam Cram 2 (Exam 620, 621, 622)$10|0768661498%6|$5|brand$5|model$3|sku$0|$28|CULTURE OF HUMAN TUMOR CELLS$10|0471270628%6|$5|brand$5|model$3|sku$11|Hwang, Andy$74|Ready-to-Eat Foods: Microbial Concerns and Control Measures (Adobe Reader)$13|9781420068634%6|$5|brand$5|model$3|sku$0|$45|TRANSATLANTIC VOYAGES & SOCIOL (Adobe Reader)$13|9781409405009%6|$5|brand$5|model$3|sku$0|$51|Practical Manual of Thyroid and Parathyroid Disea
Skip_Counter: 0
Exec_Master_Log_Pos: 374486582
Relay_Log_Space: 863347015
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: NULL

FIX:
(none)> STOP SLAVE;
(none)> CHANGE MASTER TO master_log_file='mysql-bin.000916', master_log_pos=863345946;

[Here master_log_file and master_log_pos will get from your slave status .]

(none)> slave start;

3. Replication rebuild [ Rebuild from Scratch ]
Once we can not fix the replication error in any of our known ways we can do the last fix as Rebuild replication from Scratch

Please note down the following steps

on Master Server

1. flush tables with read lock; --> no write operation on databases only read operation will be available
2. reset master; ---> master will reset
3. show master status; ---> check for the out put

+------------------+-----------+
| File | Position |
+------------------+-----------+
| mysql-bin.000001 | 98 |
+------------------+-----------+

4. quit from mysql

5. stop mysql server service

6. make a copy of all files from /var/db/mysql to a local disk at first. If you copy over network it will take more time to complete

7. start mysql-server

On the slave box

1. Stop mysql-server service
2. move binary files /var/db/mysql/ from the master to /var/db/mysql/

3. chown -R mysql /var/db/mysql/

4. Start mysql-server service

enter into Mysql
5. STOP SLAVE;
6. CHANGE MASTER TO master_log_file='mysql-bin.000001', master_log_pos=98;
7. slave start;

Finish thats all please confirm whether your slave is running fine

Tuesday, June 29, 2010

CPANEL

Cpanel notes:-

To check the current cpanel version please run the following command
# /usr/local/cpanel/cpanel -V

Directory -->/usr/local/cpanel/

3rdparty/ tool like fantastico, mailman are located here.

addons/ --> phpBB forums are here
base/ --> phpmyadmin, squirrelmail, skins, webmail etc located here
cgi-sys/ --> cgi files like cgiemail, formmail.cgi and formmail.pl etc
logs/ --> cpanel access log and error log
whostmgr/ ---> whm related files

WHM
Directory -> var/cpanel/
bandwidth/ – rrd files of domains
users/ or resellers --> reseller account details

10-07-2010
HOW TO CHANGE THE CPANEL THEME FOR ALL ACCOUNT TOGETHER WITH SAME THEME

The cPanel theme for an account is in the directory /var/cpanel/users/username file in the format CPMOD=x3... where x3 is the theme name. In WHM we can change the theme for accounts one at a time. But here we need to change all the account theme to X3 or etc theme for that please do the following steps
1. Execute the command
for i in `ls -la /var/cpanel/users/ | awk ‘{print $9}’ | sed ‘1,3 d’`; do sed -i “/CPMOD/d” $i; echo “CPMOD=x3″ >> $i; done;

2. As the files are updated manually, we need to execute /scripts/updateuserdomains to rebuild the cache.

12-07-2010:-
TO COPY CPANEL /backup folder to your local computer
The best command to use this SFTP as root login details
[ajeesh@tech24 ~]$ sftp root@

Then > cd /
> get backup

3. WHM/Cpanel Email

A. MAX EMAIL SEND PER HOUR
Server Configuration >> Tweak Settings we can see Mail section-> The maximum each domain can send out per hour (0 is unlimited)
B. If you like to give some extra no of mail sending permission to selected domains
1. [~]# vi /var/cpanel/maxemails
yourdomainname=100
2. After that you need to run the following command
/scripts/build_maxemails_config

APACHE

Notes:-
configuration --> /etc/httpd/conf/httpd.conf

/usr/local/apache/domlogs/sutename -> domain log files are stored here
libexec --> shared object(.so) files are stored eg libphp4.so,mod_rewrite.so

Restart commands
/etc/rc.d/init.d/httpd restart
/scripts/restartsrv_httpd -> cpanel script restart option

APACHE CONNECTIONs
To check the number of connections to port 80:
netstat -alntp | grep :80 | wc -l

To List the remote IPs connecting to your server on port 80:
netstat -alntp | grep :80 | awk ‘{print $5}’ | cut -d: -f1 | sort

To List the uniq remote IPs and the number of connections from particular IP:
netstat -alntp | grep :80 | awk ‘{print $5}’ | cut -d: -f1 | sort | uniq -c | sort -n

To find out apache down time

To find the location for httpd.pid
# ll -d /usr/local/apache/logs/httpd.pid

How to find out when apache last restarted
# ll /usr/local/apache/logs/httpd.pid

Apache config test, for test whether your apache settings are correct before restarting
#apachectl configtest

Thursday, June 24, 2010

Kmail Recovery

Kmail Recovery

Once we took the back of the /home/username directory , using the following commands you can recover your kmail without configuring the kmail settings.

These are should be done after closing the kmail application in your system.

1. scp -r [source server/pc]/home/ajeesh/.kde/share/apps/ [destination]ajeesh@10.10.2.24:/home/ajeesh/.kde/share/apps/kmail2
if you use root permission again we need to change the permission by using
chown -R ajeesh. .kde/share/apps/kmail2/

in destination server/PC
2. apps]$ mv kmail bak_kmail
3. apps]$ mv kmail2 kmail

After that
in Source server
4. scp -r [source server]/.kde/share/config/kmail* [destination server]kde/share/config/

we have to copy mail.eventsrc, kmail.notifyrc , kmailrc, kmailsnippetrc files

Now you have done the Kmail recovery without configuring your mail settings again.

cPanel Great Success Appearing on new account creation.

While accessing your website , It will show the Cpanel page instead of your correct web page

"cPanel Great Success Appearing on new account creation."

While restarting the httpd we will get the error as

]# service httpd restart
httpd: Syntax error on line no of /usr/local/apache/conf/httpd.conf: Cannot load /usr/local/apache/modules/mod_dav_svn.so into server: /usr/local/apache/modules/mod_dav_svn.so: undefined symbol: dav_xml_get_cdata

Solution
1. We have to take a back of httpd config file
#] cp /etc/httpd/conf/httpd.conf /etc/httpd/conf/httpd.conf.bkp

Then

# /scripts/rebuildhttpdconf
Built /usr/local/apache/conf/httpd.conf OK
# service httpd restart

Wednesday, June 23, 2010

FFMPEG Installation

FFMPEG Installation

http://ajeeshbkannan.blogspot.com/search?q=ffmpeg

There is an easy script , we can install all the packages coming with this video streaming software

http://www.ffmpeginstaller.com/download/

This will cover all the packages such as ->ffmpeg * Mplayer * A52decoder
* Amrnb and Amrwb * All codecs * facc and faad2 * Ruby * flvtool2 * ffmpeg-php * Mplayer and Mencoder * Lame * Libogg * Libvorbis * Libtheora * Libwmf * opencoreamr * vorbistools * X264 * MP4Box * Xvidcore

I have tried this script and my fedora machine needed a re installation. Do this scripts at your own risks.

=====================
http://sourceforge.net/projects/ffmpeginstall/
#cd /usr/local/src
#wget http://sourceforge.net/projects/ffmpeginstall/files/ffmpeginstall.3.x/ffmpeginstall.3.2.1/ffmpeginstall.3.2.1.tar.gz/download
Servertechs#tar -xvzf ffmpeginstall.3.2.1.tar.gz

#cd ffmpeginstall.3.2.1

#./install.sh

Tuesday, June 22, 2010

Domain Transfer From Remote Server

In the Source server using cpanel user name and password first we have to create a backup for particular domain in -> Cpanel -> Backup
(This will be in the form of cpanelbackup.username.tar.gz)
2nd
From our server the destination server, make the ftp connection to the source server

ftp>get cpanelbackup.username.tar.gz

after downloading this file to our server

#mv cpanelbackup.username.tar.gz /home
#script/restorepkg

Last we have to change the ownership to the file and folder (ie, chown username.username -R *)

Next Idea is from Mr.Dan Thopson
After correcting the file permission even if particular is owned by root. So in WHM area the domain we just transferred wont be visible. So to fix that:
1. Log into WHM
2. Click on Reseller Center
3. Under "Reseller Modifications", select reseller account from the drop down list
4. Click on Change ownership of multiple accounts
5. Scroll down to find the account we just moved over, and check the box next to it, Then click the "Change owner of selected accounts" at the bottom of the page
He is saying " As far as I know this is the quickest way to do it, there's no simple way to do it via SSH ".

Unable to remove files within ./cache/. Please check directory

Whenever you are trying to change something on the forum /board/ like a members settings or the boards settings ou will encounter the error such as " Unable to remove files within ./../cache/. Please check directory permissions."
Like this situation we can do the following steps to avoid these errors.
1. Set folder permission for the cache to 755 (777 on some servers)
2. set all file inside the cache to 644 (666 on some servers).

Sunday, June 20, 2010

vzquota : (warning) Incorrect quota shutdown for id , recalculating disk usage

In VPS VEID is not restarting and VEID is DOWN / Locked
Error:-
[root@server ~]# vzctl start VEID
Starting VE ...
vzquota : (warning) Incorrect quota shutdown for id VEID, recalculating disk usage
vzquota : (error) quota check : lstat `sess_f3d432f92a5366e68dd74bce1aa437b7': No such file or directory
vzquota on failed [1]
[1]+ Killed vzctl start VEID

In Open VZ server particular VEID is down and we can not manually start that VEID.
[root@server~]# vzctl start 9894802
Starting VE ...
vzquota : (warning) Incorrect quota shutdown for id VEID, recalculating disk usage

[1]+ Stopped vzctl start VEID

[root@server~]# vzctl start VEID
VE already locked

This situation first we have to check the process ID for the locked VEID and kill that process.
[root@server ~]# vi /vz/lock/VEID.lck
we will get the process ID and
[root@server ~]# kill -9 PID

[root@server ~]# vzquota off VEID
[root@server ~]# vzquota on VEID
[root@server ~]# vzctl start VEID

2. [root@server]# vzctl enter 9896XX
enter into CT 989XXX failed
Unable to open pty: No such file or directory

Solution
# vzctl exec 9896556 /sbin/MAKEDEV pty
# vzctl exec 9896556 /sbin/MAKEDEV tty

After that restart your VE

Or try
# vzctl exec 9896xx update-rc.d -f udev remove

Saturday, June 19, 2010

TO SET TIME AND DATE IN LINUX

TO SET TIME AND DATE In LINUX
You must login as root
[root@tech24 ajeesh]# date -s "19 Jun 2010 01:09:00"
Sat Jun 19 01:09:00 IST 2010
[root@tech24 ajeesh]# date
Sat Jun 19 01:11:07 IST 2010

Friday, June 18, 2010

Downgrade MySQL from MySQL 5

Here we are discussing how to downgrade mysql version 5 to version 4.1
First we can take a full back of our existing MySQL database

1. ]$ mysqldump --compatible=mysql40 --all-databases > databases.sql

]$ rpm -qa | grep -i mysql-
2. Then remove /var/lib/mysql:]$ rm -Rfv /var/lib/mysql
3. Then edit /var/cpanel/cpanel.config and set as -> mysql-version=4.1
4. Make sure that the skip file isn't there:
rm -fv /etc/mysqlupdisable
5. Then install MySQL ----> /scripts/mysqlup --force
6. Then update the perl module --> /scripts/perlinstaller --force Bundle::DBD::mysql
7. Then recompile apache/php --> /scripts/easyapche
8. /scripts/upcp --force

Last restore the old MySQL database
]# mysql database < databases.sql

Thursday, June 17, 2010

FANTASTICO De Luxe

FANTASTICO Cannot install here, there's already a script installed in the root

If you remove wordpress or Joomla file manually in your server and latter if you install these surly you will encountered a error. This section i will show you how you can avoid this.

1. Delete the files fantversion.php and fantastico_filelist.txt in the public_html directory
2. Edit the file /home/user name/.fantasticodata/installed_in_root.php and remove the text
$installed_in_root["yourdomain.com"] = 1;
3. Remove the file:
/home/username/.fantasticodata/WordPress/yourdomain.com|

**Idea got from Dan Thompson

2. Fantastico is not installed at the default location /usr/local/cpanel/3rdparty/fantastico. Either move the Fantastico directory from it's current location to /usr/local/cpanel/3rdparty/fantastico OR enable ioncube loaders in WHM -> Tweak settings.

first need to check ioncube enabled in WHM->Tweak Settings
a.root@hyrule [~]# /scripts/phpextensionmgr install IonCubeLoader
And restart cpanel and retry, again the same issue persist
b. /scripts/makecpphp
And restart cpanel and retry, again the same issue persist
c. cpanel update with /scripts/upcp --force
==============
Fantastico Installation

Fantastico De Luxe is a free scripts installer with a variety of over 40 PHP-based web applications.
In Fantastico there is a list of the most popular and widely used Third party open source software. With just a few clicks of the mouse any of the applications can be installed and set up on a web site.

Please do the following steps for installing Fantastico on your cpanel server
cd /usr/local/cpanel/whostmgr/docroot/cgi
wget -N http://files.betaservant.com/files/free/fantastico_whm_admin.tgz
another alternate location, wget -N http://netenberg.com/files/free/fantastico_whm_admin.tgz
tar -xzpf fantastico_whm_admin.tgz
rm -rf fantastico_whm_admin.tgz
Now go here:
WHM -> Plugins-> Fantastico De Luxe WHM Admin install using the default setting

After the installation complete go to setting

After installation go to" Settings " do the settings, * are essential for a proper functioning of Fantastico installations.
PHPsuexec (*) = check whether you have enabled suphp on your server or not

/usr/local/cpanel/bin/rebuild_phpconf --current
Available handlers: suphp dso cgi none
DEFAULT PHP: 5
PHP4 SAPI: suphp
PHP5 SAPI: suphp
SUEXEC: enabled
This output shows that Suphp is enableb on your server

After dowing this Fantastico is not enabled on your Cpanel you need to check whether tick mark is there on Main >> Packages >> Feature Manager

Wednesday, June 16, 2010

GOOGLE APPS SETTINGS for Mail

Hi this section i would like to introduce how to setup Mail Application

After setup a primary master account with google[http://www.google.com/apps/intl/en/business/index.html]. After that setup mail application for your domain.

Next we have to edit the DNS zone file for the MX record
We have to delete the existing MX record before entering the new MX record. And enter the following MX server address.

ASPMX.L.GOOGLE.COM.
ALT2.ASPMX.L.GOOGLE.COM.
ASPMX3.GOOGLEMAIL.COM.
ASPMX5.GOOGLEMAIL.COM.

You may not need to enter the priority values.(default it will take 1,5,10,10]

MYSQL

MYSQL

Main Configuration file is in /etc/my.cnf

Password stored in /root/.my.cnf

Service Start and stop
/etc/rc.d/init.d/mysqld start/stop/restart

How to Reset Root Password
ISSUE:- [root@tech23 ~]# mysql
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)
1. service mysql stop
2. enter mysql command prompt using mysqld_safe --skip-grant-tables $
3. mysql>UPDATE mysql.user SET Password=PASSWORD('password') WHERE User='root';
4. Quit from Mysql and run the command " service mysql restart "

[root@tech23 ~]# service mysqld stop
[root@tech23 ~]# service mysqld status
mysqld is stopped
[root@tech23 ~]# mysqld_safe --skip-grant-tables &
[2] 18962
[root@tech23 ~]# 110222 17:19:53 mysqld_safe Logging to '/var/log/mysqld.log'.
110222 17:19:53 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql

[root@tech23 ~]# mysql -u root
mysql> use mysql;
mysql> update user set password=PASSWORD("ajeesh-your new password") where User='root';
Query OK, 1 row affected (0.02 sec)
Rows matched: 4 Changed: 1 Warnings: 0

mysql> flush privileges;
Query OK, 0 rows affected (0.06 sec)

Thats all your

GRANT PRIVILEGES

mysql> GRANT ALL PRIVILEGES ON DBname.* TO user@ IDENTIFIED BY 'password';

mysql> flush privileges;
What all privileges that user has got?
mysql> show grants for user@localhost;

How to Find Out MYSQL Users
1. Enter into the Mysql database
2. Mysql> use mysql;
3. mysql> select distinct user from user;

If Any Table In the Databases are Corrupted
1. cd /var/lib/mysql/
2. From there run the command myisamchk -r *.MYI

Max_connection
mysql> show variables like "%max_connection%";
OR
in linux command prompt
#mysqladmin -p'password' ex|grep Threads_connected

DO SECTION

MYSQL Database Bachup
1. $ mysqldump -h host -u user -p database > backup.sql --> to create the backup
2. mysql -h host -u user -p -D database < backup.sql --> to restore the database from backup

Increase the No Of Mysql Connection
mysql> show processlist;
#vi /etc/my.cnf
max_connections= increase the no
# service mysql restart

ERROR SECTION

A Mysql Is Not Getting Started
1. vi /etc/my.cnf add line ' innodb_force_recovery = 4 '
2. comment out skip-innodb like #skip-innodb
3. Restart

ADDING REMOTE MYSQL CONNECTIONS
1. Go to WHM Main >> SQL Services >> Additional MySQL Access Hosts, Here we have add the IP address which we needs to access the database
2. Go to Cpanel under 'mysql' section, click on 'remote mysql' and Type that Remote Host IP

MYSQL DROP , DELETE , TRUNCATE TABLEs (01-12-2010)
Differences
DROP -> Delete the table from your database
Syntax : drop table friends(table name);

TRUNCATE -> Delete the contents from the table and if you have any auto increment it will reset to 0(from start again). It is very safe command for deleting contents from a table.
Syntax :
mysql> truncate table friends;
Query OK, 0 rows affected (0.00 sec)

DELETE -> Delete all the contents from table and will not reset your auto increment to 0 it will start again from the next value
Syntax : delete from friends(table name);

Linux / Open Source Forum