Once you SSH into your VPS Hosting, type the following commands to have LAMP installed:

Install Apache

yum install httpd
chkconfig httpd on
service httpd start

Install MySQL Database Server

In the last step when installing mysql, you will be asked to set a root MySQL password. You will also be asked to remove anonymous users, disallow root login remotely, and a few other questions. Answer "yes" all the way to the last question and the installation is finished.

yum install mysql-server mysql php-mysql php -y
service mysqld start
chkconfig --levels 235 mysqld on
/usr/bin/mysql_secure_installation

The LAMP installation is done. Visit your default page:

http://[IP Address] or http://[domain name]