(RESOLVED) Database server down, website might be a bit slow

Questions, comments, issues, and suggestions about the new SRRP website.

(RESOLVED) Database server down, website might be a bit slow

Postby Gen.Jersey » Wed Apr 08, 2015 1:56 am

The database server used by this site (and all my other websites) experienced a failure earlier today. The SRRP website is now using its backup database server. Unfortunately, the most recent database backup is from 4:30 AM April 7th, so any posts made in the past 21 hours are currently lost. Once the primary database server is back up and running I may be able to retrieve them, it all depends on what was going on when it crashed.

The backup server is not as powerful as the primary server, so the site might be a bit slower than usual until I can get the primary back up.

I'll be taking backups every 6 hours (instead of every 24 hours) until everything is resolved. I would do more frequent backups but don't want to take the chance of overloading the backup server.

I'll post a reply once everything is back up and running. In the meantime, continue to use the site normally. If you experience any errors, problems, or excessive slowness (pages taking more than 10 seconds to load), please post a reply to this thread with as much information as possible (precise date and time, what you were doing when the problem occurred, the exact error message if you received one, etc.).
General Jersey
SRRP Website Designer and Administrator
____________________________________________________________

Please let me know if you have any issues, questions,
or comments about the SRRP website and server.

____________________________________________________________

Add me on Steam @ [SRRP]Gen.Jersey

PM me here:
Image

____________________________________________________________

Looking to buy a real gun?

Check out WhichGun.com, my firearms reference site and buying guide.
User avatar
Gen.Jersey
SRRP Senior General
 
Posts: 681
Joined: Sat Sep 24, 2011 8:57 am
Location: New Jersey

Re: Database server down -- Website might be a bit slow

Postby Gen.Jersey » Sat Apr 11, 2015 7:46 am

We were back up and running on the primary database server for a couple of hours, but more problems began to crop up. I put the site's database back onto the backup server while I try to figure out what the heck is going on with the primary. Hopefully the site is not loading too slow right now for anyone -- for me, the backup server is noticeably slower loading webpages, but it's not too bad (2-3 seconds to load, versus < 0.5 seconds normally).

Part of the reason why it's slower is because the backup server is less powerful than the primary server, but also because the backup server is located in a different datacenter 2,500 miles away in Las Vegas. The webserver in New Jersey needs to connect to the backup database server in Las Vegas to retrieve data, instead of connecting to the primary database server just a few feet away in the same room.

Please let me know if you see any error messages or pages are taking a very long time to load (> 10 seconds).
General Jersey
SRRP Website Designer and Administrator
____________________________________________________________

Please let me know if you have any issues, questions,
or comments about the SRRP website and server.

____________________________________________________________

Add me on Steam @ [SRRP]Gen.Jersey

PM me here:
Image

____________________________________________________________

Looking to buy a real gun?

Check out WhichGun.com, my firearms reference site and buying guide.
User avatar
Gen.Jersey
SRRP Senior General
 
Posts: 681
Joined: Sat Sep 24, 2011 8:57 am
Location: New Jersey

Re: Database server down -- Website might be a bit slow

Postby Maj.Gen.NeckBBreaker » Thu Apr 16, 2015 8:05 am

Thanks for the update, I was wondering why the site was down when I tried to connect. Hope your other sites are still up as well and you will find the problems. Good luck with it!
User avatar
Maj.Gen.NeckBBreaker
SRRP General
 
Posts: 815
Joined: Fri Sep 30, 2011 5:07 am
Location: Cologne, Germany

Re: Database server down -- Website might be a bit slow

Postby Gen.Jersey » Fri Apr 24, 2015 1:27 pm

I'm going to be switching everything over to the new server soon, most likely this weekend. There should be zero downtime; it will be a seamless transfer.

In case anyone is interested in how this is done, it really is quite simple:

  • This site is running on two separate servers, the web server, and the database server.
  • The web server holds all the files (website code, images, avatars, attachments, etc.). It processes the PHP code for the site, and sends the HTML to your web browser. The database server holds user data, profiles, passwords (encrypted), and all posts.
  • Because they are two separate machines, the webserver has to connect to the database server in order to retrieve this data.

So, to get a seamless transition to a new database server, you do the following:

  1. Set up and configure the new database server.
  2. Use a process called replication to "sync" the old server with the new server. The old server is called the master, the new is the slave. This means that anytime data is added or changed (posting a new topic, or editing a post, for example), the master updates its database. It then automatically connects to the slave, and tells the slave to perform the same operations to its own copy of the database.
  3. Once replication is set up and confirmed to be working properly, you adjust the site's configuration on the webserver, telling it to connect to the slave (new) server.
  4. Now the site is using the new database server, and nobody using the site would even notice a thing.
  5. You can then stop the replication, or you can reverse the roles and set the slave to be the master. The old master is now the slave, and can be used as a backup.

It's a very simple but very effective solution. When you have a massive website with thousands of hits every second, and/or you need to be 110% sure that no data is lost (a banking system, stock trading system, or the system that logs/records the instruments for a nuclear power plant, for example), things can get much more complex, but the overall idea remains the same.
General Jersey
SRRP Website Designer and Administrator
____________________________________________________________

Please let me know if you have any issues, questions,
or comments about the SRRP website and server.

____________________________________________________________

Add me on Steam @ [SRRP]Gen.Jersey

PM me here:
Image

____________________________________________________________

Looking to buy a real gun?

Check out WhichGun.com, my firearms reference site and buying guide.
User avatar
Gen.Jersey
SRRP Senior General
 
Posts: 681
Joined: Sat Sep 24, 2011 8:57 am
Location: New Jersey

Re: Database server down -- Website might be a bit slow

Postby Gen.Jersey » Wed Apr 29, 2015 2:30 am

We're up and running on a brand new database server located in the same datacenter as the web server. Now the site doesn't need to make a trip from New Jersey to Las Vegas and back again to retrieve its data, so things should be just as fast as they used to be.

Please let me know if you experience any issues or receive any error messages.
General Jersey
SRRP Website Designer and Administrator
____________________________________________________________

Please let me know if you have any issues, questions,
or comments about the SRRP website and server.

____________________________________________________________

Add me on Steam @ [SRRP]Gen.Jersey

PM me here:
Image

____________________________________________________________

Looking to buy a real gun?

Check out WhichGun.com, my firearms reference site and buying guide.
User avatar
Gen.Jersey
SRRP Senior General
 
Posts: 681
Joined: Sat Sep 24, 2011 8:57 am
Location: New Jersey

Re: (RESOLVED) Database server down, website might be a bit

Postby Gen.Hannibal » Wed Apr 29, 2015 9:15 am

Thank you for your time
Gen.Hannibal
SRRP Senior General
 
Posts: 1131
Joined: Thu Nov 17, 2011 12:47 am
Location: Louisville, Kentucky

Re: (RESOLVED) Database server down, website might be a bit

Postby Maj.Gen.Greenwood (SRO) » Wed Apr 29, 2015 3:30 pm

All good, Great job Jersey!
Major General Greenwood
Senior Recruiting Officer
___________________________________________________________________________________
Swamp Rat's Rat Pack Member; 2009 - Current
SRRP Recruitment; 2011 - Current
SRRP SRO; 2014 - Current
SRRP General; 2014 - Current
___________________________________________________________________________________
Want to join the scrim team? PM Me.
User avatar
Maj.Gen.Greenwood (SRO)
SRRP General
 
Posts: 838
Joined: Wed Nov 02, 2011 7:40 pm
Location: Utah


Return to Website Discussion & Support

Who is online

Users browsing this forum: No registered users and 1 guest

cron